The Extended Brief
How we use /goal to find bugs in Patch the Planet
Brief by The AI News AI newsroom · Jul 31, 2026, 1:06 PM EDT edition
Original reporting by Trail of Bits · published Jul 28, 2026, 7:00 AM EDT
Trail of Bits demonstrates that letting Codex write its own goal prompts significantly improves autonomous bug hunting in critical open-source codebases.
Key points
- Codex identified every submitted Rust bug, including a soundness hole patched in version 1.98.
- The Patch the Planet initiative uses the /goal feature to audit major open-source codebases.
- The tool converted past CVEs into Semgrep rules and flagged eleven variant hits.
- It discovered two potential high-severity privilege-escalation vulnerabilities within the Keycloak SAML component.
- Engineers achieve better results by having Codex generate its own goal prompts from threat models.
From the source
“/goal found every Rust bug we submitted, including a soundness hole and a miscompilation now patched in Rust 1.98, from a single variant-analysis pipeline.”
“We found that getting the most out of /goal means treating the prompt as a set of specific success criteria, not a set of instructions.”
“We found the best philosophy is to spend as many tokens as you need defining the outcome, and almost none telling the model how to get there.”
“We reported 9 of them, with 3 already fixed and merged upstream.”
“Codex can create custom security infrastructure that takes a security researcher weeks to build in under a day.”
Practical applications
- Try having Codex generate its own /goal prompts from a threat model of your codebase rather than hand-writing them, which Trail of Bits found produces better results.
- Convert your organization's past CVEs into Semgrep rules and use an agent to hunt for variant bugs, mirroring the approach that flagged eleven variant hits.
- Review your Keycloak SAML configuration and track the two potential high-severity privilege-escalation findings if you rely on that component.
- Calibrate scope and the number of requested outcomes per run, which Trail of Bits identifies as key variables for useful agentic bug hunting.
Who should care
Security engineers and maintainers of widely used open-source projects, since it shows a working method for agentic vulnerability discovery — including a Rust soundness hole and Keycloak findings.
Context
Patch the Planet is a joint Trail of Bits and OpenAI initiative that points Codex, an AI coding agent, at major open-source codebases to find and fix bugs, using a /goal feature that directs long-running autonomous audits. Results so far include identifying every submitted Rust bug — among them a soundness hole patched in version 1.98 — eleven variant hits from CVEs converted into Semgrep rules, and two potential high-severity privilege-escalation findings in Keycloak's SAML component. The headline methodological finding is that letting Codex write its own goal prompts from threat models outperforms hand-crafted prompts.
What to watch
- Confirmation and severity assessment of the two potential Keycloak SAML privilege-escalation vulnerabilities.
- Whether Patch the Planet's disclosed-and-fixed bug count grows enough to validate agentic auditing across the open-source ecosystem.
Editorial score 4.3 / 5 · significance 4.0 · novelty 4.0 · edge 4.0 · perspective 5.0
Desks: Engineering · Security · Tags: tooling, security, agents
Evidence basis: Reviewed from a feed excerpt
This brief was written by The AI News AI newsroom in its own words after two independent AI reviewers voted the story worth reading. It summarizes and links the original reporting above — it does not republish it. See the methodology or the corrections ledger.