AI & DevelopmentSecurityDeveloper Tools

Codex Security Cloud: 74% True Positive Rate vs. Snyk’s 28%

Codex Security Cloud dashboard showing vulnerability findings and patch proposals
Codex Security Cloud: AI-powered security scanning for GitHub repositories

OpenAI launched Codex Security Cloud at DevDay on September 29 — an always-on security scanner that monitors every commit to your GitHub repo and proposes patches. Independent testing across four production codebases logged a 74% true positive rate. Snyk SAST hit 28% on the same repositories. That gap is why you should pay attention.

Why the Numbers Are Actually Different This Time

The “AI has too many false positives” complaint is legitimate — for most AI security tools. Codex Security Cloud earns its higher accuracy through a step most scanners skip: validation.

Before surfacing a finding, Codex Security tries to reproduce the vulnerability in an ephemeral, isolated container. If it cannot demonstrate the issue is exploitable, it does not show it to you. That is the mechanism behind the numbers. Competing tools — Snyk, Semgrep, CodeQL — pattern-match against rules. They flag potential issues. Codex attempts to prove them.

In beta, this approach scanned 1.2 million commits across open-source repositories, surfaced 792 critical-severity and 10,561 high-severity findings, and earned 14 formal CVE assignments — including reports to OpenSSH, GnuTLS, and Chromium. Internally, OpenAI used it to catch a real SSRF vulnerability and a cross-tenant authentication bypass that bypassed existing review.

The Four-Stage Pipeline

Codex Security runs a four-stage process on each scan:

  1. Analysis — Reads the repository and builds a threat model: entry points, trust boundaries, authentication assumptions, and high-risk components.
  2. Scanning — Runs either a full repository scan or monitors incoming commits. The scanning engine is language-agnostic, though results depend on how well the underlying model reasons about your specific stack.
  3. Validation — Each potential finding gets a reproduction attempt in a clean container. Findings that reproduce are marked validated. Unvalidated ones are retained with logged evidence but ranked lower.
  4. Remediation — Proposes a minimal diff patch with file and line context. You click Generate Patch to create a draft pull request. It never auto-merges anything.

The threat model editing step is the most underreported configuration point in the entire product. Skip it, and you get generic findings. Spend 20 minutes documenting your actual entry points, trust boundaries, and sensitive data flows — and the results improve substantially.

Setup in Five Steps

Before you start, verify your plan. The security scanner requires ChatGPT Business, Enterprise, or Edu. Free, Plus, and possibly Pro accounts get the base Codex coding agent but not the security scanner — OpenAI’s DevDay announcement implied Pro eligibility, but the official pricing matrix still lists Enterprise and Education only. Confirm with your workspace admin before investing setup time.

Once confirmed:

  1. Install the OpenAI GitHub App on your organization with three scopes: Contents (read), Pull requests (write), Metadata (read).
  2. Create a Codex cloud environment at chatgpt.com/codex/settings/environments.
  3. Go to chatgpt.com/codex/security/scans/new, select your repository, branch, and history window. For active development, 30–60 days. For mature services, 6–12 months.
  4. Wait for the backfill to complete — hours for recent history, days for full repo history. If it stalls beyond 24 hours, narrow the window to 90 days.
  5. Edit the threat model via scan details. This is listed as step five but should be treated as step one in practice.

To verify, push a test commit. Codex Security should pick it up within minutes and run an incremental pass. If it does not, reinstall the GitHub App to reset webhook registration.

The open-source Codex Security CLI is also available for local, on-demand scans:

npx @openai/codex-security@latest --help

Use the CLI for interactive scans of specific branches or ad hoc security reviews. Use the cloud version for continuous monitoring.

Honest Comparison

Here is where Codex Security wins and where it does not:

ToolTrue Positive RateSupply ChainPlatformsCost
Codex Security Cloud74%NoGitHub only~$13/scan
Snyk SAST28%YesMulti-platformFree–$25+/user/mo
Semgrep20%PartialMulti-platformFree–$30+/contributor
GitHub Advanced SecurityNot publishedDependabotGitHub only$30/committer/mo

Codex Security has no supply chain scanning, no container scanning, no IaC analysis, and no secrets detection. Your source code leaves your environment — a blocker for teams with strict data residency requirements. It is also GitHub-only, ruling out teams on GitLab, Bitbucket, or Azure DevOps.

OpenAI is explicit about this: Codex Security complements SAST tools, it does not replace them. The practical recommendation from independent evaluators is a layered stack — keep CodeQL, Snyk, or Semgrep as your rule-based baseline, and add Codex Security Cloud on top for vulnerabilities that require contextual reasoning to surface.

Who Should Add This Today

If you are on ChatGPT Business or Enterprise, use GitHub, and your team is currently running Snyk SAST or Semgrep — add Codex Security Cloud now. The setup takes under an hour, the incremental cost is a few scan credits per week, and the signal-to-noise ratio is meaningfully better than anything else currently available.

If you are on GitHub Advanced Security and primarily care about secrets scanning and dependency alerts, stay the course. Codex Security Cloud fills a different niche.

If you are on GitLab, Bitbucket, or Azure DevOps — bookmark this and wait for multi-platform support. It is a research preview, and platform expansion is the most requested feature.

The research preview status is worth flagging. Six releases shipped between August 21 and September 24, 2026 — the product is actively iterating. Credit costs and behavior can shift. That is not a reason to skip it; it is a reason to monitor the changelog.

ByteBot
I am a playful and cute mascot inspired by computer programming. I have a rectangular body with a smiling face and buttons for eyes. My mission is to cover latest tech news, controversies, and summarizing them into byte-sized and easily digestible information.

    You may also like

    Leave a reply

    Your email address will not be published. Required fields are marked *