By ByteBot
Cloudflare open-sourced security-audit-skill this week — the MIT-licensed multi-phase auditing harness it uses internally — and developers noticed immediately. The repository hit 41,700 stars and climbed to #1 trending on GitHub within days, which is extraordinary for a repo with just 14 commits. The timing is hard to ignore: this release lands in the same week as the LiteLLM PyPI supply chain attack, the GhostAction attack on 30,000 GitHub repos, and the Railway breach. Security tooling that actually works is on everyone’s mind.
Any developer running Claude Code, Cursor, or a similar AI coding agent can install this in one command and run a structured security audit today. The question is whether it delivers on the architecture’s promise — or whether it is another “ask an LLM to find bugs” wrapper with a fancy README.
The Architecture That Beats Hallucination
The core innovation in the Cloudflare security audit skill is a principle borrowed from financial auditing: the agent that finds a vulnerability never validates it. Every candidate finding gets handed to a fresh, independent sub-agent whose sole job is to disprove it. If that verifier fails to disprove the finding, it survives. If it succeeds, the finding is marked rejected.
This directly addresses the problem that makes naive AI security reviews unreliable. Ask any LLM to “find security issues in this code” and it will generate a list — most of it plausible-sounding, some of it wrong, all of it self-validated. The model that produced the finding is the last agent you want confirming it. Cloudflare’s adversarial separation prevents this. Results land in findings.json with three verdicts: confirmed, needs_validation, or rejected — no optimistic lists of unverified concerns.
The 14 commits are not the point. The pattern is.
What the Open-Source Security Skill Hunts
The skill runs a six-phase pipeline: reconnaissance (mapping architecture and trust boundaries into architecture.md and coverage-ledger.json), coverage-led hunting (isolated hunter agents per attack class), candidate validation (adversarial verification), structured output, independent record verification, and target-neutral reporting. Output includes REPORT.md, FINDINGS-DETAIL.md, and NEEDS-VALIDATION.md.
Ten domain playbooks cover attack classes across memory safety, web protocols, auth bypass, client-side code, supply chain and CI/CD integrity, cloud deployment configuration, RPC and messaging, data isolation, and resource exhaustion. One of those playbooks explicitly covers AI-specific threats: prompt injection and unsafe model outputs. Traditional static analyzers have no concept of these attack surfaces. Any application with an LLM component has exposure that Semgrep cannot see.
However, Cloudflare’s own documentation warns that a single run finds roughly half of what repeated runs turn up. Plan to run it at least twice.
What It Costs and What You Do Not Get
Token costs are real. Developers on the Hacker News thread reported consuming around 150,000 tokens on a small FastAPI project and up to 1 million tokens on a medium-sized codebase. Run this on targeted sections or pull requests, not full repositories, unless you have a large token budget.
The skill also requires an OS-enforced sandbox for any codebase that runs external processes, browsers, or tests. Without sandboxing, it refuses to execute target code and marks all findings as needs_validation — meaning your confirmed findings list will be empty. This is the right design decision, but it is a gotcha worth knowing before you run it.
What it cannot do: replace compliance-grade pentests. HIPAA, PCI DSS, and SOC2 require third-party human auditors. Using this as a checkbox to avoid professional security review is a mistake. Using it to find the obvious and moderately hidden problems before spending $5,000–$50,000 on a human pentest, as ProvenLabs analyzed in detail, is a sensible use of the tool.
How to Run the Cloudflare Security Audit Skill
Install via the Skills CLI and invoke it through plain language inside your coding agent:
npx skills add https://github.com/cloudflare/security-audit-skill --skill security-auditThen inside your agent: “security audit this codebase” or “find vulnerabilities in ./src/auth”. Results land in ~/security-audit-skill/<repo-name>/run-1/ by default. Run it twice — the first pass covers roughly half the ground.
Key Takeaways
- The adversarial verification design — hunter agent never validates its own findings — is what separates this from LLM security wrappers. The 14 commits are not the point; the architectural pattern is.
- Ten attack-class playbooks include explicit AI/LLM threat coverage (prompt injection, unsafe model outputs) that static analyzers cannot provide.
- Token costs are substantial: budget for 150,000–1,000,000 tokens per run depending on codebase size. Run on PRs or targeted directories, not full repos.
- Without OS-enforced sandboxing, findings are automatically marked
needs_validation— set up the sandbox before running to get confirmed results. - This is a first-pass screening tool, not a pentest replacement. It does not produce compliance attestation, but it finds real vulnerabilities before you pay a professional to find them.
