Security tools for AI-generated code: build a release pipeline that catches real risks
Compare Trivy, Semgrep, TruffleHog, Codex Security, Claude review, SkillSpector, PyRIT and more by the risk they detect. Turn findings into tested fixes and explicit release gates.
BY BRANDON FOSTER · FDEINTERVIEWS EDITORIAL · UPDATED NOVEMBER 16, 2026 · 8 MIN READ
PRACTICE THIS:Practise AI security design ·Validate generated code ·Learn harness engineering
Secure AI-generated code with several checks that answer different questions: does the change violate a code rule, expose a secret, introduce a vulnerable dependency, or permit an action the caller should not have? Add model-assisted review to investigate context, then require reproducible evidence and a regression test for each accepted finding. A list of popular repositories is useful for discovery; the release decision needs a threat model and an owner.
The tool comparison was checked November 15, 2026; the agent-policy section was added November 16. It does not claim a benchmark across twelve tools. The projects below cover distinct jobs, and their source licenses do not automatically include free model inference or hosted services.
Choose the check by what can go wrong
| Job | Official project | What the result can establish |
|---|---|---|
| Source-pattern analysis | Semgrep | A rule or supported analysis matched code; inspect reachability and context |
| Dependency and configuration review | Trivy | A supported scanner identified an advisory, secret or configuration issue |
| Credential discovery | TruffleHog | Candidate secrets, with optional provider verification |
| Agent-driven repository audit | Cloudflare Security Audit Skill | Structured audit coverage and findings with validation status |
| Code investigation and repair | OpenAI Codex Security | Candidate vulnerability evidence, proposed patches and verification results |
| Trusted-PR model review | Claude Code Security Review | Contextual findings under the Action's prompts and filters |
| Agent-skill inspection | NVIDIA SkillSpector | Static and optional model-based findings in instructions and scripts |
| Template-based application checks | Nuclei | Selected protocol requests matched a template's conditions |
| Agentic penetration testing | Strix | An investigated attack path, potentially with a proof of concept |
| Uploaded-file malware scanning | ClamAV | Supported file/signature checks, subject to unpacking and scan limits |
| Generative-AI risk assessment | Microsoft PyRIT | Behavior observed under chosen attacks and scoring rules |
| Resistance to automated analysis | Blind LLM Agents | Artifact transformations intended to frustrate reverse engineering |
The last row needs particular care. BLA modifies artifacts with obfuscation, decoys and adversarial material. It does not find vulnerabilities or make embedded credentials safe. Keep it outside a vulnerability-detection checklist, and do not treat its structural verification as proof of complete behavioral equivalence.
Build three layers around the pull request
Start with fast checks on source and dependencies. Add targeted contextual investigation for sensitive changes. Run behavioral security tests against the built application in an isolated environment. In the figure, every layer produces evidence for the same release decision; an unavailable check has its own blocked outcome.
For a proposed document-download endpoint, ordinary tests might show that a user can download a document. The security test asks whether a user from tenant B can download tenant A's document by guessing its identifier. The expected result is denial without returning the file. A fluent implementation can omit that ownership check while passing syntax, types and happy-path tests.
Our generated-code validation question explains the ordering. Security failure modes of generated code supplies the kinds of omissions to test.
Model-assisted review needs its own trust boundary
Cloudflare's audit skill asks the host coding agent to organize an investigation, challenge candidates and validate finding records. Valid JSON establishes the record format; it does not prove the vulnerability exists. Preserve unresolved findings as unresolved until a reproducible test or other sufficient evidence closes them.
Codex Security's public CLI and SDK are Apache-2.0. Its model backend and optional Cloud service remain separate dependencies. Inspect the data path before sending private source to a review system, and put real execution timeouts around jobs rather than assuming an advisory investigation budget will stop them.
Anthropic's MIT-licensed Security Review Action explicitly limits itself to trusted PRs and warns that it is not hardened against prompt injection. Its default filtering excludes categories such as denial of service, resource exhaustion and open redirects. A clean result therefore cannot close those parts of the threat model. This repository is also distinct from Anthropic's hosted security offering.
Treat repository text and proposed patches as untrusted inputs to the reviewer. Execute a reproduction in a disposable environment with scoped credentials, not in a release job holding production secrets. The sandboxing concept applies to review agents as much as coding agents.
Check the network and license assumptions
Local installation does not imply offline analysis. TruffleHog's verification can contact the credential provider; --no-verification disables that stage. Filtering output to verified secrets can hide unverified candidates and verification failures. An exposed credential still needs investigation even if its provider cannot currently be reached.
SkillSpector's optional LLM stage sends content to its configured provider. Its dependency checks can contact OSV even with model review disabled. Our executed SkillSpector review shows why findings matter more than the aggregate SAFE label. Pair security inspection with SkillEvaluator's performance checks when deciding whether a skill belongs in the agent at all.
Semgrep's Community Edition engine uses LGPL-2.1; the published rules collection uses a separate Semgrep Rules License. TruffleHog is AGPL-3.0, and ClamAV's core is GPLv2. Review the actual components and deployment terms instead of labeling the whole stack “free open source.”
Active tests belong in an explicit environment
Nuclei sends requests to the target. Some templates can execute code on the scanner host through its code protocol, so template provenance matters. Strix can investigate and execute attack paths. Run either only against an authorized test environment with defined scope, a stop condition and disposable data.
PyRIT exercises generative-AI behavior through configurable attacks and scores. A successful jailbreak demonstrates an observed behavior under those conditions; whether it crosses a production security boundary depends on the application's tools and permissions. Test the whole action path, including denials and approvals, rather than grading only the model's text.
For uploads, ClamAV serves another purpose. Quarantine the file until the scan completes, distinguish a scan error from a clean verdict, and enforce decompression and file-size limits. Antivirus does not replace authorization, safe rendering or protection from an overly expensive parser.
Separate intent filtering from tool-policy enforcement
A remediation agent can pass a content filter and still make a damaging change. The release pipeline needs checks on what the agent can actually execute. Three recent sources illustrate different parts of that boundary:
| Source | What it establishes | What it leaves open |
|---|---|---|
| Soteria, November 13 | A framework combining verified hierarchical plans with runtime specification enforcement | Whether your specifications cover the harms you care about |
| DGA, November 13 | Evaluated malicious requests can pass model-based safety defenses | Whether a particular tool-policy monitor would permit an action |
| Cogent/TRACE, November 10 | A vendor-described deterministic check of tool calls against enterprise policies | Independent evidence of coverage or resistance to these attacks |
Soteria checks structured plans before execution and enforces specifications at runtime. Its appendix A.3.5 makes the assumptions explicit: faithful tool contracts, runtime satisfaction of the stated llm_eval postconditions, and acyclic plans. It describes both strict plan-following and a guided mode that still checks specifications. A guarantee about those specifications cannot cover a business restriction nobody wrote down. Full paper.
Soteria's Table 2 covers 207 tasks: 50 airline, 114 retail and 43 selected Toolathlon tasks, across four models. The authors report 100% conformance and 4.7–43.9 percentage-point gains in conformance-gated task success over the best baseline per model/benchmark pair. The 93% baseline conformance headline is a maximum across settings, not a pooled comparison. They also corrected and added specifications during result auditing. These are author-run results under those specifications, not independent reproduction or general safety evidence.
DGA evaluates a complete attack pipeline across eleven victim models and three datasets using automated judging. The authors explicitly say their results do not isolate rich detail as the cause of success; their user study also does not measure the filter's usability cost. Nothing in that evaluation establishes that Soteria would stop DGA. Keep content-safety failures and unauthorized tool execution as separate outcomes. Paper, introduction and evaluation.
OpenAI's Cogent case study describes TRACE checking proposed tool calls before execution without a separate model call. It also describes an independent authorization gateway and optional isolation for higher-risk work. Treat this as a published architecture account, not an independently reproduced security result. We have not tested TRACE.
The blue check in this diagram asks whether the written policy allows the call. The amber path exposes a different defect: a permitted action can still violate an omitted business requirement. This is our evaluation design, not a diagram of any of the three systems.
For a synthetic maintenance example, suppose the agent may restart any service it owns. Restarting a payment service during settlement could satisfy that rule while breaking the business's availability requirement. Add the missing maintenance-window restriction with the service owner, then test both the blocked settlement-time restart and the allowed scheduled restart. Do not let a policy repair pass by disabling useful work entirely.
Give each specification an owner, revision and regression cases. Include attempted violations, harmful-but-permitted actions and legitimate work incorrectly blocked. Record denied calls separately from effects that actually happened. The AgentSpec breakdown develops runtime rules; the CaMeL breakdown covers data provenance and permitted recipients. Neither substitutes for deciding which actions the customer authorizes.
Make each accepted finding leave a useful test
A release record should identify the source revision, tool and rule versions, what ran, what was skipped, the evidence, the affected boundary and the person responsible for disposition. Give exceptions an expiry and a reason. A tool crash or malformed report must never silently become zero findings.
For the tenant-download example, keep the denied cross-tenant request as a regression test after the fix. Add the same-owner success case so the repair cannot pass by disabling downloads entirely. Re-run both against the patched artifact, then retain the report with the commit.
Measure confirmed findings and escaped regressions alongside review time. Counting every warning as a vulnerability rewards noisy scanners and trains engineers to ignore the output. The AI security question bank and policy-as-code exercise develop this into a release process a team can maintain.
Turn it into offers. Work the real questions and concepts this maps to:
FAQ
Choose by failure class. Use source analysis for insecure code patterns, dependency and configuration scanning for artifacts, secret scanning for credentials, and application-level tests for authorization and business rules. Add model-assisted review where it provides useful evidence. No single tool covers all of these.
The official Codex Security CLI and TypeScript SDK repository is Apache-2.0 licensed. That license is separate from model access, inference charges and the hosted Cloud offering. Installing the client locally does not make inference offline.
The repository says the Action is not hardened against prompt injection and is intended for trusted PRs. Untrusted contributions require a separate execution and credential boundary. Do not give a model-driven review job privileged credentials merely because it runs in CI.
It means the selected checks produced no qualifying findings under their configuration. Scanners can miss authorization errors, unsupported ecosystems, unavailable evidence and new attack patterns. Record skipped checks and tool failures separately from clean results.
It establishes conformance to the specified properties under the verification assumptions. Missing business rules and inaccurate tool contracts remain gaps. Test prohibited actions and harmful actions that an incomplete policy still permits, alongside legitimate task completion.
Discussion (3)
The most useful report field is often evidence: the source revision, relevant path, failing request and expected boundary. That lets a second engineer reproduce the problem without trusting the scanner's prose.
A scanner update is a policy change if it changes which releases pass. Pin rule versions, inspect changed findings and make exceptions expire; otherwise the same commit can receive different decisions with no explanation.
The reviewer should not execute an agent's proposed patch with deployment credentials. A review process can become the attack path when it inherits more authority than the application it is inspecting.
