GO TO A SECTION

↑ ↓ to navigate · Enter to openExplore all sections →
FDEInterviews logo

Security tools for AI-generated code: build a release pipeline that catches real risks

Compare Trivy, Semgrep, TruffleHog, Codex Security, Claude review, SkillSpector, PyRIT and more by the risk they detect. Turn findings into tested fixes and explicit release gates.

BY BRANDON FOSTER · FDEINTERVIEWS EDITORIAL · UPDATED NOVEMBER 16, 2026 · 8 MIN READ

PRACTICE THIS:Practise AI security design ·Validate generated code ·Learn harness engineering

Secure AI-generated code with several checks that answer different questions: does the change violate a code rule, expose a secret, introduce a vulnerable dependency, or permit an action the caller should not have? Add model-assisted review to investigate context, then require reproducible evidence and a regression test for each accepted finding. A list of popular repositories is useful for discovery; the release decision needs a threat model and an owner.

The tool comparison was checked November 15, 2026; the agent-policy section was added November 16. It does not claim a benchmark across twelve tools. The projects below cover distinct jobs, and their source licenses do not automatically include free model inference or hosted services.

Choose the check by what can go wrong

JobOfficial projectWhat the result can establish
Source-pattern analysisSemgrepA rule or supported analysis matched code; inspect reachability and context
Dependency and configuration reviewTrivyA supported scanner identified an advisory, secret or configuration issue
Credential discoveryTruffleHogCandidate secrets, with optional provider verification
Agent-driven repository auditCloudflare Security Audit SkillStructured audit coverage and findings with validation status
Code investigation and repairOpenAI Codex SecurityCandidate vulnerability evidence, proposed patches and verification results
Trusted-PR model reviewClaude Code Security ReviewContextual findings under the Action's prompts and filters
Agent-skill inspectionNVIDIA SkillSpectorStatic and optional model-based findings in instructions and scripts
Template-based application checksNucleiSelected protocol requests matched a template's conditions
Agentic penetration testingStrixAn investigated attack path, potentially with a proof of concept
Uploaded-file malware scanningClamAVSupported file/signature checks, subject to unpacking and scan limits
Generative-AI risk assessmentMicrosoft PyRITBehavior observed under chosen attacks and scoring rules
Resistance to automated analysisBlind LLM AgentsArtifact transformations intended to frustrate reverse engineering

The last row needs particular care. BLA modifies artifacts with obfuscation, decoys and adversarial material. It does not find vulnerabilities or make embedded credentials safe. Keep it outside a vulnerability-detection checklist, and do not treat its structural verification as proof of complete behavioral equivalence.

Build three layers around the pull request

Start with fast checks on source and dependencies. Add targeted contextual investigation for sensitive changes. Run behavioral security tests against the built application in an isolated environment. In the figure, every layer produces evidence for the same release decision; an unavailable check has its own blocked outcome.

Every check must leave evidence for releaseevidence complete. failure / gap. Source and artifacts: Code patterns, dependencies, configuration and secrets. Example tools: Semgrep, Trivy and TruffleHog.. Contextual review: Investigate sensitive changes. Keep findings separate from verified vulnerabilities.. Behavioral tests: Tenant boundaries, application requests and agent misuse. Execute only in a scoped environment.. Release decision: Revision, check versions, coverage, findings, exceptions and regression tests.. Approve the artifact: Required checks passed; remaining exceptions have an owner and expiry.. Hold and investigate: A crash, skipped check or unresolved boundary is not a clean result.. Source and artifact scanning, contextual review, and isolated behavioral testing independently produce evidence for one release decision. Complete evidence allows approval under policy. Failures and unresolved gaps hold the release. Every check must leave evidence for release Recommended pipeline. Tools cover different failure classes; their verdicts are not interchangeable. THREE COMPLEMENTARY CHECKS evidence complete failure / gap Source and artifacts Code patterns, dependencies, configuration and secrets. Example tools: Semgrep, Trivy and TruffleHog. Contextual review Investigate sensitive changes. Keep findings separate from verified vulnerabilities. Behavioral tests Tenant boundaries, application requests and agent misuse. Execute only in a scoped environment. Release decision Revision, check versions, coverage, findings, exceptions and regression tests. Approve the artifact Required checks passed; remaining exceptions have an owner and expiry. Hold and investigate A crash, skipped check or unresolved boundary is not a clean result.

For a proposed document-download endpoint, ordinary tests might show that a user can download a document. The security test asks whether a user from tenant B can download tenant A's document by guessing its identifier. The expected result is denial without returning the file. A fluent implementation can omit that ownership check while passing syntax, types and happy-path tests.

Our generated-code validation question explains the ordering. Security failure modes of generated code supplies the kinds of omissions to test.

Model-assisted review needs its own trust boundary

Cloudflare's audit skill asks the host coding agent to organize an investigation, challenge candidates and validate finding records. Valid JSON establishes the record format; it does not prove the vulnerability exists. Preserve unresolved findings as unresolved until a reproducible test or other sufficient evidence closes them.

Codex Security's public CLI and SDK are Apache-2.0. Its model backend and optional Cloud service remain separate dependencies. Inspect the data path before sending private source to a review system, and put real execution timeouts around jobs rather than assuming an advisory investigation budget will stop them.

Anthropic's MIT-licensed Security Review Action explicitly limits itself to trusted PRs and warns that it is not hardened against prompt injection. Its default filtering excludes categories such as denial of service, resource exhaustion and open redirects. A clean result therefore cannot close those parts of the threat model. This repository is also distinct from Anthropic's hosted security offering.

Treat repository text and proposed patches as untrusted inputs to the reviewer. Execute a reproduction in a disposable environment with scoped credentials, not in a release job holding production secrets. The sandboxing concept applies to review agents as much as coding agents.

Check the network and license assumptions

Local installation does not imply offline analysis. TruffleHog's verification can contact the credential provider; --no-verification disables that stage. Filtering output to verified secrets can hide unverified candidates and verification failures. An exposed credential still needs investigation even if its provider cannot currently be reached.

SkillSpector's optional LLM stage sends content to its configured provider. Its dependency checks can contact OSV even with model review disabled. Our executed SkillSpector review shows why findings matter more than the aggregate SAFE label. Pair security inspection with SkillEvaluator's performance checks when deciding whether a skill belongs in the agent at all.

Semgrep's Community Edition engine uses LGPL-2.1; the published rules collection uses a separate Semgrep Rules License. TruffleHog is AGPL-3.0, and ClamAV's core is GPLv2. Review the actual components and deployment terms instead of labeling the whole stack “free open source.”

Active tests belong in an explicit environment

Nuclei sends requests to the target. Some templates can execute code on the scanner host through its code protocol, so template provenance matters. Strix can investigate and execute attack paths. Run either only against an authorized test environment with defined scope, a stop condition and disposable data.

PyRIT exercises generative-AI behavior through configurable attacks and scores. A successful jailbreak demonstrates an observed behavior under those conditions; whether it crosses a production security boundary depends on the application's tools and permissions. Test the whole action path, including denials and approvals, rather than grading only the model's text.

For uploads, ClamAV serves another purpose. Quarantine the file until the scan completes, distinguish a scan error from a clean verdict, and enforce decompression and file-size limits. Antivirus does not replace authorization, safe rendering or protection from an overly expensive parser.

Separate intent filtering from tool-policy enforcement

A remediation agent can pass a content filter and still make a damaging change. The release pipeline needs checks on what the agent can actually execute. Three recent sources illustrate different parts of that boundary:

SourceWhat it establishesWhat it leaves open
Soteria, November 13A framework combining verified hierarchical plans with runtime specification enforcementWhether your specifications cover the harms you care about
DGA, November 13Evaluated malicious requests can pass model-based safety defensesWhether a particular tool-policy monitor would permit an action
Cogent/TRACE, November 10A vendor-described deterministic check of tool calls against enterprise policiesIndependent evidence of coverage or resistance to these attacks

Soteria checks structured plans before execution and enforces specifications at runtime. Its appendix A.3.5 makes the assumptions explicit: faithful tool contracts, runtime satisfaction of the stated llm_eval postconditions, and acyclic plans. It describes both strict plan-following and a guided mode that still checks specifications. A guarantee about those specifications cannot cover a business restriction nobody wrote down. Full paper.

Soteria's Table 2 covers 207 tasks: 50 airline, 114 retail and 43 selected Toolathlon tasks, across four models. The authors report 100% conformance and 4.7–43.9 percentage-point gains in conformance-gated task success over the best baseline per model/benchmark pair. The 93% baseline conformance headline is a maximum across settings, not a pooled comparison. They also corrected and added specifications during result auditing. These are author-run results under those specifications, not independent reproduction or general safety evidence.

DGA evaluates a complete attack pipeline across eleven victim models and three datasets using automated judging. The authors explicitly say their results do not isolate rich detail as the cause of success; their user study also does not measure the filter's usability cost. Nothing in that evaluation establishes that Soteria would stop DGA. Keep content-safety failures and unauthorized tool execution as separate outcomes. Paper, introduction and evaluation.

OpenAI's Cogent case study describes TRACE checking proposed tool calls before execution without a separate model call. It also describes an independent authorization gateway and optional isolation for higher-risk work. Treat this as a published architecture account, not an independently reproduced security result. We have not tested TRACE.

A policy pass can hide a missing ruledenied. permitted. Proposed service restart: Synthetic agent action during settlement.. Check the written policy: Does this principal own the service? Apply the configured rule before execution.. Call blocked: An attempted policy violation. Confirm that no side effect occurred.. Call executes: The owner may restart it, but the rule omitted the settlement window.. Business requirement fails: Record the harm, repair the policy, and preserve allowed maintenance.. A proposed restart reaches a policy check. Denial blocks execution. Permission can still lead to a business failure when the policy omits the settlement window. This is a synthetic evaluation example, not a product architecture. A policy pass can hide a missing rule Illustrative evaluation paths. Record enforcement and business outcomes separately. denied permitted Proposed service restart Synthetic agent action during settlement. Check the written policy Does this principal own the service? Apply the configured rule before execution. Call blocked An attempted policy violation. Confirm that no side effect occurred. Call executes The owner may restart it, but the rule omitted the settlement window. Business requirement fails Record the harm, repair the policy, and preserve allowed maintenance.

The blue check in this diagram asks whether the written policy allows the call. The amber path exposes a different defect: a permitted action can still violate an omitted business requirement. This is our evaluation design, not a diagram of any of the three systems.

For a synthetic maintenance example, suppose the agent may restart any service it owns. Restarting a payment service during settlement could satisfy that rule while breaking the business's availability requirement. Add the missing maintenance-window restriction with the service owner, then test both the blocked settlement-time restart and the allowed scheduled restart. Do not let a policy repair pass by disabling useful work entirely.

Give each specification an owner, revision and regression cases. Include attempted violations, harmful-but-permitted actions and legitimate work incorrectly blocked. Record denied calls separately from effects that actually happened. The AgentSpec breakdown develops runtime rules; the CaMeL breakdown covers data provenance and permitted recipients. Neither substitutes for deciding which actions the customer authorizes.

Make each accepted finding leave a useful test

A release record should identify the source revision, tool and rule versions, what ran, what was skipped, the evidence, the affected boundary and the person responsible for disposition. Give exceptions an expiry and a reason. A tool crash or malformed report must never silently become zero findings.

For the tenant-download example, keep the denied cross-tenant request as a regression test after the fix. Add the same-owner success case so the repair cannot pass by disabling downloads entirely. Re-run both against the patched artifact, then retain the report with the commit.

Measure confirmed findings and escaped regressions alongside review time. Counting every warning as a vulnerability rewards noisy scanners and trains engineers to ignore the output. The AI security question bank and policy-as-code exercise develop this into a release process a team can maintain.

PRACTICE THIS

Turn it into offers. Work the real questions and concepts this maps to:

FAQ

Choose by failure class. Use source analysis for insecure code patterns, dependency and configuration scanning for artifacts, secret scanning for credentials, and application-level tests for authorization and business rules. Add model-assisted review where it provides useful evidence. No single tool covers all of these.

Discussion (3)

Lukas HoffmannEditor

The most useful report field is often evidence: the source revision, relevant path, failing request and expected boundary. That lets a second engineer reproduce the problem without trusting the scanner's prose.

Emily CarterEditor

A scanner update is a policy change if it changes which releases pass. Pin rule versions, inspect changed findings and make exceptions expire; otherwise the same commit can receive different decisions with no explanation.

Arjun MehtaEditor

The reviewer should not execute an agent's proposed patch with deployment credentials. A review process can become the attack path when it inherits more authority than the application it is inspecting.