GO TO A SECTION

↑ ↓ to navigate · Enter to openExplore all sections →
FDEInterviews logo
RecentarXiv preprint

Constitutional Classifiers++: Efficient Production-Grade Defenses Against Universal Jailbreaks

Anthropic's production jailbreak defense pairs a probe with a classifier that reads prompt and reply together: 5.5% of traffic escalated, about 40x less compute than the exchange classifier alone.

Breakdown by FDE Interviews · 12 min read · Updated

About the original · 2026
AUTHORS
Cunningham, Wei, Wang, Persic, et al.
FROM
Anthropic
PUBLISHED
Jan 2026 · arXiv preprint
IN THIS BREAKDOWN · 10 SECTIONS

You can read 16% of this breakdown.

01

The paper in one minute

Constitutional Classifiers are Anthropic's defense against universal jailbreaks: prompting strategies that get a model to answer most questions in a dangerous domain, here chemical, biological, radiological and nuclear (CBRN) weapons. The previous version used an input classifier plus a separate output classifier, cost 23% extra compute and refused 0.38% of production traffic.

Further red teaming found two ways around it. Reconstruction attacks scatter a request across a long, harmless-looking context and have the model reassemble it. Output obfuscation has the model answer in substitutions or metaphors that look harmless unless you also read the prompt. Both degrade the model's capability, but by varying amounts, so better versions are likely.

The paper makes three changes. An exchange classifier judges the output in the context of its input, token batch by token batch as the reply streams. A two-stage cascade runs a cheap classifier on everything and escalates only suspicious exchanges to an expensive one, which can therefore flag freely. And linear probes on the protected model's own activations serve as an almost free first stage, ensembled with an external classifier fine-tuned from Claude Haiku 4.5.

The production system uses 3.5% of the old system's compute, refused 0.05% of production traffic during a month of shadow deployment on Claude Sonnet 4.5, and had the lowest rate of high-risk vulnerabilities in about 1,736 hours of red teaming. No red teamer found a universal jailbreak that answered all eight target questions at half the detail of an unguarded model.

THE IDEA, VISUALLY
PRODUCTION CASCADE (SECTION 6) Exchange input + output Linear probe all traffic, ~0 cost pass ~94.5% ~5.5% escalated Haiku 4.5 classifier exchange classifier Blend logits refuse if high Scores stream as tokens are sampled; the probe is smoothed (EMA) so one spiky token can't flag a whole response. Final logit: 0.55 probe + 0.45 classifier WHY EXCHANGE, NOT INPUT + OUTPUT (SECTION 2) Reconstruction request split across a long context, reassembled in the reply Output obfuscation reply only harmful when read next to its prompt (code words) Both evade classifiers that read input and output apart. Both cost capability: GPQA Diamond fell from 74.2% to 32.3% and 49% under two such attacks. Probe: 2Ld = 377K FLOPs/token vs ~8B for a 4B-model pass Needs the protected model's activations: not an API option FOUR SYSTEMS (TABLE 1) compute overhead, % of last generation last gen 100% exchange 150% two-stage 27.8% production 3.5% high-risk vulns per 1K red-team queries last gen 0.01871 exchange 0.00885 two-stage 0.00878 production 0.00505 refusal rate on production traffic last gen 0.073% exchange 0.038% two-stage 0.036% production 0.050% Production: ~1,736 red-team hours, 198K queries, 1 high-risk vulnerability found after 30 h; no universal jailbreak hit all 8 CBRN questions at half an unguarded model's detail. Lowest refusal rate is the two-stage cascade, not production.
A linear probe on the protected model's own activations screens every exchange almost for free; only about 5.5% of traffic reaches the expensive classifier, and the two scores are blended for the final call.
Research notes: the problem, approach and results
~40x

less compute than the single exchange classifier (3.5% vs 150% of the last generation)

0.05%

refusal rate on production traffic, shadow deployment on Claude Sonnet 4.5, Dec 2025

0.005

high-risk vulnerabilities per thousand red-team queries, lowest of the four systems (last generation: 0.019)

THE PROBLEM

Last-generation Constitutional Classifiers read inputs and outputs separately, which attackers beat by splitting requests across long contexts or disguising replies, and they cost 23% extra compute with a 0.38% refusal rate on production traffic.

THE IDEA

Replace the pair with an exchange classifier that judges each output in the context of its input; run a cheap first stage on all traffic and escalate only suspicious exchanges; make the first stage a linear probe on the protected model's activations and blend it with the external classifier.

THE RESULT

The production system uses 3.5% of the last generation's compute, refused 0.05% of production traffic in a month of shadow deployment, and had the lowest vulnerability rate across about 1,736 hours of red teaming, with no universal jailbreak found.

02

Why it matters to an FDE

Guardrail projects usually die on cost and false positives, not on the security idea. This paper is a rare public account of a large provider designing a classifier system against both constraints at once, with numbers for each.

Judge the exchange, not the message. Checking outputs without their inputs misses anything that's only harmful in context. The same holds for a customer's data-leak or policy classifier.

Cascades let the first stage over-flag. Because a first-stage flag means "look closer", not "refuse", a weak, cheap model is good enough there. It's the same economics as model routing, applied to safety.

Know what you can and can't build. Activation probes need access to the protected model's internals, so a team calling a hosted API can't copy that part. The cascade, the exchange framing and the streaming evaluation all transfer to any stack.

THE FULL BREAKDOWN

Understand the mechanism. Try it on a real problem.

Unlock this breakdown and 157 more papers.

Premium opens every paper in full, plus every answer, concept, course and guide on the site. One payment for six months of access.

6 months · One payment · No auto-renewal

Get Premium · ₹2,000 / $25
  • ◇1 architecture diagram
  • </>1 runnable code experiment
  • ▦2 results tables
  • ?5 interview questions with answers
  1. 03Key takeaways

    Two attack classes, three design changes, what red teaming measured, three misreadings

    Free account
  2. 04How it works

    Exchange classification, the cascade, probe training, the ensemble and the red-teaming protocol

    Premium
  3. 05The numbers that matter

    Red-teaming results per system, cost ratios, probe ablations and the deployment month

    Premium
  4. 06Try it yourself

    Check Table 1 and the cost claims, then simulate a probe-then-classifier cascade

    Premium
  5. 07Where it breaks

    Internals access, CBRN-only scope, shifting red-team rules, comparability and slips

    Premium
  6. 08Using it on a deployment

    Exchange-level checks, a cascade with a compute budget, streaming evaluation and the probe question

    Premium
  7. 09Interview angles

    Five questions this paper prepares you for, with answer hints

    Premium
Back to the paper library →