constrained decoding

JSON mode, strict schemas, constrained decoding, validate-and-retry: the escalation ladder for structured output, plus the catch nobody mentions: schema-valid can still be factually wrong.

The escalation ladder is settled; the at-scale tradeoff is not. Why retries quietly multiply your bill and tail latency, why constrained decoding has its own taxes, and the hybrid that most high-volume pipelines actually run.

Constrained JSON/grammar decoding usually means a Python callback per token, which serializes the GPU. The staff answer keeps the DFA on-device and turns the mask into a single coalesced gather.

A code model that occasionally produces ransomware, reverse shells, or exfiltration scripts. The strong answer builds an automated loop: adversarial prompt generation, static and behavioral output scanning, layered mitigations, and attack-success-rate per category to prove the mitigations worked.