IN THIS BREAKDOWN · 10 SECTIONS
You can read 17% of this breakdown.
The paper in one minute
A language model can solve a problem in prose yet struggle when required to emit a strict JSON object from its first token. Grammar-constrained decoding removes invalid next tokens, but that operation changes the distribution from which the model generates. It does not merely wrap an already-computed answer in braces.
Draft-Conditioned Constrained Decoding, or DCCD, separates the work into two stages. First, a model produces an unconstrained draft. Then a constrained generation pass uses that draft as context to produce the required structure. The same model can perform both stages, or different models can be assigned to them.
The paper reports higher strict structured accuracy across its evaluated reasoning tasks. It also analyzes how much probability the unconstrained model assigns to valid next tokens, calling the distortion from masking a projection tax.
The engineering lesson is to evaluate meaning and format separately. A schema can restrict permissible strings and types. It cannot establish that an amount, identifier or proposed action is correct, and a second pass adds cost that must be measured across the complete request.
Research notes: the problem, approach and results
unconstrained draft, then constrained realization
Llama 3.2 1B GSM8K structured accuracy, Table 2
instruction-tuned model sizes in the main evaluation
Forcing valid output tokens can change the model's semantic decisions.
Generate a free-form draft, then condition a grammar-constrained final pass on that draft.
Structured accuracy improves in the reported tasks, at the cost of a second inference stage.
Why it matters to an FDE
An enterprise agent may generate an API request that parses cleanly but selects the wrong account or calculates the wrong adjustment. A dashboard showing near-perfect JSON validity would miss that failure. The useful metric combines structural acceptance with independently checked task correctness.
This paper offers one way to improve that combination when format constraints interfere with reasoning. It is worth testing when a model performs better in an unconstrained task than under the production schema. It is not evidence that every modern structured-output endpoint needs another model call.
Start with structured output and validation. Keep authorization, business rules and final state checks outside the model. Treat a draft as untrusted intermediate material, and ensure only the validated final result can reach an executable tool boundary.
Understand the mechanism. Try it on a real problem.
Unlock this breakdown and 199 more papers.
Premium opens every paper in full, plus every answer, concept, course and guide on the site. One payment for six months of access.
6 months · One payment · No auto-renewal
- ◇1 architecture diagram
- </>2 runnable code blocks
- ▦3 results tables
- 03Key takeaways
Distinguish syntax, draft quality, probability distortion and deployment cost
Free account - 04How it works
Derive the masking cost and follow draft generation into a separately constrained output stage
Premium - 05The numbers that matter
Read structured accuracy and runtime within their own experimental settings
Premium - 06Try it yourself
Verify the KL identity and show why high feasible mass can accompany a wrong answer
Premium - 07Where it breaks
Watch for wrong drafts, unsupported schemas, incomplete outputs and misleading efficiency claims
Premium - 08Using it on a deployment
Pilot two-stage generation with matched budgets and separate semantic and structural checks
Premium - 09Interview angles
Explain what grammar constraints guarantee and how to evaluate a draft-then-constrain design
Premium - 10Read next
Connect constrained generation with reasoning and exact speculative acceleration
Premium