decision models

The cheap swap is not a model swap. The prompt becomes atomic typed questions, the parsed label becomes a distribution with a confidence, and the release gate becomes a per-version threshold sweep on the customer's own labeled tickets.

Cheap typed judgments make it affordable to check every input, tool call and output. The design question is which checks are semantic, which are deterministic, and how you prove the layer catches what it claims to.

The pitch is a model that runs the loop. The design that survives production is a loop that code runs, with the model answering three narrow questions inside it: which tool, is this done, does this need a stronger model.

One call per company, a pinned domain, an unknown option on every question, and a write policy that checks whose website the evidence came from. Without that check, a same-name company's GitHub page updates your customer record.

Fourteen of seventeen answers right looks fine until you split by stratum: every recently changed fact was wrong, with near-total confidence. The eval is a dated golden set, stratified by how facts fail on the web, re-verified before every run.