A field guide to decision models

Imagine a technician standing beside a pump, holding a phone with a photo of its control panel. The app needs to choose a troubleshooting checklist. There are three approved checklists, the connection is unreliable, and a blurry photo should trigger a request for a better one.

That is a useful place to try a decision model: a model that evaluates supplied evidence and returns a bounded result that software can use. Here, the result could be a probability for each checklist, plus an option for insufficient evidence. The app still controls what opens and what the technician is allowed to do.

Start with the choice your software needs

Three common output shapes cover a lot of useful work:

• A yes/no probability: does this recording contain the requested wake phrase?
• A choice among named options: which checklist matches the visible panel?
• A score against described levels: is the submitted photo unusable, partly readable or clear enough to inspect?

The newer System One models read the state and questions, then return typed answers without generating a prose response token by token. That can remove decoding and parsing work from a repeated decision. It does not make the answer correct, make inference free or replace a model that must write an explanation.

Choose the simplest credible baseline first. A barcode lookup should stay a lookup. Logistic regression or a small boosted-tree model may be the right answer for labeled tabular data. An embedding classifier can suit stable text labels. A general decision model becomes interesting when the judgment depends on language, images or changing natural-language criteria. A generative model still belongs where the output itself must be new text or code.

In the picture, B wins the ranking and still does not open. The probability threshold, supported equipment and user permissions all belong in the application. The model proposes; the application checks whether the proposal is usable. Our public harness-boundary lesson explains that division.

A shortlist worth investigating

These are starting points for an evaluation, not a ranking. Model notes were checked in November 2026; follow the linked instructions for the exact checkpoint and runtime.

• TypeSafe's Jev offers hosted typed decisions, including Choice, Score and yes/no questions called Noul. It suits a quick API experiment. Treat provider latency, data handling and version changes as dependencies.
• Laya is an open-weight text-decision family with English, multilingual and task-tuned checkpoints. Its project documents local serving and fine-tuning. Check the chosen checkpoint's input limit; a language router does not remove that limit.
• Liquid AI documents d1-2B for text, pictures and video, and d1-Omni-2B for audio as well as text and pictures. The smaller d1-omni-600M is an experimental option for text with images or short audio. Open weights make local evaluation possible; each target still needs a supported runtime and license review.
• Cloudflare's Clef-omni accepts text, images, audio and video through Workers AI. Its weights are also published. It is an option for a managed service, not evidence that a network round trip fits your deadline.

A shared request shape does not make these models interchangeable. Their supported outputs, score semantics, modalities and failure patterns differ. Re-run your evaluation when switching.

Six places to look for a useful decision

These are proposed designs to evaluate on your own data, not accounts of customer deployments.

On a phone: help someone finish a task offline

Return to the technician. A local model could select a checklist or ask for another photo without uploading the image. Keep the approved documents cached too; offline inference is little help if the next screen needs a network request. Offer manual selection when the model cannot decide. Measure cold starts, memory and sustained battery use on the actual phone.

On the critical path: decide whether expensive work is needed

A support system could choose between opening an existing status page, retrieving a document and escalating to a reasoning model. The router adds its own delay, so measure the whole request, including fallback. Watch cases where it confidently selects the cheap path and misses a problem. Give it a deadline and a defined timeout route. A fast benchmark on a short input says little about a long, queued request.

On a server: sort work before people see it

Route incoming tickets to teams, find messages that lack required information, or check whether a draft reply addresses the customer's actual request. Keep account lookup and access checks in code. For a stable label set with enough examples, compare against a supervised classifier before taking on a more general model. The useful metric is incorrectly routed work and review load, not valid JSON.

In retrieval: choose where to search next

DecisionRAG explores using decision models to navigate a document hierarchy and score candidate passages. This suggests an alternative for manuals whose relevant section is hard to find with similarity alone. The hierarchy still has to be built, maintained and evaluated. A wrong early branch can hide the answer. Our public retrieval walkthrough explains how this differs from searching screenshots with PixelRAG.

In voice: judge whether you have enough usable audio

An audio model could flag a clip for repetition or route an utterance to a known handler. Evaluate this against simple audio-quality checks and a transcript-based baseline with comparable information. Accents, microphones and background noise can change the result. Our public Prosodia explainer examines an experimental audio-decision architecture and why a well-calibrated score can still be unhelpful. An affect score should not determine a customer's eligibility or permission.

In physical hardware: separate interpretation from control

A camera-equipped station could flag a missing packing insert for inspection. A maintenance device could classify an unusual sound and request a technician. Try these first as advisory decisions. An embedded GPU, a phone and a microcontroller have very different budgets. On a tiny sensor, a task-specific network using LiteRT for Microcontrollers may be more appropriate than a general language model. Keep emergency stops, actuator limits and safety interlocks independent of model output.

A probability needs an operating policy

A score of 0.90 is useful only if you understand what it estimates. For a calibrated binary predictor, cases assigned roughly 0.90 should turn out positive roughly 90% of the time on comparable data. That is a property of groups of predictions, not proof about this photo.

Also check the field name. TypeSafe documents its confidence field as a measure derived from the answer distribution. It is not the same as the selected option's probability. A rule tuned to one must not silently read the other.

Consider this authored example with 100 labeled decisions. We use the selected option's predicted probability to decide whether to handle a case automatically. At a threshold of 0.80, 80 cases proceed and eight of those choices are wrong. At 0.95, 40 proceed and two are wrong. The remaining cases go to review.

The higher threshold avoids six wrong automatic choices but sends 40 more cases to review. Can the team handle that queue? Are those six mistakes costly enough to justify it? The figure's counts are synthetic, not a benchmark or proof that higher thresholds always work better. Review can also be wrong or late.

Track error among automatic decisions alongside coverage, the fraction handled automatically. Break both down by input type and user group. A model that is unsure about everything may look safe while leaving almost all the work to people. Calibration, useful separation and workflow outcomes need separate checks.

Tune the decision before training the model

1. Define the answer space. Write one narrow question with distinguishable options. Include missing evidence and out-of-scope inputs in the design. “Other” is an option; abstaining is what your application does when it cannot justify acting. They serve different purposes.
2. Collect representative examples and reviewed labels. Keep related customers, devices or conversations together when splitting data, so near-duplicates do not leak into evaluation. Include ambiguous cases and realistic failures, not just clean successes.
3. Test the baseline, then improve the question and evidence. If repeated domain errors remain, fine-tune a supported open checkpoint on labeled decisions. Laya's training notebook provides one concrete starting point. RLCD is a training approach used by some decision models; it is not a prerequisite for every useful classifier or a guarantee on your data.
4. Fit any probability calibration on held-out predictions, choose thresholds on validation data, and evaluate the frozen policy on an untouched test set. Calibration adjusts how probabilities map to outcomes; threshold tuning changes which action you take. Neither repairs bad labels.

For a small dataset, use a documented cross-validation procedure instead of repeatedly tuning against the final test set. Save the model revision, input preparation, question schema and thresholds together.

Deploy the exact thing you evaluated

Start in shadow mode: record the proposed decision while the existing workflow stays in charge. Inspect false confident answers and the fallback queue. Then introduce a reversible slice with a rollback path.

For local inference, use the publisher's model-specific instructions. Verify that the runtime supports the decision head and requested modalities; an automatic “chat with this model” snippet is not sufficient. Review any custom model code before executing it. Recheck held-out behavior after quantization, which changes numerical precision and can move scores across a threshold.

Measure the deployed path: input capture, preprocessing, model loading, queueing, inference and policy handling. A clip-based audio decision must wait for its required audio. Keep missing input, timeout and unsupported hardware as explicit outcomes. Monitor review volume and error by slice, sample the automatic decisions too, and retain enough versioned evidence to explain a routing choice without logging unnecessary private data.

A small exercise for this week

Pick one recurring judgment in your product. Write its options, the evidence needed, and what happens when it is uncertain. Label a small pilot set and sweep a few thresholds. Put the number of wrong automatic choices beside the number sent to review. Use that table to discuss the workflow with the person who will operate it; a small pilot is a starting point, not release evidence.

---

Keep learning

Give a model one typed decision at a time opens with a free account. The lesson teaches output validation and application policy using recorded decisions; it does not require a paid model API. It leads into a runnable Python harness project.

The public lesson Read an agent evaluation without losing the failures develops the next question: what can a higher overall score conceal?

For a deeper reference, System One Decision Models is an optional Premium concept. The two blog walkthroughs above are free to read.

Have a workflow you would like us to work through? Email the team with the decision, the available input and what a wrong answer would cost. Please leave out private customer data. If this issue would help a colleague, forward it; they can find FDE Field Notes and subscribe completely free here.

Thanks for reading,

Team FDE Interviews