A harness turns a model proposal into accountable work

TL;DR: A harness turns model proposals into bounded, inspectable work by controlling context, tools, execution, verification and recovery. Start from the customer operation and decide which facts the runtime must establish before it accepts completion.

Where you are. This is the starting point for a course that builds both a coding harness and a domain-specific procurement harness.

One customer, two kinds of work

Northstar Components is a fictional distributor. Its buyers use a small workbench to inspect purchase requests and propose quantity changes. Every record, trace and result in this course is synthetic. You can reproduce the exercises without customer data or a paid model account.

The engineering team wants an agent to repair defects in the workbench. The procurement team wants an agent to find the applicable purchasing rule, explain a discrepancy and prepare an amendment for a buyer. Both agents might use the same model. They should have different tools and different definitions of success.

A coding task can finish when a specific patch passes the protected acceptance checks and is ready for human review. Finishing does not mean the patch has been deployed. A procurement task can finish when the buyer has approved an exact amendment and the service has recorded a receipt for it. The receipt does not mean a supplier has accepted the order, let alone shipped anything.

Design starts from those boundaries. A model can suggest a patch, a search query or a purchase amendment, but it cannot give itself authority by describing an action as safe. The harness must obtain identity and permission from trusted application state, execute only admitted operations, and collect evidence from the system being changed.

The control path through the boundary

Both harnesses in this course share the control path below. The coding and procurement adapters plug in different tools and verifiers, while the interface the model sees stays small.

The model has no edge leading directly to the accepted result; only the runtime can make that transition. If a tool times out, the recorded observation may be “outcome unknown,” and a model summary saying “the update probably worked” cannot turn that into a receipt.

A harness can be a few functions around one model call, or a durable service that coordinates workers and reviewers. Size is not what defines it; the responsibilities it implements are. An SDK supplies building blocks, but your deployed configuration determines the actual permissions and recovery behavior. An evaluation harness runs trials and grades outcomes. This course builds an execution harness and then evaluates it with a separate test system.

Who owns each decision

For Northstar, write the ownership table before choosing a framework:

The model still earns its place where judgment is needed: locating a defect, choosing the relevant document or drafting an explanation. Parts that can already be written as exact rules belong in deterministic code. If every input maps to a known action without judgment, start with an ordinary workflow, because an agent would add another failure surface without adding a capability.

A stronger model may need less planning or fewer repair attempts, yet it still cannot grant itself access to a customer's account. Keep performance scaffolding, such as plan reminders, separate from authority controls, such as service-side authorization. That separation lets you experiment with the first without casually removing the second.

Do this before moving on

Use these supplied Northstar events:

1. The coding agent says, “The amount comparison is fixed.” No check has run.
2. The test runner reports that candidate revision c17 passes the threshold cases.
3. A buyer approves order N-104, revision 7, changing quantity from 12 to 16.
4. Before submission, another buyer changes the order to revision 8.
5. The service times out after receiving the amendment request.

Deliverable: label each event as a proposal, evidence, authority or unresolved effect. State whether the task may be called complete.

Worked review. Event 1 is a proposal about completion. Event 2 is evidence about c17 and only the tested cases; it can support a review-ready patch, not a deployment claim. Event 3 supplies authority for one revision-bound amendment. Event 4 invalidates that approval for a changed order. Event 5 leaves the effect unresolved until the service can reconcile it. None of these events authorizes silently applying the old approval to revision 8.

Change the input. Suppose the model explains, convincingly, that revision 8 changed only a comment. It still cannot reuse the approval, because the supplied contract binds the whole revision. A narrower approval digest is possible, but the service and the approver must agree on that contract before it is used; the worker cannot invent the exception mid-run.

Go deeper

• AI agents: Review the model and tool loop before adding runtime responsibilities.
• Bounded autonomy: Separate permissions, stop conditions and human intervention in the customer workflow.
• Production AI Agents: Build the complementary support workflow when you want more practice with action authority.
• Harness research: Examine controlled experiments before assuming another team’s harness settings will transfer.

Key takeaways

• Define completion as an observed customer outcome with a stated boundary.
• Keep proposal, authority, execution and verification as distinct events.
• Reuse runtime mechanisms while keeping domain permissions and verifiers explicit.

Check yourself

Answer before you look. Recalling it is what makes it stick; recognising it does not.

1. 1A model says a patch is complete, but no acceptance check has run. What may the harness record?

   Accepted patchUnverified completion proposalSuccessful deployment

2. 2What can change when replacing the model, and what must remain enforced?

   Answer it, then reveal