A harness turns a model proposal into accountable work

TL;DR: A harness turns model proposals into bounded, inspectable work by controlling context, tools, execution, verification and recovery. Start from the customer operation and decide which facts the runtime must establish before it accepts completion.

Where you are. This is the starting point for a course that builds both a coding harness and a domain-specific procurement harness.

One customer, two kinds of work

Northstar Components is a fictional distributor. Its buyers use a small workbench to inspect purchase requests and propose quantity changes. Every record, trace and result in this course is synthetic. You can reproduce the exercises without customer data or a paid model account.

The engineering team wants an agent to repair defects in the workbench. The procurement team wants an agent to find the applicable purchasing rule, explain a discrepancy and prepare an amendment for a buyer. Both agents might use the same model. They should have different tools and different definitions of success.

A coding task can finish when a particular patch passes protected acceptance checks and is ready for human review. It has not thereby been deployed. A procurement task can finish when the buyer has approved an exact amendment and the service has recorded its receipt. That receipt does not mean a supplier has accepted the order or shipped anything.

Begin your design with those boundaries. A model can suggest a patch, a search query or a purchase amendment. It cannot establish its own authority simply by describing an action as safe. The harness must obtain identity and permission from trusted application state, execute only admitted operations, and collect evidence from the system being changed.

Follow the proposal through the boundary

Here is the course's common control path. The coding and procurement adapters provide different tools and verifiers at the bottom. The model interface stays small.

Notice that the model does not have an edge directly to the accepted result. The runtime owns that transition. When a tool times out, the observation may be “outcome unknown.” A summary saying “the update probably worked” cannot convert that into a receipt.

A harness can be a few functions around one model call. It can also be a durable service coordinating workers and reviewers. The useful distinction is which responsibilities it implements. An SDK supplies building blocks; your deployed configuration determines the actual permissions and recovery behavior. An evaluation harness runs trials and grades outcomes. This course builds an execution harness and then evaluates it with a separate test system.

Put each decision with its owner

For Northstar, write the ownership table before choosing a framework:

The model remains useful: locating a defect, selecting a relevant document or drafting an explanation can require judgment. Deterministic code should handle the parts already expressible as exact rules. If every input maps to a known action without judgment, start with a normal workflow. Adding an agent introduces another failure surface without creating a new capability.

A stronger model may need less planning or fewer repair attempts. It still cannot grant itself access to a customer's account. Keep performance scaffolding, such as plan reminders, separate from authority controls, such as service-side authorization. You can experiment with the former without casually removing the latter.

Do this before moving on

Use these supplied Northstar events:

1. The coding agent says, “The amount comparison is fixed.” No check has run.
2. The test runner reports that candidate revision c17 passes the threshold cases.
3. A buyer approves order N-104, revision 7, changing quantity from 12 to 16.
4. Before submission, another buyer changes the order to revision 8.
5. The service times out after receiving the amendment request.

Deliverable: label each event as a proposal, evidence, authority or unresolved effect. State whether the task may be called complete.

Worked review. Event 1 is a proposal about completion. Event 2 is evidence about c17 and only the tested cases; it can support a review-ready patch, not a deployment claim. Event 3 supplies authority for one revision-bound amendment. Event 4 invalidates that approval for a changed order. Event 5 leaves the effect unresolved until the service can reconcile it. None of these events authorizes silently applying the old approval to revision 8.

Change the input. Suppose the model provides a convincing explanation that revision 8 changed only a comment. Can it reuse the approval? Under the supplied contract, no. The contract binds the whole revision. A narrower approval digest is possible, but the service and approver must agree on that contract before it is used. The worker cannot invent the exception mid-run.

Go deeper

• AI agents: Review the model and tool loop before adding runtime responsibilities.
• Bounded autonomy: Separate permissions, stop conditions and human intervention in the customer workflow.
• Production AI Agents: Build the complementary support workflow when you want more practice with action authority.
• Harness research: Examine controlled experiments before assuming another team’s harness settings will transfer.

Key takeaways

• Define completion as an observed customer outcome with a stated boundary.
• Keep proposal, authority, execution and verification as distinct events.
• Reuse runtime mechanisms while keeping domain permissions and verifiers explicit.

Check yourself

Answer before you look. Recalling it is what makes it stick; recognising it does not.

1. 1A model says a patch is complete, but no acceptance check has run. What may the harness record?

   Accepted patchUnverified completion proposalSuccessful deployment

2. 2What can change when replacing the model, and what must remain enforced?

   Answer it, then reveal