Choose a runtime by the controls you can demonstrate

TL;DR: Choose a runtime by demonstrating the controls your customer requires on the same task. Compare extension boundaries, state persistence, permissions, observation and recovery; product names and feature lists do not establish the behavior of your configured integration.

Where you are. You have built the course's control mechanisms. This public lesson helps you evaluate current coding and domain-agent runtimes against those contracts.

Start with a portable acceptance packet

Northstar's packet contains a broken threshold function, exact acceptance cases, protected evaluator, scoped purchasing evidence, a pending approval and a worker-crash scenario. Run the same packet through each candidate integration. Keep the starting artifact, task scope and measurement policy fixed.

A useful comparison asks what the runtime owns and what your application must supply. Does it persist a session transcript, a resumable workflow state or both? Can a hook prevent an action at the required boundary? Where are credentials injected? Which code can write acceptance results? How does a model refusal reach your controller?

Do not compress those questions into one “supports agents” checkbox. A runtime can be excellent at interactive coding while leaving business approvals and durable effect reconciliation to the host application. That may be the right choice if you implement those responsibilities explicitly.

Compare mechanisms before products

These categories can overlap. A product can expose several layers, and a single application can combine them. The decision is which layer will own each required contract in your deployment.

The first gate is required behavior. Lower cost or a convenient API cannot compensate for missing authority enforcement in a workflow that changes customer records.

Normalize the model boundary explicitly

Your adapter should distinguish completed output, refusal, incomplete output and transport failure. A truncated JSON object is not a partial action to execute. A model timeout can leave usage unknown even though no customer tool has run. Keep provider request IDs and bounded usage metadata for diagnosis without copying credentials into traces.

The companion's adapters.py defines a provider-neutral reply contract and tests those states with recorded replies. It does not pretend to be a tested integration with every current SDK. The dated framework worksheet identifies the live checks to perform: actual instruction loading, timeout behavior, tool-call normalization, permissions, process restart and retained evidence.

For a real adapter, enforce the transport deadline where the call is made. A wrapper that catches TimeoutError cannot interrupt an underlying function that never raises it. Test cancellation during an actual call and inspect whether any already-issued tool effect still needs reconciliation.

The Premium companion also includes http_model.py, a complete optional HTTP adapter with local-service tests and a separate live command. It makes one bounded request and returns a proposal plus available usage; it never executes that proposal. Its tested protocol and untested external-provider boundary are documented in FRAMEWORKS.md.

Do this before moving on

Two synthetic runtime candidates both pass the threshold repair. Candidate A can persist a transcript but loses its workflow checkpoint on process death. Candidate B resumes the graph but requires you to implement the buyer-approval adapter. The customer requires unattended recovery and exact buyer approval before amendments.

Deliverable: write a fit decision and the minimum missing work for each candidate. Do not invent a winner from the incomplete evidence.

Worked review. A needs durable workflow state and demonstrated restart recovery; a saved transcript alone does not establish either. B needs an authenticated, proposal-bound approval adapter and service-side enforcement. Evaluate whether those missing boundaries can be implemented and operated within the engagement. Neither passes the full contract yet.

Change the input. The task becomes an interactive, read-only coding assistant where a human owns each session and no business write occurs. Durable approval graphs may no longer be necessary. Reassess the required packet while preserving repository permissions and honest verification. The right runtime can change because the task changed, not because one product became universally superior.

The field notes below were checked against official documentation in October 2026. Treat them as a starting map and verify the specific version and configuration you deploy.

Go deeper

• Agent framework failure modes: Compare runtime responsibilities without confusing an SDK feature with an enforced contract.
• Harness design decisions: Use the design vocabulary while reading the survey’s evidence limits carefully.
• Harness experiments: Evaluate planning, tools and context choices on the model and workload you actually use.
• Production AI Agents: Extend the adapter comparison to a complete support-agent implementation.

Key takeaways

• Choose a runtime by a portable packet of required behavioral evidence.
• Normalize refusal, incomplete output and timeout separately from valid actions.
• Recheck product-specific assumptions against the deployed version and configuration.

Check yourself

Answer before you look. Recalling it is what makes it stick; recognising it does not.

1. 1A runtime saves every conversation. Does that establish durable workflow recovery?

   Yes, a transcript is equivalent to a checkpointNo, the workflow state, replay and effect contracts still need demonstrationOnly if the transcript is long

2. 2What should a provider adapter do with incomplete JSON output?

   Answer it, then reveal