Choose proof that reaches the user-visible behavior

TL;DR: Choose verification that reaches the behavior the customer actually uses. Unit checks, service integration and browser observation cover different failure surfaces; tie each result to the tested artifact and report the important paths that remain untested.

Where you are. Northstar's runtime can recover. You are deciding what evidence is sufficient to accept a coding change or a procurement outcome.

A correct function can still produce a broken product

The threshold predicate passes at 99999, 100000 and 100001 cents. The browser still displays “Below review threshold” for every amount because the UI reads the wrong response field. The unit test is correct about the function and incomplete about the product.

Now reverse the situation. A browser test clicks one order at 100001 cents and sees the expected label. The equality defect at 100000 remains. The browser test is correct about the observed path and incomplete about the numeric boundary.

Neither result replaces the other. Use fast deterministic checks for the calculation, integration checks for the request and response contract, and browser checks for the actual interaction. Add property or adversarial tests where the behavior has a large input space. Test selection follows the failure surface, not a belief that one level subsumes every other level.

Design an evidence ladder for one task

For T-17, the verification packet can be small and specific:

The packet identifies the candidate revision, fixture version and environment. The verifier should inspect an immutable candidate snapshot so a passing result cannot accidentally be labeled with bytes edited after the check ran. If the application has multiple deployed components, identify the relevant versions and configuration together.

The checks converge into one acceptance decision without pretending they prove the absence of defects. A result is strongest when a reviewer can say what it would have caught and what it would not.

Exercise the product as a sequence

For the synthetic workbench, open the page, enter 100000 cents, run the check and observe “Buyer review required.” Change the amount to 99999 and observe “Below review threshold.” Then ask about the purchasing policy, open its citation, copy the answer and ask an unsupported shipping question.

Check empty inputs and failure states as well. The answer history should retain the earlier question. An unavailable clipboard should produce an honest error. A delayed response should remain associated with the submitted question even if the input field changes while waiting.

These details matter because a user experiences the sequence, not the isolated function. A technically correct answer hidden behind a broken copy control or mislabeled question is still a poor workflow. At the same time, a polished page can display a plausible answer from the wrong tenant. Keep usability and authority checks separate in the acceptance record.

Do this before moving on

Use these supplied observations:

• Predicate checks at the three boundary amounts pass.
• HTTP returns needs_review: true for 100000.
• The browser reads review_required and shows the below-threshold label.
• A screenshot exists, but no interaction was recorded.

Deliverable: identify the smallest repair and the checks needed afterward.

Worked review. Correct the UI's response-field mapping, then rerun the HTTP contract and both browser boundary journeys against the candidate. Preserve the passing predicate evidence if its exact artifact and dependencies remain unchanged, or rerun it if that is simpler. A screenshot alone does not show that entering an amount and clicking the action produced the visible state.

Change the input. The browser journey passes, but the user cannot copy the answer. The acceptance decision depends on the stated task contract. If copy is a required part of the evidence workflow, the feature remains incomplete. Record the failing interaction and repair it; do not average a required usability failure away with passing numeric checks.

When a check cannot run, record “unverified” with the reason and owner. That is more useful than an unqualified completion claim and gives the next worker a precise starting point.

Extend the journey: history that survives a reload

The supplied workbench keeps answers in the current page only. To extend it, define a history record with an immutable request ID, submitted question, answer, source references, completion timestamp and optional parent request ID. Capture the question before awaiting the response. A follow-up should point to the selected prior answer; it must not silently use whichever answer happened to finish last.

Exercise: design storage and verify two responses on opposite sides of midnight, a reload, a changed policy and a storage failure. Use synthetic questions only. State whether date groups follow UTC or the viewer's time zone.

Worked solution: persist the record after a successful response, render stored text as text, and preserve citation identity through reload. Group using the declared time zone rather than slicing an arbitrary local timestamp. Show the answer's historical policy revision; revalidate access and freshness before using it as evidence for a new action. On storage failure, retain the visible answer and report that history was not saved. A saved answer is not a fresh permission grant.

A minimal persistence check runs without a browser:

import json
import sqlite3
import tempfile
from pathlib import Path
from datetime import datetime, timezone

with tempfile.TemporaryDirectory() as folder:
    path = Path(folder) / "history.db"
    db = sqlite3.connect(path)
    db.execute("CREATE TABLE history(id TEXT PRIMARY KEY, parent TEXT, question TEXT, answer TEXT, sources TEXT, completed TEXT)")
    db.execute("INSERT INTO history VALUES(?,?,?,?,?,?)", (
        "Q-1", None, "When is review required?", "At least 100000 cents.",
        json.dumps([{"id": "P-4", "revision": 4}]), "2026-10-17T23:59:59+00:00"))
    db.execute("INSERT INTO history VALUES(?,?,?,?,?,?)", (
        "Q-2", "Q-1", "Does that include equality?", "Yes, in revision 4.",
        json.dumps([{"id": "P-4", "revision": 4}]), "2026-10-18T00:00:01+00:00"))
    db.commit()
    db.close()
    reopened = sqlite3.connect(path)
    rows = reopened.execute("SELECT id,parent,sources,completed FROM history ORDER BY completed,id").fetchall()
    reopened.close()
    days = [datetime.fromisoformat(row[3]).astimezone(timezone.utc).date().isoformat() for row in rows]
    assert days == ["2026-10-17", "2026-10-18"]
    assert rows[1][1] == rows[0][0]
    assert json.loads(rows[1][2])[0]["revision"] == 4

Change the input: policy P-4 advances to revision 5 after Q-1. Preserve Q-1 as a historical answer, but fetch and verify revision 5 before answering a new operational question. Check the browser's reload, citation, copy and keyboard flows separately; a database assertion does not establish that the UI works.

Go deeper

• Evaluation gates: Connect task-level proof to the release conditions the customer accepts.
• The HTTP 200 lab: Practice distinguishing transport success from the intended user-visible outcome.
• Verifier gaming question: Challenge checks that can pass while the application still fails its purpose.

Key takeaways

• Use complementary checks for calculation, service wiring and user interaction.
• Identify the exact candidate and scope behind every passing result.
• Keep missing evidence and required usability failures visible in acceptance.