IN THIS BREAKDOWN · 10 SECTIONS
You can read 15% of this breakdown.
The paper in one minute
An agent is a model plus a harness: the system prompt, standing rules, memory files, skills, helper scripts, subagents and hooks around a frozen model. Ruiyao Xu, an intern at Google Cloud AI Research, and colleagues there ask whether that harness can be improved while the agent works, task by task, without any answer key, and whether the improver itself can get better at the job as tasks go by.
The method, Harness² (Harness squared), has two loops. Inside each task, the current harness runs once to produce a reference. Two editors read that trajectory and each write a candidate harness: one changes how the agent works (planning, tool use, checking its output), the other adds what a practitioner of the domain would know. Both candidates run on the same task. A composer reads the three runs side by side and writes the successor harness from the differences it can point to, or keeps the old harness. The successor runs and its output is delivered. No score, rubric or grader is visible at any stage; the pipeline refuses to hand an editor a directory containing one.
The second loop runs after each task. A reflector records which edits changed behavior and writes at most one lesson, only when an execution contrast supports it. That lesson and the adopted harness go into a library that later editors read. The next task still starts from the common base harness.
The evaluation covers three professional benchmarks (WorkBuddy-Bench for code, web and office deliverables, JobBench for whole job tasks, and Harvey's Legal Agent Benchmark) with two frozen executors, Gemini 3.5 Flash and GLM-5, each under two stock harnesses, OpenCode and Codex. The editor is Claude Opus 4.8. Code is linked from the paper at github.com/google-research/harness2.
Research notes: the problem, approach and results
points, GLM-5 under OpenCode on WorkBuddy-Bench average, one step (Table 1)
WB-code score after editing the harness blind, without runs or library, from 73.4 (Figure 7)
settings where one step beat the base harness's mean (Tables 1 and 2)
Harnesses (prompts, skills, scripts, subagents, hooks) are tuned by hand or searched offline against a fixed training set. Deployed agents meet new tasks with no answer key, so there is nothing to score a harness edit against.
For each task, run the current harness, let two editors propose edits from different angles, run both, and compose a successor from the behavioral differences. After the task, keep the adopted harness and any lesson an execution contrast supports in a library that guides later edits.
One step improved all twelve executor, harness and benchmark settings over the base harness, by 1.2 to 11.8 points. Five candidates per scope added 2.6 to 6.7 more. Margins over the strongest baseline were mostly within run-to-run noise.
Why it matters to an FDE
Most customer agents never get a labeled eval set big enough to tune against. Tasks arrive one at a time, each a little different, and the person who could judge the output is busy. This paper is a careful attempt at improving the harness in exactly that setting, and its design choices are useful even if you never run the whole pipeline.
Run the edit before you trust it. The most practical result is a negative one. An editor that rewrote the harness without running anything and without the library made it worse: WB-code fell from 73.4 to 63.2 and WB-web from 69.9 to 65.0. Edits written cold read well and hurt. Every edit in the working method traces to a moment in a trajectory.
Compare behavior, not verdicts. Without a grader, the evidence is how two runs of the same task differed: one reconciled the files, the other did not; one reopened its output, the other trusted the write. That kind of contrast is something a reviewer on a customer team can check by reading.
Keep the lessons, not just the harness. Each library entry is a working harness plus a note on when and why it helped. The authors removed the library and lost 3.1 points on WorkBuddy-Bench and 1.8 on JobBench.
Read the gains with the noise next to them. Every setting improved over the base harness, but most margins over the strongest competing method are smaller than the run-to-run spread of three runs. That is the right standard to hold your own harness changes to.
Understand the mechanism. Try it on a real problem.
Unlock this breakdown and 199 more papers.
Premium opens every paper in full, plus every answer, concept, course and guide on the site. One payment for six months of access.
6 months · One payment · No auto-renewal
- ◇1 architecture diagram
- </>1 runnable code experiment
- ▦3 results tables
- ?5 interview questions with answers
- 03Key takeaways
The loop, the evidence it uses, the results by benchmark, and the misreadings to avoid
Free account - 04How it works
The two loops, the eight components, the prompts' rules, and the evaluation protocol
Premium - 05The numbers that matter
The main tables, what recursion added, the ablations, and what it cost
Premium - 06Try it yourself
Recompute the headlines, count the runs each k costs, and test which wins beat the noise
Premium - 07Where it breaks
The judge overlap, the noise, the cost, and the failure cases the authors document
Premium - 08Using it on a deployment
Adopting the loop, or just its rules, for a customer agent with no answer key
Premium - 09Interview angles
Five questions this paper prepares you for, with answer hints
Premium - 10Read next
Where to go from here on this site
Premium