IN THIS BREAKDOWN · 10 SECTIONS
You can read 15% of this breakdown.
The paper in one minute
An agent harness contains a completion check because an earlier version kept stopping without running tests. After a model upgrade, that check starts causing loops. Deleting it may restore progress while bringing back the original failure. The code tells you what the check does; it may not explain the evidence that justified it.
Self-Improvement Memory, or SIM, stores that evidence for the agent that edits the harness. Each recurring problem has a history of selected modifications, their rationale, predicted and observed outcomes, and the model and benchmark used to evaluate them. When optimization resumes on a different setting, the optimizer inherits both the evolved harness and this record.
The distinction is easy to miss: this memory belongs to the harness optimizer. The task-solving model's weights remain frozen in the main experiments. The method is also different from Meta's similarly named Self-Improving Memory research, which optimizes task-agent memory instructions and then distills demonstrations into model weights.
The strongest repeated result is a paired coding study. Starting from the same evolved harness within each pair, continuation with SIM finishes ahead on average. One pair goes the other way. The paper supports testing evidence-rich optimization history; it does not establish that an agent will keep improving indefinitely.
Research notes: the problem, approach and results
mean paired final-score advantage over harness-only continuation (Table 3)
independent coding source-run pairs favor SIM (section 4.4)
95% paired t interval for that coding advantage (section 4.4)
A useful harness change can become harmful after a model or benchmark change, while the surviving code no longer explains why it was introduced.
Carry problem-indexed edit histories into continued optimization, preserving rationale, predictions, measured outcomes and evaluation context.
The paired coding experiment favors SIM on average, with one negative pair. Most other conditions use one optimization run and are descriptive.
Why it matters to an FDE
Customer agents accumulate special cases. A retry rule handles one tool's transient errors, a context filter suppresses noisy files, and a verifier blocks a recurring false success. A new model or a changed customer workflow can alter the value of all those decisions.
SIM suggests a useful operational habit: save the observation behind a change alongside the change itself. Record the conditions under which it helped and the failures it did not fix. That gives a later optimization run a basis for revising a workaround instead of either preserving every rule or starting over.
The evaluation design matters as much as the memory format. Compare continuation from the same starting harness under equal search budgets. Otherwise, a better initial artifact or more evaluated proposals can masquerade as a memory benefit. Our Meta-Harness breakdown explains the broader search-over-harnesses setting that this paper extends.
Understand the mechanism. Try it on a real problem.
Unlock this breakdown and 294 more papers.
Premium opens every paper in full, plus every answer, concept, course and guide on the site. One payment for six months of access.
6 months · One payment · No auto-renewal
- ◇1 architecture diagram
- </>2 runnable code blocks
- ▦3 results tables
- 03Key takeaways
Separate optimizer memory, ordinary run history and evidence of transfer
Free account - 04How it works
Trace a problem record through source optimization and target continuation
Premium - 05The numbers that matter
Read the paired study separately from single-run transfer charts
Premium - 06Try it yourself
Recompute the paired result and preserve a selected edit's failed prediction
Premium - 07Where it breaks
Compatible interfaces and trustworthy feedback limit what the memory can transfer
Premium - 08Using it on a deployment
Compare equal-budget continuations before adopting optimizer memory
Premium - 09Interview angles
Explain the experimental controls and why retaining code is not retaining evidence
Premium - 10Read next
Connect optimizer history to harness search, agent memory and regression gates
Premium
