Context Language Models

A standard language model appends: each turn's output is added to the context, so the next context is the old one plus the new tokens. Agents on long tasks hit the window limit, and the fix so far has been a harness. Codex-style compaction summarizes the history at a fixed threshold; tools such as Self-Compact, ACM and Context Folding give the model a few predefined actions. In each case a human picked the strategy.

Rulin Shao, Pang Wei Koh and colleagues at the University of Washington, Meta Superintelligence Labs, MIT and Trillium Labs propose Context Language Models (CLMs), models that manage their own context. Formally, the next context is any function of the current one the model chooses, rather than the current one plus new tokens. The implementation is simple: the live context is mirrored to a file whose path is in the system prompt, the model edits that file with ordinary Bash, and each edit is synced to the server before the next turn. If the model does not edit, generated tokens are appended as usual. Multiple agents get multiple context files.

The paper reports emergent behaviors: an orchestrator keeping a scoreboard of subagents through 163 in-place edits at 6 to 8K tokens, a new "notes" chat role the model invented, and a compact_turns helper function it defined and called 37 times.

It then adds four pieces: ContextBench, four synthetic tasks that test context management alone (separate from the coding-retrieval benchmark with the same name); prefix-reuse FLOPs, a cost metric that charges for re-prefilling after an edit; in-context steering and skill evolution, plus an RL recipe with a success-gated efficiency advantage; and Suffix Cache Reuse (SCR), an SGLang patch that keeps cached states after an edit. Across coding, deep research, math optimization and day-long repository work, CLMs beat or matched the harnesses at lower compute in most settings.

Every long-running agent you deploy has a context strategy, usually inherited from a framework default: summarize at 75% full, truncate old tool output, or nothing. This paper argues the model can do better if you let it, and gives evidence on public benchmarks.

Smaller harness, more model. CLM keeps the harness minimal: one file, Bash, a reminder near the limit. The strategy lives in the model and in instructions, which you can change per customer with one sentence. The paper shows one-sentence instructions changing when and how the agent compacts.

Edits have a serving cost. Prefix caching only reuses an unchanged prefix. Rewriting something early in the context forces everything after it to be prefilled again. A context strategy is also a cache strategy, and the paper's metric makes that cost visible.

Editable context is an attack surface. If the model can write its own context, injected text can persist by being copied into notes, and the paper cites a report of a model inserting unauthorized instructions into its own compaction summary. Log context edits like any other privileged write.

It needs a capable model. The 9B model edited less, filled its context more and needed RL to catch up. On small models, a fixed harness may still win.