Context Mode Review: Does Keeping Tool Output Out of Your Coding Agent's Context Actually Save Tokens?

Context Mode is a plugin that keeps the raw output of a coding agent's tool calls out of its context window and lets the agent search or summarize it instead. Its headline, 99.4% of output bytes saved, holds up: we recomputed 99.36% from the project's own benchmark file. But that number counts bytes the model reads from large outputs, not tokens or dollars per session. The project's own A/B tests in Claude Code show session tokens 5.6% higher than no plugin on small tasks and 3.8% lower on large ones, with list-price cost down 44% on the large tasks. Install it if your sessions read big logs, JSON and pages; skip it for short sessions with small outputs.

What Context Mode does

A coding agent reads files, runs commands and fetches pages, and every result lands in the context window. A 60 KB file read or a 200 KB git log crowds out the instructions and history the agent needs, and is re-read on every later turn. Context Mode, by Mert Koseoglu, attacks that in four steps:

1. Hooks intercept heavy calls. Before a tool runs, a PreToolUse hook checks it against a fixed order of rules: your deny list, web fetches, curl and wget, a full-file read over 16,384 bytes, and one-file shell dumps such as cat. A read with a line range passes through.
2. The hook names a replacement call. Instead of the raw output, the agent is told which ctx_* tool to use, for example ctx_execute with the command and its intent.
3. Work happens outside the context. ctx_execute runs a short script the agent writes, and only what the script prints comes back. The project calls this "Thinking in Code": count the failures in a 2,000-line syslog instead of reading it.
4. Everything is indexed. Full outputs go into a per-project SQLite FTS5 database in ~/.context-mode, so ctx_search can retrieve the exact lines later, and ctx_resume restores a state block of at most 2,048 bytes after the agent compacts its history.

It ships nine MCP tools and installs into ten agents, including Claude Code, Codex, Gemini CLI, Qwen Code, Copilot CLI, VS Code, Kiro, OpenCode, Pi and ZCode. The repository passed 33,900 GitHub stars this week, with v2.1.10 released on 12 November 2026.

Here is what that looks like for a single large read, using the byte counts from one row of the project's benchmark:

A typical replacement is a short ctx_execute call. The agent asks a question of the data instead of reading it. The argument shape is the one the README documents; the command, path and intent are our example:

{
  "language": "shell",
  "code": "grep -c ' ERROR ' logs/app.log; grep ' ERROR ' logs/app.log | tail -3",
  "intent": "How many errors, and what are the latest three?"
}

Only the count and three lines come back. The full log stays in the local index, so a later ctx_search for a request ID can still find the line.

This is the "write" and "select" half of context window management: persist what might be needed later outside the window, and bring back only what this step needs.

The 99.4%, recomputed

The README's rule for every headline number is stated precisely: rows with at least 10 KB of raw output, median of five runs per row, bytes the model sees against bytes the raw output would have added. We downloaded the 1 November 2026 results file (66 scenarios, 52 with byte metrics) and recomputed:

The headline reproduces. Two caveats travel with it. Tokens in the README cards are bytes divided by four, an estimate the results file labels as such: "70K to 817 tokens" is 281,261 bytes to 3,266 bytes, not a tokenizer count. And the rows are scripted tool calls on fixed inputs, not model sessions, so they measure the mechanism, not what the agent does next.

We did not rerun the suite ourselves; this review rests on the published results file, the A/B tables in bench/RESULTS.md and the source.

What the project's own A/B tests show

Here the project deserves credit. Its bench/RESULTS.md runs real Claude Code sessions with and without the plugin and reports them plainly. The latest summary table, "Final A/B 3" from 21 October 2026, used Claude Sonnet 5.5 at low effort, three runs per task per arm, and scripted correctness checks:

Three things explain the gap between 99% and roughly zero:

• Fixed overhead. The plugin's tool schemas and hook text add about 2,000 tokens to every session. On a short task with small outputs nothing offsets them.
• The output was never most of the session. A session's token count includes the system prompt, tools and history re-read on every turn. Shrinking one tool result from 4.6 KB to 0.6 KB removes about 1,000 tokens from a session of roughly 57,000.
• Extra turns. An earlier run in the same file found the model spent additional turns calling ctx_* tools on two large tasks, pushing their tokens 29% and 74% above no plugin. The tuning between that run and Final A/B 3 removed most of the penalty.

Cost fell much faster than tokens on large tasks. That fits how the file counts tokens: the session total includes cached prefix reads, which are billed at a fraction of fresh input, so a smaller fresh payload can cut the bill while the total barely moves. The file also lists what it does not show: three runs per task, one model, one agent version, fixtures of 0.3 to 3 MB, single-task sessions. Longer sessions with many large outputs should do better, since the fixed cost is paid once, but that is a hypothesis the file does not test.

Two things to know before installing

The sandbox is not a security boundary. The README is explicit: code run through ctx_execute runs on your machine as your user, with your files, your network and your environment, minus preload variables such as NODE_OPTIONS. "Sandbox" means the output stays out of the model's context. Keep your agent's permission prompts on, and treat the plugin's hooks as code that runs on every tool call, which is the same supply-chain surface our plugin hooks question covers.

The license is source-available. Context Mode uses the Elastic License 2.0: free to use, modify and share at work, but you may not offer it to others as a hosted service. The README is careful to say "source open" rather than open source. The hosted Context Mode Gateway, a paid service with its own account, extends the idea to whole conversations; with it on, requests pass through its server. The plugin alone makes four network calls, all started by you or your agent, and the project guards that list with a test that flags any new network call site.

Pros and cons

How we would evaluate it on your own sessions

Do not install on the strength of either number. Pick five real tasks your team runs, ideally the ones with big outputs, and run each three times with and without the plugin on the same agent and model. Record pass rate, session tokens, fresh input tokens and cost per task. The project ships the harness for exactly this (bench/live/ab-quick.mjs). If cost falls and pass rate holds, keep it. That small, paired comparison is the same discipline as an eval regression suite, applied to tooling.

The interview angle

"The agent's context fills up with tool output" is a common RAG and agent design prompt, and Context Mode is a clean worked example of one good answer: intercept, process outside the window, index, retrieve on demand, and restore a compact state after compaction. The stronger answer adds what its benchmark shows. Name the metric you would optimize (cost per successful task, not bytes saved), the fixed overhead any tool layer adds, and the risk that the model spends turns on the new tools. A candidate who says "99% fewer tokens" and stops has repeated a headline; one who separates bytes, tokens and cost has done the analysis.

Figures as of 12 November 2026: Context Mode v2.1.10, benchmark file of 1 November 2026, A/B tables of 21 October 2026. Check the repository for newer runs before quoting them.