Context Window Management for FDE Agents

TL;DR: A frontier context window is a fixed budget you spend on the customer's token bill, not free space to fill. The FDE job is four moves: write context out to durable storage, select only what the step needs, compress what you keep, and isolate one tenant or tool from another. Get this wrong and the agent is slow, costly, and confidently wrong by turn twenty.

The FDE version of the problem

The context window concept covers what the window is and why input and output share it. This concept is about the harder job on top of that: running an agent thousands of times a day, at a customer, against their data, on their invoice.

In a lab you tune a prompt once and move on. Embedded with a customer, the same agent loops for dozens of turns per task, reads from a private corpus you do not control, and bills every token to an account someone in procurement is watching. A long-running agent's context grows on every step: each tool call appends output, each turn appends history. Left alone it climbs past the point where the model attends well and past the point where finance stops asking questions. Context management is what keeps the deployment both accurate and affordable, which is why it shows up in staff and principal loops as a systems question, not a prompting tip.

The growth is arithmetic before it is a feeling. Take a hypothetical agent with a 3,000-token system prompt and a 2,500-token tool catalog, where each turn adds about 450 tokens of exchange and a 2,000-token tool result. Left alone, the window holds 7,950 tokens at turn 1, 17,750 at turn 5, 30,000 at turn 10 and 54,500 at turn 20; the tokens billed across those twenty turns total 624,500, because every turn re-reads everything before it. Prune each tool result to the 300 tokens the next step uses and summarize turns older than five into an 800-token digest, and turn 20 runs on 10,050 tokens with 189,500 billed across the run, about a third of the cost, with the last five turns still verbatim (all computed). The customer notices the third; the model notices that the instruction it needs is no longer buried under forty thousand tokens of tool output.

The four moves

The working taxonomy is write, select, compress, isolate. Each is a lever on what actually lands in the window on a given step.

The spine above is the same four moves in the order a single step runs them, with the reason each one earns its place.

Write means persisting context outside the window instead of carrying it inline. A scratchpad file, an external memory store, or a structured run log holds the state the agent would otherwise re-paste every turn. At a customer this doubles as the audit trail a stakeholder can read after the fact, so it earns its keep twice.

Select means loading only what this step needs. That is RAG over the customer corpus, tool schemas pulled on demand rather than all at once, and few-shot examples chosen by similarity to the current input. The failure mode is dumping the whole knowledge base into the prompt to be safe: you pay to process thousands of irrelevant tokens and bury the passage that mattered.

Compress means shrinking what you do keep. Summarize old turns once they fall out of active use, prune a tool's JSON response to the three fields the next step reads, and collapse a retrieved 6,000-token document to the 400 tokens that answer the question. Uncompressed history is the single most common reason a long agent run gets slow and starts drifting.

Isolate means splitting work so contexts do not bleed. Scoped sub-agents, per-tenant sessions, and separate windows for a noisy tool keep one customer's data out of another's prompt and stop a chatty API from crowding out the instructions. In a multi-tenant deployment isolation is a data-governance requirement that guardrails enforce, not a performance nicety you add later.

Why interviewers probe this

It is a fast proxy for whether you have run an agent on someone else's bill. A candidate who reaches for a million-token model and stops there has read the spec sheet but never watched a per-turn cost curve. The follow-up interviewers hold in reserve: "your agent's context grows every turn, and by turn twenty it is slow and answering from stale history, what do you cut and in what order?" The strong answer names a per-step budget, compresses history first, selects over the corpus instead of dumping it, and pins the system prompt and the current task so they never get trimmed. Said as a budget: 2,000 tokens for the system prompt, 1,500 for the tools this step can call, 4,000 for retrieved context, 3,000 for the last few turns verbatim, 1,000 for the running summary, and the rest of a 16,000-token target reserved for the reply. Each line has an owner and a trimming rule, and the two that never get trimmed are the first and the current task. A budget with numbers is a design; "we manage context carefully" is a hope.

Common misconceptions

• "A huge context window means I can stop managing context." Cost and latency scale with tokens processed, so a bigger window is a bigger invoice, not a free pass. You still choose what goes in.
• "Selection is just RAG." Retrieval over documents is one case. Selection also decides which tools, which memories, and which examples load this step, each of which spends budget.
• "Isolation is premature optimization." In a multi-tenant customer deployment, keeping one tenant's data out of another's window is a compliance line, not an optimization you defer.
• "Compression loses information, so avoid it." Carrying every raw turn and full tool dump is what makes long agents slow and unfocused. Summarize deliberately and keep the source in the durable store if you need to reread it.

Key takeaways

• Treat the window as a per-step budget you allocate, measured against the customer's real token cost, not space to fill because the model allows it.
• Write state to durable storage, select only what the step needs, compress what you keep, and isolate tenants and noisy tools from each other.
• The reserve interview follow-up is a long-running agent whose context has bloated; answer with an explicit budget and a compress-then-select order.
• Isolation in a multi-tenant deployment is a data-governance requirement, so design it in from the first customer, not after the second.