Sierra RAG & Agent System Design interview questions

Arguing against agents reads senior; it's an explicit rubric signal at AI labs. Learn the cost ladder from single call to autonomous loop, and the three conditions that actually justify an agent.

Sierra's signature design exercise. The grading hinges on three things most candidates underweight: tool API shape, the read/write trust boundary, and eval cases written before the agent. Walkthrough inside.

A Sierra-signature scenario where 'put it in the system prompt' scores zero. The winning answer is defense-in-depth with a measured catch-rate per layer, plus an honest number for what still gets through.

Perceived latency is a design surface, not an infrastructure detail. Covers progressive disclosure, SSE vs WebSockets, resumability, and the mid-stream failure question most candidates have never thought about.

Memory is a retrieval problem plus a write-policy problem plus a privacy problem. Most candidates only design the first. Covers memory taxonomies, the curation pipeline, and the leak vectors that fail the question.

The question that separates demo-builders from production engineers. Idempotency keys, the saga pattern, dry-run modes, approval tiers: the distributed-systems toolkit applied to agents, with the trap most answers miss.

Serving 50 enterprise customers off one platform means four leak surfaces most designs miss, including the semantic cache and the eval set. A tier-by-tier isolation model with the noisy-neighbor math.

The most FDE question in the bank: every real deployment hits an unchangeable legacy system. The answer is an architectural principle, concentrate brittleness in one owned layer, plus what to do when the GUI is the only API.

Standard A/B machinery assumes a crisp conversion event; agents give you fuzzy, delayed, gameable outcomes. The metric hierarchy, the reopen-window trick, and the sample-size reality check that win this question.

Multi-agent is the most over-prescribed architecture in the industry. A staff-level answer is a decision framework driven by task structure and context limits, with the orchestration failure modes named before the interviewer asks.

Untrusted content plus write-capable tools is the lethal combination. Prompt-level defenses alone fail this question; the winning answer is an architecture where injection succeeding still can't move money.

Launch is the starting line. The staff-level capstone: how production traffic becomes eval cases, prompts ship like code, and a model deprecation becomes a Tuesday instead of an incident.

The staff-level reliability question for agent platforms. Refunds went out, emails sent, tickets closed, and now you have to triage at fleet scale without double-refunding. The recovery model interviewers actually score.

Most candidates reach for a smaller model first. The staff answer builds an explicit per-stage latency budget, parallelizes independent tool calls, and spends the time budget where the user actually feels it.

This is the lethal trifecta in one sentence, and most candidates reach for a better system prompt. The answer that scores breaks the trifecta architecturally, because no amount of prompting stops indirect injection once all three capabilities are present.

The data-isolation answer everyone gives misses the harder surface: live OAuth tokens and API keys the agent uses to act in third-party systems. The model that keeps one tenant's credentials out of another's agent loop, and out of the LLM context entirely.

Hard budgets stop the bleeding; they don't fix the cause. The real fix is progress tracking plus semantic loop detection plus a planning layer that commits to a plan instead of re-deciding every step, validated on a fixed trajectory set offline before it ships.

Self-correcting RAG is easy to demo and easy to ruin. The discipline is a gate that grades evidence before drafting, a termination rule that is not just a loop cap, and a metric that proves correction adds facts rather than fluff.

A 500-page doc is well over 200k tokens, and even if it fit, stuffing it in would rot the context. The real design is a working set in context plus an external memory the agent retrieves from and compacts into as it goes.

An agent loop optimizes whatever the verifier rewards, so a weak verifier gets gamed: tests edited, outputs hardcoded, inputs memorized. The fix is a verifier the agent cannot author or edit, run on evidence in a clean environment, with held-out and adversarial checks that track intent.

The provider bill tells you the total and nothing else. This is a forensics question, and the answer is almost never a pricey model. It is that the transcript grows every turn, so a forty step run costs six times what everyone estimated.

A wrong answer inside one conversation dies with the conversation. A wrong answer written to memory becomes a retrievable fact that conditions every later run, and the agent then derives new conclusions from it. Deleting it is the part nobody designed for.

Nobody deployed anything is true of your repository and false of the system. The model provider, a tool behind an API, the knowledge base and the traffic mix all change without a commit. Segmentation comes before theory: a broken intent and a mix shift produce the same headline and need different fixes.

A nightly suite of fifty simulated conversations per intent catches a collapsed tool with certainty and a five-point drift almost never. Knowing which is which, and sizing each layer for what it can actually see, is the whole design. The grader checks outcomes, not transcripts, or the agent learns to sound resolved.

The pitch is a model that runs the loop. The design that survives production is a loop that code runs, with the model answering three narrow questions inside it: which tool, is this done, does this need a stronger model.

70 percent is an average over tasks and runs, and it hides the number the customer cares about: does the same request get the same right outcome every time? Two agents with identical averages can differ from shippable to unusable.

Single-session evals pass while users complain, because memory fails across sessions and at three separate stages. The answer is a planted-fact harness that names which stage lost the fact, plus the two tests teams skip: forgetting on request and leaking across users.

The customer wants the agent to practice before it can write to production. Design the task world, reset semantics, tool boundary and verifier, then explain when its scores justify training or release.

Trace an old fact from durable storage into the next model call. Separate conversation history, checkpoints, summaries, and verified tool outcomes.

The fast answer is to point an LLM judge at all 1,000 and report its average. That number arrives on time and tells the customer almost nothing. What do you deliver on Friday instead, and why is it more useful?

Two decisions hide in this question: where the failure is visible (records, trace or meaning) and when you need to know (before the customer sees it, or after). Most answers collapse them into 'add an LLM judge'. What does a placement that holds up look like?

The warm-up that filters out tutorial-level candidates. Interviewers aren't grading the diagram, they're grading whether you know the three places real pipelines fail and how you'd see it happening.

Everyone says '500 tokens with overlap.' The candidates who get hired explain why chunk boundaries are a recall problem, name three strategies, and describe the experiment that picks between them.

'Evals first' is the #1 hiring signal in FDE design rounds. This is the foundational version: what a 50-case golden set looks like, which metrics split retrieval from generation, and the launch gate that convinces interviewers.

The retrieval design question with a precise rubric: candidates who can say what each stage rescues, and what it costs in latency, clear it. Includes the RRF detail and the latency budget interviewers ask for.

A reported FDE design question that punishes one-size-fits-all answers. Each corpus has a different 'semantic unit': get the three designs, the parsing tools to name, and the eval that proves the split was worth it.

Pure vector search fails on the exact queries enterprise users ask most: IDs, error codes, product names. Here's the failure-mode framing that turns a definition question into a design answer.