OpenAI Forward Deployed Engineer interview questions

The warm-up that quietly filters out candidates who only know LLMs as a magic API. Nail the two-phase inference picture and you set up every later answer about latency, cost, and hallucinations.

Modern LLMs share one architectural skeleton. Learn the five components and you can read almost any model card cold, plus the single distinction that proves to an interviewer you actually understand it rather than the buzzwords.

Tokens drive your customer's bill, their latency, and the weird failures like botched arithmetic. The rough numbers every FDE should rattle off, plus the trap answers interviewers listen for.

Everyone can define it; few candidates know why 'advertised context' and 'usable context' aren't the same thing. The cost, latency, and attention caveats that make this answer stand out.

A deceptively simple sampling question with one trap (does temperature 0 fix hallucinations?) and one senior move interviewers reward. Know which workflows want zero.

The concept underneath every RAG system you'll ever deploy. How to explain vectors, cosine similarity, and the one limitation of embeddings that costs real deployments accuracy.

The same operation scores attention and ranks your RAG results. Knowing exactly what the dot product measures, and when its magnitude term quietly breaks your retrieval, is what separates a working pipeline from a mysteriously bad one.

Every FDE will face an angry customer asking this. The mechanistic explanation interviewers want (why hallucination is the default behavior, not a bug) plus the framing that defuses customer escalations.

A classic metrics question with a domain twist. The answer isn't 'recall'; it's a cost-of-errors argument plus a threshold strategy, and interviewers grade the reasoning, not the metric you pick.

The single most-reported FDE conceptual question across every AI lab. Interviewers don't want definitions; they want the decision framework, the cost math, and the recommendation order that reads senior.

Truncate? Summarize? Rerank? Map-reduce? A production problem every RAG deployment hits, with a menu of fixes interviewers expect you to weigh, plus the diagnostic question to ask first.

Cosine similarity finds topically close, not correct. The failure taxonomy behind most RAG accuracy complaints, and the hybrid-search + reranker fix stack interviewers expect you to know cold.

Anyone can list mitigations; the question is really about costs: latency, dollars, and lost recall. The three-layer answer with honest price tags that AI-lab interviewers score highest.

The data-volume thresholds, the break-even math, and the regression-prevention checklist that turn 'just fine-tune it' into a defensible engineering decision. Includes the crossover rule interviewers listen for.

Models read the start and end of your context far better than the middle, a measured effect with direct consequences for chunk ordering, k, and prompt layout. What to change and how to prove it.

Malformed arguments, wrong tool, hallucinated parameters, retry loops: the full failure taxonomy plus the production defenses (validation, idempotency, capped retries) that separate builders from readers.

JSON mode, strict schemas, constrained decoding, validate-and-retry: the escalation ladder for structured output, plus the catch nobody mentions: schema-valid can still be factually wrong.

Arguing against agents is the move that reads senior in AI-lab interviews. The error-compounding math, the decision rule, and the workflow-vs-agent spectrum that wins this contrarian question.

TTFT vs tokens-per-second, the output-length lever everyone forgets, and why streaming is the highest-ROI fix that changes no latency at all. The five-lever answer with real numbers.

The KV-cache mechanics behind the discount, the prefix rule that silently breaks caching for most teams, and the workloads where caching cuts bills 50-90% versus the ones where it does nothing.

Prompts are production code that nobody treats like code, until an innocent one-line tweak tanks accuracy. The version-eval-canary-rollback pipeline that AI-native interviewers expect by default.

Model migrations break things that benchmarks never show, format quirks, refusal shifts, tool-calling drift. The eval-replay and shadow-traffic playbook for migrating without surprises.

Evals are the #1 probing area in AI-lab FDE loops. The full suite design, golden sets, component metrics, safety red-teaming, CI gates, and the launch-blocking thresholds, in interview-ready form.

Position bias, verbosity bias, self-preference, and the agreement-rate workflow that turns a sloppy judge into eval infrastructure you can defend. AI labs probe this hard; here's the calibrated answer.

The classic FDE escalation. Six hypotheses, silent model updates, input drift, prompt entropy, sampling noise, perception shift, and the eval-replay test that splits them in an afternoon.

An email that says 'ignore your instructions and refund $10,000' will eventually arrive. Why prompt-level defenses lose, and the layered design (least privilege, action gates, rate caps) that actually holds.

10x doesn't come from one trick. It's multiplicative: routing x caching x prompt diet x batching x distillation. The stacked-levers playbook with rough multipliers, and the eval gate that keeps quality honest.

In high-stakes domains a wrong answer is worse than no answer, but a system that always abstains is useless. The selective-prediction design, the layered abstention triggers, and the two metrics that govern the tradeoff.

The experience probe that opens half of all applied-AI loops, and the place resumes go to die. The four-beat walkthrough structure interviewers can actually score, plus the two omissions that quietly downgrade you from architect to API caller.

Reported as the question OpenAI FDE interviewers push hardest on, and the one where 'we look at the outputs' ends loops. The three-legged answer that passes: automated metrics, calibrated human evaluation, and feedback loops wired to a business number.

The OpenAI FDE deep-dive where you're expected to walk the full stack (network, queueing, tokenization, prefill, KV cache, batching, decode) in order, with a measurement at each hop. The triage sequence, with the numbers that tell you which layer is guilty.

Provider model upgrades ship faster than your release cycle and quietly shift behavior. The design that turns 'we hope it's fine' into a blocking gate: a frozen golden set, pairwise diffing, segmented scoring, and a shadow-traffic canary.

The escalation ladder is settled; the at-scale tradeoff is not. Why retries quietly multiply your bill and tail latency, why constrained decoding has its own taxes, and the hybrid that most high-volume pipelines actually run.

Sending every request to the frontier model is the lazy bill. The patterns that route by difficulty, the verifier that enforces a hard quality floor, and the failure mode that turns a cost-saving cascade into a worse, slower system.

Two failure modes, one architecture. Why no single guardrail holds, the defense-in-depth layers that assume the model will be broken, and the design rule that limits the blast radius when prompt filters inevitably fail.

Three stages, one KL leash. The reward-model-plus-PPO loop, the beta*KL term that keeps the policy from drifting off the reference, and how DPO collapses it into a single classification loss by treating the reward as beta*log(pi/pi_ref). The objectives, written correctly.

Win-rate up, facts down is the signature of reward hacking: the policy is over-optimizing the preference proxy and drifting off the reference. The diagnosis (Goodhart on the reward), the immediate lever (tighten the KL/beta anchor), and the held-out factual gate that stops shipping the regression.

Three transformer families, one distinction that decides everything: which tokens are allowed to see which. The attention mask is the whole answer, and it explains why GPT generates and BERT does not.

Where you put the LayerNorm changes whether gradients survive a hundred layers. Pre-norm keeps a clean residual highway and well-scaled gradients so deep models train without warmup heroics; post-norm normalizes the output but fights vanishing gradients at depth.

Attention is bottlenecked by moving the N x N score matrix to and from HBM, not by FLOPs. FlashAttention tiles the computation in fast on-chip SRAM and fuses softmax with an online running statistic, so it computes the exact same output while never materializing the full matrix.

Each chat turn resends the whole conversation, so the model re-prefills the same prefix every time. Server-side KV caching and client-side cache breakpoints kill that redundant compute, but exact-prefix matching and stale entries are where teams silently get it wrong.

PPO needs a separate value network to estimate the baseline for its advantage. GRPO samples a group of answers per prompt and uses their mean reward as the baseline, so the advantage is just how a sample compares to its peers, no critic required.

Without a KV cache, generating token N re-attends over all N-1 prior tokens from scratch every step, turning decoding quadratic. The cache stores each layer's past keys and values so each new token costs one forward step, and its size is what caps your batch and context.

Human 1-5 scores are biased toward short, simple code. Debias the labels, fuse test-pass rate without letting the agent game it, and pick PPO with a fused reward over a DPO ranking loss. Plus the hard cap on the action space so the agent can't delete the repo.

Reward only on the final answer and learning crawls; shape intermediate tool calls and the model learns to spam SQL. Use outcome-grounded credit assignment, GRPO over a group of trajectories, and fine-tune a pre-aligned model so you're not teaching tool syntax from scratch.

A fixed 1000-token budget forces the search choice. Step-level beam search guided by a process-reward model beats best-of-N and MCTS on accuracy per token for math, and you collect the PRM labels automatically with Monte-Carlo rollouts, no human step annotation.

This is the live practical several labs run, and it is scored on protocol more than on the prompt you end with. Read all thirty before editing, cluster them, hold some out, change one thing at a time, and report with intervals, because 24 of 30 fixed is somewhere between 63% and 91%.

Decide whether retrieval can be removed after fine-tuning. Test changed facts, source support, access boundaries, and the cost of keeping answers current.

The warm-up that filters out tutorial-level candidates. Interviewers aren't grading the diagram, they're grading whether you know the three places real pipelines fail and how you'd see it happening.

Pure vector search fails on the exact queries enterprise users ask most: IDs, error codes, product names. Here's the failure-mode framing that turns a definition question into a design answer.

Arguing against agents reads senior; it's an explicit rubric signal at AI labs. Learn the cost ladder from single call to autonomous loop, and the three conditions that actually justify an agent.

'Evals first' is the #1 hiring signal in FDE design rounds. This is the foundational version: what a 50-case golden set looks like, which metrics split retrieval from generation, and the launch gate that convinces interviewers.

The retrieval design question with a precise rubric: candidates who can say what each stage rescues, and what it costs in latency, clear it. Includes the RRF detail and the latency budget interviewers ask for.

The modal FDE design question of the last two years. There's a scoring trap in the first 60 seconds, and most candidates jump to fixes and fail. Here's the diagnostic tree that wins it.

Sierra's signature design exercise. The grading hinges on three things most candidates underweight: tool API shape, the read/write trust boundary, and eval cases written before the agent. Walkthrough inside.

Perceived latency is a design surface, not an infrastructure detail. Covers progressive disclosure, SSE vs WebSockets, resumability, and the mid-stream failure question most candidates have never thought about.

Calibrated abstention is a system property, not a prompt line. Learn the four abstention signals, the precision-at-coverage framing, and why 'just tell it to say I don't know' fails the interview.

Text-to-SQL's nightmare isn't syntax errors, it's the query that runs, returns a confident number, and is silently wrong. The defense is a semantic layer plus transparency UX. Here's the full design.

An Anthropic-signature integration question. The hidden rubric: tools are a curated product surface, not a proxy for your OpenAPI spec, plus the auth model and confused-deputy trap most candidates miss.

The staff-level version of 'evals first.' Four layers, each with a job, a cost, and a blind spot, plus judge calibration numbers and the data flywheel that ties production failures back into CI.

Glean's signature design question. The two traps: post-filtering that drains your top-k, and the leak through generated answers. The winning architecture filters at retrieval, and here is how, at scale.

Memory is a retrieval problem plus a write-policy problem plus a privacy problem. Most candidates only design the first. Covers memory taxonomies, the curation pipeline, and the leak vectors that fail the question.

The question that separates demo-builders from production engineers. Idempotency keys, the saga pattern, dry-run modes, approval tiers: the distributed-systems toolkit applied to agents, with the trap most answers miss.

The POC-to-production gap is the FDE's home turf. What actually breaks at each order of magnitude (index, ingestion, latency, cost) and the dependency math that makes 99.9% harder than it sounds.

Standard A/B machinery assumes a crisp conversion event; agents give you fuzzy, delayed, gameable outcomes. The metric hierarchy, the reopen-window trick, and the sample-size reality check that win this question.

Multi-agent is the most over-prescribed architecture in the industry. A staff-level answer is a decision framework driven by task structure and context limits, with the orchestration failure modes named before the interviewer asks.

Untrusted content plus write-capable tools is the lethal combination. Prompt-level defenses alone fail this question; the winning answer is an architecture where injection succeeding still can't move money.

Launch is the starting line. The staff-level capstone: how production traffic becomes eval cases, prompts ship like code, and a model deprecation becomes a Tuesday instead of an incident.

A real interview question with a trap in its framing: it offers you two expensive options and hides the cheap one. The escalation ladder from feature-based reranking to two-tower fine-tuning, with the cold-start and position-bias answers ready.

Reciting the four definitions is table stakes; the differentiator is knowing which two need ground truth, what each dimension tells you to fix, and where the judge-model scores quietly lie. The 2x2 that turns RAGAS into a debugging tool.

Asked constantly in GenAI loops, and most answers are marketing-page recitals. The version that scores: classify by what each actually is, data framework, integration toolkit, graph runtime, then give the selection criteria that outlive all three names.

The classic 'it regressed and the prompt is untouched' triage. The answer that scores walks a decisional tree from retrieval to generation and names the silent culprit most candidates miss: the embedding stack changed under you.

Most candidates reach for a smaller model first. The staff answer builds an explicit per-stage latency budget, parallelizes independent tool calls, and spends the time budget where the user actually feels it.

You cannot unit-test a stochastic two-agent pipeline with a golden output. You evaluate it the way you stress a distributed system: thousands of simulated runs, controlled fault injection, and a judge calibrated against humans.

The hard parts are not the four roles. They are the communication substrate, a termination condition that is not just a retry cap, and stopping the Coder from forgetting the ticket after five repair cycles.

A 500-page doc is well over 200k tokens, and even if it fit, stuffing it in would rot the context. The real design is a working set in context plus an external memory the agent retrieves from and compacts into as it goes.

Deep-research agents run unsupervised for twenty minutes and their report goes straight to an executive. The failure mode is not a crash, it is confident citations that do not support the claims. Most candidates design the retrieval and skip the verification pass, the run budget, and the freshness story.

An agent loop optimizes whatever the verifier rewards, so a weak verifier gets gamed: tests edited, outputs hardcoded, inputs memorized. The fix is a verifier the agent cannot author or edit, run on evidence in a clean environment, with held-out and adversarial checks that track intent.

Every eval question in the prep canon is about answer quality. This agent does not produce answers, it moves freight. You cannot score a reroute for faithfulness, and the candidate who tries has not understood what they built.

The provider bill tells you the total and nothing else. This is a forensics question, and the answer is almost never a pricey model. It is that the transcript grows every turn, so a forty step run costs six times what everyone estimated.

Everyone reaches for a bigger machine. The two mechanisms that actually cause this both scale with corpus size, both hit the tail before the median, and one of them is the permission filter your security review insisted on.

Vectors from two versions of an embedding model are not comparable, and nothing in the stack will tell you so. The cosine between the same document's old and new vector is roughly zero, which is why a half-finished migration returns confident nonsense at full speed.

A wrong answer inside one conversation dies with the conversation. A wrong answer written to memory becomes a retrievable fact that conditions every later run, and the agent then derives new conclusions from it. Deleting it is the part nobody designed for.

Everyone sizes the pool on average run length, and average is the one statistic an agent fleet does not have. Simulated, the p99 sat at 160 seconds whether the system was half loaded or nearly saturated, which means capacity was never the problem.

The rate limiter is working correctly and that is the problem. It counts requests, and one hundred-thousand-token request occupies the KV cache of fifty short ones, so admitting both tenants equally hands one of them eighty percent of the machine.

Extraction accuracy cannot reveal a document that never arrived. Reconcile expected evidence with received and parsed artifacts, and keep completeness separate from any claim decision.

Cheap typed judgments make it affordable to check every input, tool call and output. The design question is which checks are semantic, which are deterministic, and how you prove the layer catches what it claims to.

The pitch is a model that runs the loop. The design that survives production is a loop that code runs, with the model answering three narrow questions inside it: which tool, is this done, does this need a stronger model.

Most answers draw an API gateway with OAuth and stop. The bank's problem is the hop from agent to tool: whose identity reaches the CRM, which of 40 tools this agent may call, what stops a runaway loop, and what happens when the gateway itself is down.

70 percent is an average over tasks and runs, and it hides the number the customer cares about: does the same request get the same right outcome every time? Two agents with identical averages can differ from shippable to unusable.

Single-session evals pass while users complain, because memory fails across sessions and at three separate stages. The answer is a planted-fact harness that names which stage lost the fact, plus the two tests teams skip: forgetting on request and leaking across users.

The customer wants the agent to practice before it can write to production. Design the task world, reset semantics, tool boundary and verifier, then explain when its scores justify training or release.

A frozen policy jumps from 70% to 90% after parallelizing its evaluation. Reconstruct the denominator, separate runner faults from policy behavior, and design the experiment that could invalidate the gain.

The interval pattern shows up everywhere in FDE loops: calendars, flight segments, log windows. Master the sort-then-sweep idiom here and three other interview questions fall out for free.

Top-K is the most reused primitive in FDE interviews: it reappears inside log parsers, analytics questions, and retrieval ranking. Three solutions exist; knowing which one to lead with is the real test.

Looks like a sliding-window problem; isn't one. The prefix-sum + hashmap trick that solves it is the same idea behind sessionization and cumulative-metrics questions later in the loop: learn it once, reuse it three times.

The gentlest 'design a class' question in FDE screens, and the warm-up interviewers use before rate limiters and sessionization. The deque trick is easy; the API and time-window follow-ups are where the signal is.

Anthropic-style screens open with deceptively simple builds like this, then extend them three times. The grade isn't the counting; it's whether your first version survives the extensions without a rewrite.

A tree question that's secretly a format-design question, which is why FDE loops love it. The preorder-with-null-markers trick is clean, but the delimiter and malformed-input follow-ups are where offers are decided.

A multi-part favorite at OpenAI-style screens: simple store, then time travel, then deletes that don't actually delete. The append-only insight plus one bisect call solves the whole thing, if you set up the invariant correctly.

A verbatim-reported multi-part at OpenAI and Anthropic. Each level breaks the previous design on purpose: the burst-at-the-boundary flaw is planted, and naming it before the interviewer does is how you win the round.

The format that filters out LeetCode-only candidates: unfamiliar code, failing tests, 45 minutes. There's a repeatable protocol for it, and a reason the first five minutes decide the round.

The coding question that doubles as a RAG-fundamentals check. Greedy packing is easy; the oversized-sentence case, the overlap-progress trap, and the tests you write unprompted are what AI-lab interviewers actually score.

The question where production scars are the rubric. Anyone can write a retry loop; the grade lives in which errors you DON'T retry, why jitter exists, and what an idempotency key actually protects you from.

A three-level build that quietly tests whether your L1 design survives L3. Regex substitution wins level one and loses the round; here's the token-based structure that carries through conditionals and loops.

Exact-match pub/sub is a dict of lists. The round is decided at level two (wildcard matching on hierarchical topics) and at level three, when the interviewer asks what happens when a subscriber's callback throws.

Kahn's algorithm gets you level one. The round is won at level three, where 'run up to K tasks at once' breaks naive topo sort, and the indegree bookkeeping you chose at L1 either saves you or sinks you.

The practical build that's secretly a graph problem: formulas are a dependency DAG, evaluation is DFS with memoization, and the three-color cycle trick decides whether A1=B1, B1=A1 crashes you or earns the offer.

The OpenAI FDE signature question, reported near-verbatim for two years: four levels in ~60 minutes, where L4 silently breaks every naive L3 implementation. The lazy-expiry design that survives all four levels, with the pacing plan.

The other OpenAI signature build, reported in four parts for two years. Part C silently punishes anyone who hardcoded single-condition filtering in part B; here's the predicate-compiler design that absorbs all four parts.

An OpenAI bank regular: next() over a list, then save/restore position, then resume across multiple JSON files where some are empty. The whole question is one decision (what counts as state) and most candidates get it wrong on part one.

cd('/foo/bar', 'baz') is easy. cd('/foo/../', './baz') is where the OpenAI screen actually starts, and the symlink follow-up is where it ends. The stack solution plus the loop-guard most candidates forget.

The OpenAI variant where the data structure is the warm-up: the round is graded on whether your tests control time and whether you can argue global vs per-key vs optimistic locking with actual reasons. Most candidates ace part one and lose the round in parts two and three.

A reported OpenAI build that looks like billing and is actually a data-structure question in disguise: credit grants that expire, usage that must burn the right grant first, and balance queries at arbitrary times. The earliest-expiry-first invariant carries the whole problem.

No JSON libraries allowed, and the test data contains commas, newlines, and your escape character itself. Length-prefix encoding solves it in fifteen lines, and knowing why it beats escaping is the actual interview.

The build behind every access policy and routing rule a customer wants to edit without a deploy. The trap is operator precedence: candidates who tokenize fine still get `a or b and c` wrong, and that's the whole grade.

The retry loop's grown-up sibling. Retries protect one call; a breaker protects the whole dependency from a retry storm. The grade is in the half-open state, where letting through one probe instead of all of them is the difference between recovery and re-killing the server.

A 10% rollout where the same user flips on and off between requests is worse than no rollout. The whole problem is determinism: hash the user into a stable bucket so ramping 10% to 25% only ever adds users, never reshuffles them.

Sliding-window limiters count requests; an API that bills by tokens needs to reserve a variable cost up front and refund it if the call fails. The trick is lazy refill: never run a timer, compute the current balance from elapsed time on each check.

Google's FDE loop added an AI-first round, and this is the opener. The skill being graded is not prompting, it is decomposition: turning a fuzzy sentence into scoped units with named interfaces an assistant can one-shot. Get the seams wrong and no prompt saves you.

When the model writes the code, your job shifts to specifying and checking it. The strong answer is a multi-layer validation pipeline that tests behaviors and contracts, not strings, plus the uncomfortable reframe that testing is now the bottleneck skill, not typing.

You can't read the model's weights, so you debug the inputs you control. There's a four-bucket triage, prompt, context, model, or spec, that localizes the fault fast, and the real deliverable is feeding each diagnosis back so the same failure can't recur.

If you read every AI PR line by line, you become the throttle the assistant was meant to remove. The staff answer is a risk-tiered trust model: route by blast radius, let gates clear the low-risk volume, and spend human attention only where reversibility is low.

The function reads a file, parses it, computes a metric, and prints, all tangled together. The move interviewers want is not prettier code, it's pulling pure logic out from side effects so each piece can be tested in isolation.

Brute force is O(N*d) per query and dies at ten million vectors. The answer is approximate nearest neighbor: trade a little recall for orders of magnitude less work, and know exactly which knob buys what.

Everyone quotes p95 in latency SLOs, but few can compute it without a library and fewer can explain why the index is ceil(p*n)-1, not p*n. The warm-up that screens whether you actually understand the metric you live by.

The arithmetic is three sums; the signal is what you do when a vector is all zeros and the denominator vanishes. The warm-up that gates every RAG and retrieval coding round.

Real LLM logs are JSONL, not JSON, and real log files have a malformed line halfway through. The warm-up that screens whether you can stream a file and aggregate without one bad record nuking the whole run.

Batching inputs is how you keep a GPU fed, but the source is an iterator you cannot index or len(). The warm-up that screens whether you can write a lazy generator that handles the ragged final batch.

A dashboard needs the average tokens/sec over the last N seconds, updated on every sample, forever. The warm-up that screens whether you evict by time and keep a running sum instead of re-summing the window each call.

Every classifier and every LLM-as-judge eval reduces to a confusion matrix, but the screen is whether you handle the zero-denominator cases without crashing and can say which metric matters for the problem at hand.

Every chat model decodes with top-p, but few candidates can build it from logits: softmax, sort, take the smallest set of tokens whose cumulative probability crosses p, renormalize, sample. The screen for whether you understand decoding, not just call an API.

The loss every language model trains on, built from logits with no torch in sight. The screen for whether you understand log-softmax and the numerical-stability trick, not just import nn.CrossEntropyLoss.

BPE is how GPT and most LLMs turn text into tokens, and building a tiny trainable version from a corpus is a favorite deep-dive. The screen for whether you understand the merge loop, deterministic tie-breaking, and round-trip decode, not just tiktoken.encode.

softmax(QKᵀ/√d_k)·V in four lines, but the grade is the three details people drop: the √d_k scale, subtracting the row max for numerical stability, and getting the causal mask to add -inf before the softmax, not after.

A grid where some cells start infected, and each step every infected cell infects its orthogonal neighbors. How many steps until everything is infected, or -1 if some cell is unreachable? This is rotting oranges in disguise, and the multi-source BFS framing is what they want to hear.

The 2026 successor to 'explain precision and recall', interviewers now test whether your classical ML evaluation discipline survives contact with GenAI. The decomposition, the RAGAS dimensions, and the judge-calibration step everyone skips.

A budget-constrained design problem, not an ideology question. The per-call math that rules an LLM in or out, the cascade that keeps quality while cutting cost 90%, and the SLO arithmetic that decides where the LLM is even allowed to sit.

Bandits sound strictly better and usually are not. The regret-vs-inference tradeoff, the three conditions that actually favor a bandit, and the production failure that makes adaptive allocation a debugging nightmare.

Triplet loss contrasts an anchor against one negative; modern contrastive losses like InfoNCE contrast against a whole batch at once, which is why CLIP-style training scales and old triplet pipelines stalled. The mining problem, the batch-size lever, and the call interviewers want.

Gradient descent is walking downhill on the loss surface, one step proportional to the slope. Knowing when to stop is the part that separates people who have trained models from people who have read about them: it is the validation curve, not the training curve, that tells you.

The naive pipeline gives you 10 million 'summarize this paragraph' pairs and a model that can only summarize paragraphs. The hard part is diversity and quality control at 100-billion-document scale, not the generation call.

Averaging two scalar scores tells you almost nothing. Blind pairwise battles plus a win-rate confidence interval do, and the result is only trustworthy once you've controlled position bias, verbosity bias, and rater fatigue.

There's no single number. The trap is reaching for perplexity or BLEU because they're easy to compute, then optimizing a score that has nothing to do with whether the output is good. The move is to match the metric to the task and name where each one lies to you.

The thing interviewers actually screen for isn't the Bellman equation, it's whether you know when NOT to use RL. Most problems that look like RL are better solved by supervised learning or a bandit, and reaching for full RL when you have labels is a classic over-engineering tell.

Every queue, webhook, and retry loop you'll ever deploy at a customer hides this distinction. Interviewers use it to separate people who've shipped from people who've read about shipping.

A staple at OpenAI and Anthropic in both coding and design rounds. Most candidates can name the algorithms; few can say which one lets 2x your limit through, and that's the part that gets scored.

Averages lie, and interviewers know most candidates quote them anyway. The fastest way to sound production-grade in any latency discussion is to reason about percentiles correctly, including the one trap almost everyone falls into.

A Fermi estimate with a paycheck attached. The interviewers' favorite filter: candidates who jump to a dollar figure miss the two numbers that actually break deployments, peak QPS and tokens-per-minute limits.

A signature FDE triage question with a hidden rubric: interviewers score the *order* of your moves, not just the list. There's a reason 'what changed?' beats 'check the dashboards', and a reason mitigation beats diagnosis.

The signature FDE debugging genre: a moving fault, an opaque client, and a customer who's sure it's your fault. The winning method splits the problem at the boundary, and knows the four classic culprits hiding on their side.

The bug only appears under load, the code 'hasn't changed,' and the customer just got charged twice. There's one root-cause pattern behind almost every version of this incident, and a fix with a subtle race most candidates miss.

Standard observability tells you the request returned 200 in 800ms. It cannot tell you the answer was wrong. The strong answer names the new failure plane LLMs introduce, and takes a real position on the prompts-and-PII question.

Prompts are production code with a worse failure mode: the regression returns 200 OK. The answer interviewers reward treats a one-word prompt edit with the same machinery as a schema migration, with one LLM-specific twist at every stage.

A 'simple' queue question with four traps wired in: minutes-long tasks break every default timeout, cancellation has to actually stop spend, progress needs a contract, and one bad job must never wedge the lane.

The Palantir decomposition round has no right answer, it has a right METHOD, and it's the most-failed interview in the FDE loop. Here is the four-move rubric interviewers actually score: clarify, model, spine, iterate.

The decomposition prompt we use to teach the skill that actually decides these interviews: graceful iteration under perturbation. Your score isn't your design; it's the delta when the design gets punched.

You control the sender; thousands of customer endpoints you've never seen control your fate. The rubric hides in three places: what 'ordering' really costs, why retries need the receiver's cooperation, and the debugging surface most designs forget.

Rate limits protect your servers; quotas protect your business model, and most candidates design one and think they've designed both. The strong answer also covers the part customers actually feel: what hitting the limit looks like.

The CFO sees API invoices; nobody sees the cost of the inference team they're about to hire. The strong answer models utilization, the variable that decides this question, and names the break-even honestly enough to be trusted.

Two questions disguised as one: classic multi-region (where the hard part is data, not compute) and dependency failover (where the hard part is that your fallback model gives different answers). Degraded modes are designed, not improvised.

The model is an untrusted code author with an unlimited imagination, and prompt injection means it can be *aimed*. Strong answers pick an isolation primitive deliberately, then close the channel most designs leave wide open: the data the code is allowed to touch.

The capstone FDE question, where every topic in this category shows up as one workstream. The rubric isn't the target architecture; it's whether you can sequence the gap-closing while the customer keeps using the pilot.

Reported loop detail: roughly 5 hours to build a RAG system, agent, or eval harness on OpenAI's APIs, a recorded video walkthrough, then a 60-minute defense. The hour-by-hour allocation, why the eval section is the differentiator, and what the video is actually screening for.

Active-active is easy until two regions write the same agent's memory at once. The strong answer picks a write model on purpose, names what conflicts and what cannot, and refuses to pretend a long-running agent run survives a regional failover for free.

Latency, cost, and quality are one budget with three claims on it. The strong answer treats the p99 SLA and the spend ceiling as a joint constraint, finds where they fight (batching), and names the lever it pulls when traffic exceeds what the ceiling can buy at SLA.

A crashed agent is easy. The hard one returns a confident answer after a tool quietly returned nothing and the model made up the rest. The strong answer instruments the steps that fail without throwing, ties traces to an outcome signal, and refuses to call a 200 a success.

Caching LLM output looks like free latency and cost savings until a cache key collides across tenants or a popular key expires under load and stampedes your model. The senior answer is mostly about what you refuse to cache.

The best FDE deployment is the one that doesn't need the FDE anymore. This question screens for whether you build for handoff from day one or quietly make yourself indispensable, which is the failure mode that looks like success.

A Copilot/Cursor-style assistant lives or dies on the latency budget. The strong answer spends every millisecond of a 200ms end-to-end budget on purpose, separates the inline-ghost-text path from the chat path, and shows why prefix caching and speculative decoding are the only ways to hit it at 10k concurrent users.

A gateway that screens every prompt and streamed token against a live policy without blowing the latency budget. The strong answer separates the policy decision from its data, runs deterministic checks inline and model checks in parallel, and redacts a leaking secret mid-stream before it renders.

Five LLM agents over a hundred steps, and the bar is zero contradictions with a 500-page contract. The real test is the state and grounding layer, not the agent prompts.

Fifty milliseconds at p99 to catch toxicity, PII, and injection on every message. An LLM in the hot path blows the budget. The strong answer tiers the checks and keeps the LLM off the critical path.

Under 500ms from end of speech to first audio out, across three models and a network. The trick is that you never wait for any stage to finish: you stream and pipeline everything.

A 2-second inference and a user who closed the tab. Naive stacks keep the GPU burning to completion. Real cancellation means propagating disconnect all the way to the batch scheduler.

One GPU, callers blocking on the response, and a hardware unit that is only efficient when fed wide batches. The whole job is trading a few milliseconds of queue wait for a large jump in throughput without ever stalling a request.

Every request emits a usage event worth real money, clients retry on timeouts, and tenants must be stopped at their quota and never overcharged. The hard parts are idempotent usage events and the succeeded-but-confirmation-lost case where the work happened but the client never heard back.

The most-reported decomposition prompt in the FDE canon. Almost every candidate builds a routing model, and almost every candidate is optimizing the wrong term of the equation.

Both payloads parse to the same object. Only one authenticates. Trace the bytes, endpoint secret and timestamp before replaying the customer's missed events.

Uptime and latency are the easy SLIs. The hard ones are whether the case was actually resolved, which you only learn days later, and what to do with an action that must never happen, which no error budget should cover.

The single most common FDE behavioral question, and the 'week one' follow-up is where most candidates collapse. Here's the structure that signals you can be dropped into chaos and produce order.

This is a flight-risk filter, not small talk, FDE teams lose hires back to pure SWE roles constantly. The answer that passes is built on evidence, not enthusiasm. Here's what it sounds like.

AI-native companies ask this to separate engineers who move fast with judgment from those who just move fast. The scored part isn't the speed, it's the corner-selection logic. Here's the framework.

The incident is the setup; the disclosure timeline is the test. Interviewers are timing the gap between 'you knew' and 'they knew', here's the answer that builds trust instead of torching it.

'The customer was dumb' is an instant fail, but so is theatrical self-flagellation. There's a narrow honest lane between the two, and this is what driving down it sounds like.

The modal opener for FDE customer rounds, and most engineers fail it in the first 90 seconds by pitching architecture. Here's the question sequence that scores, and the trap hidden in the word 'chatbot.'

Graded on two things most engineers never practice: the quality of the analogy and knowing when to stop. Here's a two-minute script that lands with a legal exec, and the depth trap to avoid.

The most common objection in enterprise AI, and arguing back is the fastest way to fail it. The winning sequence is validate, diagnose, reframe, de-risk. Here's the script.

Undefined success is how pilots die in 'one more month' purgatory. Here's the facilitation script that converts 'it should just be good' into numbers, baselines, and a decision date the customer owns.

The defining trusted-advisor dilemma, and both pure compliance and pure stubbornness fail it. The strong move is a sequence: diagnose, recommend in writing, then a test that lets the evidence decide.

The pure autonomy test: no one to ask, a customer waiting, and a scope line in front of you. Strong candidates reveal a decision rule, not a guess. Here's the rule.

'Too expensive' after a winning pilot is a solvable engineering problem wearing a procurement costume. Here's the diagnose-then-optimize playbook, including the 10x cost levers, that turns it around.

The security ambush kills more enterprise AI deals than accuracy ever will, and it's pass/fail on precision. Here's the answer stack a strong FDE has memorized, and the one response that ends the meeting.

The nightmare scenario every agent deployment signs up for. Interviewers grade the first hour hardest, containment before diagnosis, honoring the invented policy, and who you call in what order.

An integrity trap with a political twist: the person gaming your metric is usually your own champion. Blowing the whistle loudly and looking away both fail, here's the third path.

Anthropic's signature values filter, and it screens out two groups: candidates with no real views, and candidates performing views they clearly downloaded last night. Here's what an authentic, FDE-grade answer contains.

A double-layered probe: your model-selection reasoning, and, quietly, how you handle your employer's confidential information in front of a stranger. Overshare and you fail a test you didn't know was running.

FDE comp spans roughly $150K to $550K+ for the same title, because some companies pay it on the engineering ladder and some on a field ladder with variable comp. The first question to ask, and the traps in comparing OTE to base.

The eng lead isn't testing your architecture, he's already decided you should go and is performing it for the room. Trying to win the technical argument is the trap. Here's how to split his audience from his position and leave with a next meeting.

You hit 71% against an 85% bar nobody wants to say out loud, and the easy moves (quietly redefine the metric, ask for 'one more month') are how pilots rot in purgatory. Here's the honest salvage path, and how to recommend a kill without losing the account.

The conflict isn't you versus the customer, it's you versus your own quota-carrying colleague who wants the deal. The answer that passes refuses the line without grandstanding, finds the compliant adjacent path, and accepts the churn if there isn't one. Here's how to hold it without becoming the obstacle.

A board isn't an engineering audience and the postmortem they want isn't the one you'd write for your team. Lead with the wrong thing and you confirm their fear that this is unmanageable. Here's the board-grade structure, blameless but accountable, that restores confidence.

The capability isn't real, the ink is dry, and the customer found out from you, not your AE. Throwing sales under the bus and quietly building vaporware both fail. Here's how to reset the expectation without breaking the account or the internal relationship.

The graded shape is specific: learn and ship something small by day 30, own a deployment by day 60, drive a cross-customer improvement by day 90. Vague answers about 'meeting stakeholders and ramping up' fail. Here is the staged plan with what good looks like at each gate.

A research team that owns its own way of working does not adopt your process because you're right. The candidates who win this question don't argue, they run a small proof that makes the practice obviously cheaper than the status quo, then let the team claim it.

Around 95% of enterprise AI pilots produce no measurable impact, almost always from deployment, not the model. As the first FDE in a skeptical SWE shop, your month-one job is to not be pilot 95: pick a project that can win, prove it in 30 days, and turn the engineers who roll their eyes into the ones who vouch for you.

The gap is real, the date is close, and the business is about to ask if you'll make it. The wrong first move is to start tuning prompts. The strong move is to split the failure into causes you can act on, give the business an honest forecast with options, and protect the launch even if you can't fully close the gap.

At 100k QPS you cannot 10x the model for a +2% quality bump. This question screens whether you can find the actual operating point under a real constraint and get product, finance, and the customer to agree to it instead of pretending the tradeoff away.

The losing answer is a personality clash you won by being right. The graded one treats it as an organizational conflict, finds the shared metric both teams actually optimize for, and keeps escalation as a last resort you used cleanly, not a weapon.

The trap is sounding like an absolutist who would torch every deal on principle. The win is a real story where you held a line with judgment: you found what was actually at stake, drew it narrowly, and paid a cost you can name. Here's the difference.

The screen is whether you build for your own replaceability or your own indispensability. Strong answers treat the handoff as a deliverable with a graduation criterion, not a final week of doc-dumping. Here is what to leave behind, how to transfer it, and the trap that sounds generous.

The deployment worked. The software runs, the data flows, the demo landed. Nobody is using it. Two of the three research sets we vetted surfaced this from different angles, and it is the failure mode that ends renewals.

Nothing about the software changed. The single person whose belief the contract rested on is gone, and the replacement has more to gain from cancelling your project than from finishing it.

The structural tension of the job in one question. You are the customer's advocate inside your company and your company's representative at the customer, and this is the day those two roles openly disagree.

The most common real ticket in applied AI, and it contains almost no information. What you do in the first hour decides whether you spend a day on this or three weeks.

Asked in every loop, prepared for by almost nobody, and it has a single mechanical test that tells you whether your answer is any good before you ever say it out loud.

The trap is that it looks like a prioritization problem you can solve with a scoring rubric. You do not have the authority to rank these people's work, and the moment you try, you have made yourself the obstacle.

The question that separates a forward deployed engineer from a very good consultant. One delivers each engagement. The other notices that the third engagement looked like the first two and does something about it.

A working demo leaves authorship and understanding unresolved. Defend the decisions, reproduce a failure and adapt the code under a changed requirement without overstating what you tested.

Three individually reasonable requests create an impossible week. Build a capacity ledger, protect existing obligations and obtain a decision from the people who can change cross-account commitments.

A passing evaluation and an enthusiastic sponsor can coexist with a blocked release. Separate the decisions, preserve useful progress and write a recommendation with a specific approval dependency.

A high-value request arrives before its permissions and operator. Decide which work can start, who can authorize an exception and what a responsible practice launch commits to.

A fluent demonstration earned a strong hire from one assessor, while another recorded a critical gap. Reconcile the observation, assistance and role requirement without averaging away the disagreement.

The total-hours plan fits while a shared specialist cannot meet the change windows. Reconcile support obligations, skill coverage and coaching cost before adding another customer.

Services income, expansion contribution and retained business overlap in the board packet. Rebuild the economic boundary and identify which assumption changes the funding decision.

Logs, traces, and model-quality signals from hundreds of LLM products at firehose volume. The storage split that keeps queries fast, the alerting that survives drift and toxicity at scale, and why sampling has to be cost-aware and tail-biased.

Two hard numbers in the contract: 95% accuracy and $0.50 per resolution. The job is an eval and rollout system that proves both hold on real traffic before you scale, with rollback triggers wired to the same numbers.

The backbone question of every frontier-lab infra loop. The answer that scores is organized around what each strategy communicates and how often, not just what it splits.

The one derivation every GPU-infra loop expects on a whiteboard: scatter-reduce plus all-gather, 2(N−1)K/N per GPU, and why that's provably near-optimal. Plus the latency catch that motivates tree algorithms.

The 16-bytes-per-parameter breakdown that makes ZeRO's three stages obvious instead of memorized, and the communication bill each stage runs up in exchange.

A reported DeepMind research-engineer question. The winning answer opens with a memory budget in bytes, not a list of parallelism buzzwords, here's the full arithmetic and the layout it forces.

A real infra-loop question that candidates fumble by guessing. The correct map, TP on NVLink, PP and DP across the scale-out fabric, falls out of three traffic profiles, and the reasoning is the answer.

Both carry RDMA at 400Gb/s; the decision is about loss behavior, tuning burden, and who operates it. A defensible recommendation with the conditions that flip it, the format this question is scored on.

The MoE question frontier labs use to test whether you see routing as a systems problem, not just a modeling trick. Covers the collapse spiral, auxiliary losses, capacity factors, and what changes at serving time.

A reported DeepMind debugging-round prompt. What's scored is your search order, not your bug list, the overfit-one-batch ladder that finds 90% of flat-loss causes in under an hour.

The resume-walk question for infra roles, with a hidden rubric: layer ownership, scale fluency, and one defensible war story. How to answer credibly at any actual scale, and the inflation tells interviewers catch.

One formula, 2 × layers × KV heads × head_dim × bytes, unlocks every LLM serving capacity question. Worked through for Llama-70B, plus the mitigation stack from GQA to paging.

Why the batch is a pool of decode slots, not a bus that waits until it's full. The mental-model shift behind every modern serving stack, and the scheduling tradeoffs interviewers push on next.

One forward pass, two opposite performance regimes. The arithmetic-intensity argument that explains TTFT vs TPOT, GPU pool design, and why the big serving stacks split the phases onto different hardware.

Free tokens from idle FLOPs, losslessly, which surprises most candidates. The mechanism, the acceptance-rate math, and the high-batch regime where speculation quietly stops paying.

A reported OpenAI design prompt. The skeleton: gang scheduling, topology-aware placement, preemption with checkpoint contracts, and quota, plus the fragmentation problem that quietly eats 20% of naive clusters.

The back-of-envelope chain every inference-platform round expects: traffic → tokens/sec → per-GPU throughput from bandwidth math → fleet size → dollars. With the headroom factors candidates forget.

A reported OpenAI phone-screen exercise that looks trivial and is graded on edge cases: money-safe arithmetic, billing increments, unknown SKUs, and the clarifying questions you ask before typing.

The OpenAI capstone design. What separates passing answers: token-based capacity math, KV/prefix-aware routing instead of round-robin, and a degradation ladder for the day demand doubles overnight.

A sparse model with few active params per token, a brutal latency target, and an all-to-all on the critical path. The parallelism layout that hits the SLA, why expert parallelism not tensor parallelism is the lever, and the tail the all-to-all hides.

The CFO-facing infra question: a demand forecast with a fat tail, GPUs that take months to land, and three cost structures that win in different regimes. The layered commitment model that keeps you from paying for peak all year.

The fleet was fine on 8k contexts and falls over when a few 200k-token requests arrive. The math that explains why, the admission and pooling fixes that contain it, and why throwing GPUs at it is the wrong first move.

A very large sparse model, hundreds of experts, and users on every continent. The layout that keeps experts close to compute, routes around regional skew, and decides what 'global KV cache' can and cannot mean.

Constrained JSON/grammar decoding usually means a Python callback per token, which serializes the GPU. The staff answer keeps the DFA on-device and turns the mask into a single coalesced gather.

Reprefilling the whole history every turn makes a long chat O(n^2). The real design is a radix tree of KV blocks with copy-on-write branching, a weight-epoch tag, and an eviction policy that protects live turns.

64 H100s for 30 days is a real budget with a real token count. The answer sizes the run from Chinchilla, picks parallelism for this exact cluster, and has a concrete plan for loss spikes and silent data corruption.

OWASP's LLM01 is the opening question of nearly every AI security screen. Most candidates can define direct injection; the ones who get hired explain why indirect injection is the one that ruins enterprise deployments.

OWASP made system prompt leakage its own category (LLM07) because teams keep treating prompts as secrets. The strong answer flips the premise: design so extraction doesn't matter.

OWASP LLM06 is where AI security stops being novel and becomes a least-privilege problem, which is exactly why interviewers love it. The control list is short; knowing which control to lead with is the test.

Poisoning questions sort candidates who've read about backdoors from those who can name where poison enters a real pipeline, and why the embedding store is the soft target nobody audits.

A verbatim AI-security interview question with a built-in trap: candidates who promise to stop extraction fail, and candidates who shrug 'it's unpreventable' also fail. The winning answer is a layered containment design.

MCP turned 'install a plugin' into 'inject text into every conversation.' Interviewers ask this to see if you understand why tool descriptions are an attack surface, and what a least-privilege agent runtime looks like.

A reported AI-security interview staple with a known grading rubric: recon, bypass, leakage, tool abuse, multi-turn drift. Candidates who free-associate attack ideas fail; candidates who run a structured campaign pass.

An Anthropic-style system design axis: architectural safety. The answer is a pipeline diagram in words, five checkpoints, what each catches, what each costs in latency, and the streaming complication most candidates forget.

Every EU enterprise deal hits this question, and the FDE in the room is expected to carry it without legal on the call. The processor/controller split, the transfer mechanics, and the one GDPR right that's hard for LLMs.

The most-asked question in every enterprise AI deal, and precision is pass/fail: the contractual answer, what ZDR actually changes, and the retention nuance that separates FDEs from demo engineers.

A reported Anthropic system-design prompt. The skeleton is standard API platform work; the AI-specific 20%, token-denominated limits, safety classifiers in the path, abuse identity, is what the round actually grades.

A real customer ask from the field, and a layered design question in disguise: attribution, budgets, anomaly detection, and enforcement, plus the attribution problem that decides whether any of it works.

The LLM billing round has three trapdoors: retries double-charging customers, metering latency leaking into inference, and month-end numbers that don't match the provider invoice. The design that survives all three is the answer.

The capstone governance round: AI incidents break standard IR playbooks in three specific places, containment without rollback, blast-radius over probabilistic systems, and root cause for nondeterministic behavior. Interviewers grade you on knowing which three.

The agent has real money authority and a write path into the system of record, which is the exact shape attackers want. The strong answer is layered enforcement outside the model, with the spend cap and the chain rule in code, not in the prompt.

Cross-tenant or cross-user data exposure through a model is the breach that starts regulatory clocks. The runbook has to answer 'whose data, to whom, how many' from logs that most teams never designed for, and contain a leak that may live in a cache or a fine-tuned model, not a deploy.

Turn a constitution of rules into a model that follows them, with no human labels on harmful examples. The strong answer walks the two phases (self-critique SFT, then RL from AI feedback), then spends real time on the part papers gloss: adversarially validating the aligned model holds under attack.

A code model that occasionally produces ransomware, reverse shells, or exfiltration scripts. The strong answer builds an automated loop: adversarial prompt generation, static and behavioral output scanning, layered mitigations, and attack-success-rate per category to prove the mitigations worked.

A red-team model that learns over many episodes which attacks bypass your filters is only useful if it explores instead of farming one exploit, and only safe if every discovered attack becomes target training data.

SAEs can surface candidate 'deception' features in the residual stream, but a correlated feature is not a cause. The real work, and the honest answer, is causal validation and admitting what interpretability cannot yet prove.

Candidates treat computer-use agents like API agents with a screenshot. The interviewer is probing the new visual attack surface (hidden text, off-viewport UI, injected images) and how containment changes when the agent drives a real customer desktop. Most miss the per-turn-approval trap.

The inventory is the easy half. The hard half is finding the agents nobody registered, versioning behavior rather than code, and a kill switch that actually stops traffic within seconds while keeping every past action traceable.

Everyone says '500 tokens with overlap.' The candidates who get hired explain why chunk boundaries are a recall problem, name three strategies, and describe the experiment that picks between them.

A reported FDE design question that punishes one-size-fits-all answers. Each corpus has a different 'semantic unit': get the three designs, the parsing tools to name, and the eval that proves the split was worth it.

Every deployment has a skeptic, and 'I showed them data and they came around' is the answer everyone gives. The scored version starts with why they were right to be skeptical.

Three plausible fixes, one decision framework. Interviewers grade the ordering (cheapest-reversible first) and whether you can say what each option costs in data, time, and operational burden.

Why a raw base model answers your question with three more questions, what RLHF actually buys, and the niche cases where customers should pick base. A tidy answer most candidates fumble.

A Sierra-signature scenario where 'put it in the system prompt' scores zero. The winning answer is defense-in-depth with a measured catch-rate per layer, plus an honest number for what still gets through.

One malformed message can silently wedge an entire pipeline, and it happens at every customer eventually. The DLQ answer interviewers want covers what most candidates forget: what happens *after* the message lands there.

Citations are the trust interface of enterprise RAG, and the easiest thing to fake badly. Covers inline citation generation, span verification, and the verification-rate metric that interviewers actually want.