Scale AI Forward Deployed Engineer interview questions

Every FDE will face an angry customer asking this. The mechanistic explanation interviewers want (why hallucination is the default behavior, not a bug) plus the framing that defuses customer escalations.

A classic metrics question with a domain twist. The answer isn't 'recall'; it's a cost-of-errors argument plus a threshold strategy, and interviewers grade the reasoning, not the metric you pick.

The single most-reported FDE conceptual question across every AI lab. Interviewers don't want definitions; they want the decision framework, the cost math, and the recommendation order that reads senior.

Anyone can list mitigations; the question is really about costs: latency, dollars, and lost recall. The three-layer answer with honest price tags that AI-lab interviewers score highest.

The data-volume thresholds, the break-even math, and the regression-prevention checklist that turn 'just fine-tune it' into a defensible engineering decision. Includes the crossover rule interviewers listen for.

JSON mode, strict schemas, constrained decoding, validate-and-retry: the escalation ladder for structured output, plus the catch nobody mentions: schema-valid can still be factually wrong.

Prompts are production code that nobody treats like code, until an innocent one-line tweak tanks accuracy. The version-eval-canary-rollback pipeline that AI-native interviewers expect by default.

Evals are the #1 probing area in AI-lab FDE loops. The full suite design, golden sets, component metrics, safety red-teaming, CI gates, and the launch-blocking thresholds, in interview-ready form.

Position bias, verbosity bias, self-preference, and the agreement-rate workflow that turns a sloppy judge into eval infrastructure you can defend. AI labs probe this hard; here's the calibrated answer.

The classic FDE escalation. Six hypotheses, silent model updates, input drift, prompt entropy, sampling noise, perception shift, and the eval-replay test that splits them in an afternoon.

Provider model upgrades ship faster than your release cycle and quietly shift behavior. The design that turns 'we hope it's fine' into a blocking gate: a frozen golden set, pairwise diffing, segmented scoring, and a shadow-traffic canary.

The escalation ladder is settled; the at-scale tradeoff is not. Why retries quietly multiply your bill and tail latency, why constrained decoding has its own taxes, and the hybrid that most high-volume pipelines actually run.

Two failure modes, one architecture. Why no single guardrail holds, the defense-in-depth layers that assume the model will be broken, and the design rule that limits the blast radius when prompt filters inevitably fail.

Three stages, one KL leash. The reward-model-plus-PPO loop, the beta*KL term that keeps the policy from drifting off the reference, and how DPO collapses it into a single classification loss by treating the reward as beta*log(pi/pi_ref). The objectives, written correctly.

Win-rate up, facts down is the signature of reward hacking: the policy is over-optimizing the preference proxy and drifting off the reference. The diagnosis (Goodhart on the reward), the immediate lever (tighten the KL/beta anchor), and the held-out factual gate that stops shipping the regression.

PPO needs a separate value network to estimate the baseline for its advantage. GRPO samples a group of answers per prompt and uses their mean reward as the baseline, so the advantage is just how a sample compares to its peers, no critic required.

Budget covers maybe 5M of 100M LLM calls. Run a cheap first stage on everything, route only low-confidence items to the LLM, and use those LLM labels to distill a student that absorbs most of the volume. The cost-vs-coverage math that decides where the threshold goes.

The warm-up that filters out tutorial-level candidates. Interviewers aren't grading the diagram, they're grading whether you know the three places real pipelines fail and how you'd see it happening.

Everyone says '500 tokens with overlap.' The candidates who get hired explain why chunk boundaries are a recall problem, name three strategies, and describe the experiment that picks between them.

'Evals first' is the #1 hiring signal in FDE design rounds. This is the foundational version: what a 50-case golden set looks like, which metrics split retrieval from generation, and the launch gate that convinces interviewers.

A reported FDE design question that punishes one-size-fits-all answers. Each corpus has a different 'semantic unit': get the three designs, the parsing tools to name, and the eval that proves the split was worth it.

The modal FDE design question of the last two years. There's a scoring trap in the first 60 seconds, and most candidates jump to fixes and fail. Here's the diagnostic tree that wins it.

The unglamorous design question that exposes who has run LLMs in production. Batch APIs at 50% off, idempotency keys, poison records, spend kill-switches: the full checklist interviewers listen for.

The staff-level version of 'evals first.' Four layers, each with a job, a cost, and a blind spot, plus judge calibration numbers and the data flywheel that ties production failures back into CI.

The POC-to-production gap is the FDE's home turf. What actually breaks at each order of magnitude (index, ingestion, latency, cost) and the dependency math that makes 99.9% harder than it sounds.

Reciting the four definitions is table stakes; the differentiator is knowing which two need ground truth, what each dimension tells you to fix, and where the judge-model scores quietly lie. The 2x2 that turns RAGAS into a debugging tool.

The classic 'it regressed and the prompt is untouched' triage. The answer that scores walks a decisional tree from retrieval to generation and names the silent culprit most candidates miss: the embedding stack changed under you.

Hard budgets stop the bleeding; they don't fix the cause. The real fix is progress tracking plus semantic loop detection plus a planning layer that commits to a plan instead of re-deciding every step, validated on a fixed trajectory set offline before it ships.

You cannot unit-test a stochastic two-agent pipeline with a golden output. You evaluate it the way you stress a distributed system: thousands of simulated runs, controlled fault injection, and a judge calibrated against humans.

DSPy's pitch is to stop hand-writing prompts and instead declare what the pipeline does, define a metric, and let a compiler search for the prompts and examples. The catch is that the compiler is only as good as the metric, and an LLM-judge metric drags its own biases in.

Before you touch a model, check whether the two numbers measure the same thing. Field accuracy of 96% across twelve fields is 61% of documents fully correct, which means half of the drop may be arithmetic. The other half is the documents the pilot never saw.

A frozen policy jumps from 70% to 90% after parallelizing its evaluation. Reconstruct the denominator, separate runner faults from policy behavior, and design the experiment that could invalidate the gain.

The most common screen opener, and interviewers use it to check whether hashmap thinking is reflexive. Here's the one-pass answer, the narration that earns points, and the duplicate-handling edge case most candidates fumble.

A 5-minute warm-up that quietly tests the most useful idea in practical coding: choosing a canonical key. The sorted-string vs character-count tradeoff is exactly what interviewers want to hear you reason about.

Looks like a sliding-window problem; isn't one. The prefix-sum + hashmap trick that solves it is the same idea behind sessionization and cumulative-metrics questions later in the loop: learn it once, reuse it three times.

Everyone knows the sort + two-pointer outline. The interview is actually about duplicate handling: three separate skip conditions that candidates routinely botch live. Here's the clean version and the narration that goes with it.

K-way merge is the algorithm behind log aggregation, search-result merging, and LSM trees, which is exactly why FDE loops keep asking it. The heap version is table stakes; the tie-breaking detail is where candidates crash.

The quintessential FDE practical: messy real-world input, grouping, ranking, and a hidden rubric line for how you treat malformed data. Most candidates parse happily and lose the round on the lines that don't parse.

Sessionization is the analytics primitive behind every 'how do users actually behave' question, and a favorite practical screen. The algorithm is easy; out-of-order events and duplicates are where the round is actually scored.

Every data migration at every customer site needs this exact tool, which is why Palantir and Retool keep asking it. The diff is a dict comparison; the grade hides in keys, duplicates, and column drift.

A multi-part favorite at OpenAI-style screens: simple store, then time travel, then deletes that don't actually delete. The append-only insight plus one bisect call solves the whole thing, if you set up the invariant correctly.

The coding question that doubles as a RAG-fundamentals check. Greedy packing is easy; the oversized-sentence case, the overlap-progress trap, and the tests you write unprompted are what AI-lab interviewers actually score.

Palantir's bread and butter: 'Acme Corp.' and 'ACME Corporation, Inc.' are the same customer; prove it in code. Normalization does more work than the clever algorithm, and blocking is what makes it run before the heat death of the universe.

The OpenAI FDE signature question, reported near-verbatim for two years: four levels in ~60 minutes, where L4 silently breaks every naive L3 implementation. The lazy-expiry design that survives all four levels, with the pacing plan.

At-least-once delivery means duplicates are guaranteed, not hypothetical. Anyone can use a set; the round is whether your set grows forever, and what you do with an event that arrives after its window closed.

A classic FDE take-home shape: messy real documents in, structured index out. The grade is in the seams between reader, extractor, and writer, and in what happens when one file is garbage.

Every classifier and every LLM-as-judge eval reduces to a confusion matrix, but the screen is whether you handle the zero-denominator cases without crashing and can say which metric matters for the problem at hand.

All-pairs comparison is O(n^2) and dies long before 1TB. The senior move is MinHash plus LSH: hash documents so only likely-similar pairs ever land in the same bucket, then verify just those.

A stateful pass over sorted events, which most candidates get working in ten minutes. The reason it is asked is the next ten: in this domain a false positive is a person accused of something.

Every FDE loop opens with this, but the definitions are table stakes. What gets scored is whether you can map each metric to a dollar cost the customer recognizes, here's the framing that does it.

A 0.95 AUC model can still flood a review queue with garbage. Knowing when ROC AUC flatters a model, and what to report instead, separates FDEs who've shipped from those who've only run notebooks.

The accuracy trap is the easy part. Interviewers keep pushing: resample or reweight? Does SMOTE survive contact with production? What happens to your probabilities? Here's the full chain.

This exact gap kills more customer pilots than any modeling choice. There's a ranked list of culprits, and one of them hides inside innocent-looking feature pipelines at almost every enterprise.

A two-minute explanation, then the real test: learning-rate pathologies, why we use mini-batches, and the one practical diagnosis, reading a loss curve, that customer-facing engineers actually need.

On enterprise data, the feature pipeline routinely buys 2-5x the lift of a model swap. Here's the worked churn example, and the leakage rule every engineered feature must pass, that makes the argument stick.

Every RAG deployment runs on this math, and most candidates can't say why cosine beats Euclidean, or when the two are secretly identical. The 90-second version with a worked example.

The 2026 FDE differentiator: customers ask for LLMs by name, and the engineer who knows when a $0 logistic regression beats a $40k/month LLM pipeline, with the cost math, wins the room.

Almost never, and the gap between score and probability quietly breaks thresholds, expected-value decisions, and customer trust. What calibration is, how to measure it, and the two fixes that work.

The signature production-ML incident, and a pure FDE test: a ranked differential diagnosis run live on a call, because 'nothing changed' is always false, and one hypothesis explains half of all cases.

Offline wins don't justify big-bang swaps, models fail in ways staging never shows. The shadow → canary → ramp playbook, what to compare at each stage, and the rollback discipline that keeps customer trust.

When a model auto-approves loans or routes patients, the probability is the product, not a dashboard number. The calibration-plus-abstention design that lets a customer trust an automated decision, and the subgroup trap that gets it pulled in audit.

A budget-constrained design problem, not an ideology question. The per-call math that rules an LLM in or out, the cascade that keeps quality while cutting cost 90%, and the SLO arithmetic that decides where the LLM is even allowed to sit.

When treatment leaks between units, a clean A/B lies in both directions. The interference taxonomy, when to reach for switchback vs cluster vs geo designs, and the analysis trap that makes naive standard errors useless.

Not leakage, not skew, not drift: the model learned exactly what you asked, and you asked for the wrong thing. The proxy-label mismatch that passes every test, the audit that catches it, and why this is the failure no metric can see.

Most candidates recite the Adam update equation and stop. The interviewer wants the decision: when plain SGD with momentum still wins, why Adam is the safe default for new architectures, and how you diagnose a training run that is silently dying from a gradient that shrank to zero.

Backprop is just the chain rule run efficiently in reverse, but the question that separates levels is why a hundred-layer network trains when a naive one wouldn't. The credit-assignment intuition plus the three tricks that keep gradients alive.

The answer is not just 'transformers are better.' It is two specific wins: every position can attend to every other in one step (long-range dependencies) and the whole sequence trains in parallel instead of one token at a time. Plus the cost that buys, and where RNNs still make sense.

Convolution and parameter sharing are why a CNN needs a tiny fraction of the weights a dense net would, and why it generalizes across position. The conceptual answer, the feature-hierarchy picture, and the honest take on when a fine-tuned CNN still beats reaching for a giant vision-language model.

The margin idea is elegant and worth understanding, and the kernel trick is a clever move. But the question that separates levels is why a model that dominated the 2000s is now a niche pick. The scaling math that killed it for big data, and where it still wins.

The base-rate trap is the classic, and it is not academic: it is exactly how a high-accuracy fraud or disease classifier drowns ops in false alarms. The worked numbers, the tree you draw on the whiteboard, and the two levers that actually fix it in production.

MAP is MLE plus a prior, and that prior is not academic decoration: it is the same thing as regularization, and it is what stops a model from making confident nonsense out of three data points. The connection that makes this an applied answer, not a stats-class recitation.

Picking the wrong distribution silently breaks your model: counts are not Gaussian, and treating them that way predicts negative events and underestimates the variance of busy periods. The decision tree by data type, the variance gotcha, and what each choice implies for the model you reach for.

The honest version: deep learning learns its own features but is hungry for data and compute, and on a customer's 50,000-row tabular dataset a gradient-boosted tree wins on accuracy, cost, and time-to-ship. The dividing line and the conditions that flip it.

Profile before you model: shape, types, target balance, leakage check. Then the part most candidates botch, that a missing value is often the most predictive feature in the table, so you flag it before you fill it. The first-hour playbook and the imputation decisions that survive production.

Triplet loss contrasts an anchor against one negative; modern contrastive losses like InfoNCE contrast against a whole batch at once, which is why CLIP-style training scales and old triplet pipelines stalled. The mining problem, the batch-size lever, and the call interviewers want.

L2 shrinks every weight smoothly toward zero; L1 drives many weights to exactly zero, which is automatic feature selection. The geometric reason L1 hits the corners is the part interviewers actually want, plus the call on when sparsity beats smooth shrinkage.

A confusion matrix is the 2x2 of right and wrong predictions per class, and it is the reason a 99% accurate fraud model can be useless. The worked imbalance example, and how reading the matrix as precision and recall fixes the lie.

The naive pipeline gives you 10 million 'summarize this paragraph' pairs and a model that can only summarize paragraphs. The hard part is diversity and quality control at 100-billion-document scale, not the generation call.

There's no single number. The trap is reaching for perplexity or BLEU because they're easy to compute, then optimizing a score that has nothing to do with whether the output is good. The move is to match the metric to the task and name where each one lies to you.

The amateur move is to train a fresh deep net on 200 rows and overfit instantly. The senior move is to spend almost no labels on training: lean on a pretrained model, use the 200 as a validation set, and design a labeling loop that buys the most signal per label.

Random sampling is the baseline you have to beat, and honest candidates admit it sometimes wins. Active learning earns its keep by labeling where the model is least sure, but uncertainty alone gives you a batch of near-duplicates, so you also need diversity, and a cold-start plan.

Everyone spots that the customer records need resolving. The failure that actually sinks this project is quieter: the word fraud means three different things in the three datasets, so the union of their labels trains a model that is confidently wrong.

Thirty percent disagreement is not a fact about the annotators. It is a fact about the task definition, and on a skewed label set it can mean the labels are worse than chance. The week is spent on the guideline and the gold set, not on the model, because no model trains past the noise in its labels.

The most common escalation in any data deployment, and the one that quietly ends trust in the platform. Four causes produce it, one of them means nobody is wrong, and you cannot tell them apart without running the ladder in order.

Four lines of SQL, and one of the three obvious ways to write it returns an empty result set instead of an answer. Not wrong rows. Zero rows, silently, on production data that looks fine.

Everyone writes the join and the ratio. Then the result comes back dominated by people who bought one thing and sent it back, and the list is useless to the person who asked for it.

The constraint is the question. It rules out the two approaches most candidates reach for, and the approach that survives has one failure mode that can take the customer's production database down.

Latency, cost, and quality are one budget with three claims on it. The strong answer treats the p99 SLA and the spend ceiling as a joint constraint, finds where they fight (batching), and names the lever it pulls when traffic exceeds what the ceiling can buy at SLA.

No data leaves, you cannot pre-inspect the corpus, and your toolchain arrives on a single USB. Design adaptation and evaluation when you will never observe a single input or output.

Fifty milliseconds at p99 to catch toxicity, PII, and injection on every message. An LLM in the hot path blows the budget. The strong answer tiers the checks and keeps the LLM off the critical path.

A back-of-the-envelope round where the envelope has a purchase order attached. The arithmetic is the easy half. The scoring is in which assumptions you write down, which way you round, and what you say about the number being wrong.

The most-reported decomposition prompt in the FDE canon. Almost every candidate builds a routing model, and almost every candidate is optimizing the wrong term of the equation.

It reads as an optimization problem and it is not, quite. One of the three inputs can never be traded against the others, and a design that scores all three has quietly built a compliance incident with a scheduler attached.

A successful provisioning run and a green dashboard leave several paths untested. Locate the broken boundary using agent health, upgrade headers and the route the user actually takes.

The single most common FDE behavioral question, and the 'week one' follow-up is where most candidates collapse. Here's the structure that signals you can be dropped into chaos and produce order.

AI-native companies ask this to separate engineers who move fast with judgment from those who just move fast. The scored part isn't the speed, it's the corner-selection logic. Here's the framework.

FDEs change domains every deployment, so interviewers test for a repeatable learning system, and most candidates fail the second half: proving the understanding was real. Here's the method and the validation move.

The most common objection in enterprise AI, and arguing back is the fastest way to fail it. The winning sequence is validate, diagnose, reframe, de-risk. Here's the script.

Undefined success is how pilots die in 'one more month' purgatory. Here's the facilitation script that converts 'it should just be good' into numbers, baselines, and a decision date the customer owns.

The first use case decides the whole account, pick a flashy-but-doomed one and there's no second. Here's the value × feasibility × data-readiness workshop that strong FDEs run, hour by hour.

The pure autonomy test: no one to ask, a customer waiting, and a scope line in front of you. Strong candidates reveal a decision rule, not a guess. Here's the rule.

Every enterprise AI deployment hits this wall, interviewers want to know if you deliver the bad news early, with options, or sit on it hoping. Here's the conversation script, beat by beat.

An integrity trap with a political twist: the person gaming your metric is usually your own champion. Blowing the whistle loudly and looking away both fail, here's the third path.

A genuine filter, not small talk, Palantir interviewers have heard every dodge. The answer that works engages the controversy directly, with a position you can defend under two rounds of pushback.

The xAI signature round, 30 minutes of 'why?' aimed at one project, designed to expose orchestrators posing as builders. Here's how to choose the project and survive the drill.

Sounds like a softball; it's a filter. 'I read newsletters and follow people on X' is the answer that loses. Interviewers are listening for a system with a build loop, here's one you can describe and actually run.

The portfolio probe that opens Salesforce's Agentforce FDE loop and most 2026 screens. Interviewers are smelling for weekend wrappers, the scoring is on evals, users, and production scars, not project count.

You hit 71% against an 85% bar nobody wants to say out loud, and the easy moves (quietly redefine the metric, ask for 'one more month') are how pilots rot in purgatory. Here's the honest salvage path, and how to recommend a kill without losing the account.

The gap is real, the date is close, and the business is about to ask if you'll make it. The wrong first move is to start tuning prompts. The strong move is to split the failure into causes you can act on, give the business an honest forecast with options, and protect the launch even if you can't fully close the gap.

Surface knowledge recites the SOTA model name. Depth names the failure mode that bites you in production and the open problem nobody has cleanly solved. This is how to sound like you've actually shipped in your area.

The structural tension of the job in one question. You are the customer's advocate inside your company and your company's representative at the customer, and this is the day those two roles openly disagree.

Asked in every loop, prepared for by almost nobody, and it has a single mechanical test that tells you whether your answer is any good before you ever say it out loud.

The question that separates a forward deployed engineer from a very good consultant. One delivers each engagement. The other notices that the third engagement looked like the first two and does something about it.

Three serving runtimes that look interchangeable on a benchmark slide and aren't. The axes that actually separate them (scheduler, kernels, operability) and the one-line rule for which to reach for.

Do the weight math first, then choose your poison: shrink the weights, split them across GPUs, or spill to host memory. Each buys you the fit at a different cost, and one of them is almost always wrong for production.

Candidates use these terms interchangeably and security interviewers notice immediately. The distinction, whose policy is being violated, drives completely different defenses, and that's the actual question.

Reciting all ten gets you a pass on memory and a fail on judgment. The question is really a prioritization exercise, here's the two-risk answer that maps to how agents actually get breached, and the trap hiding in 'walk me through.'

Poisoning questions sort candidates who've read about backdoors from those who can name where poison enters a real pipeline, and why the embedding store is the soft target nobody audits.

ATLAS questions filter candidates who can name the framework from those who can run an incident through it. The mapping exercise, one concrete attack, kill-chain stage by stage, is what the interviewer actually wants to hear.

A reported AI-security interview staple with a known grading rubric: recon, bypass, leakage, tool abuse, multi-turn drift. Candidates who free-associate attack ideas fail; candidates who run a structured campaign pass.

The grown-up version of 'how would you red-team a chatbot', Microsoft-style loops ask for a program, not a pen test. Team composition, harm taxonomy, iteration cadence, and which attack classes you script on day one.

SOC 2 fluency is the price of admission to enterprise deals, and interviewers check it with two traps: what a SOC 2 report actually proves, and what it doesn't. Engineers who can read one are rarer than you'd think.

NIST AI RMF is voluntary, which is exactly why a CISO uses it as a shared yardstick. The strong answer turns Govern, Map, Measure, Manage into artifacts you already produce, and uses the Generative AI Profile to name the LLM-specific risks the core framework leaves abstract.

A red-team model that learns over many episodes which attacks bypass your filters is only useful if it explores instead of farming one exploit, and only safe if every discovered attack becomes target training data.

The product-analytics staple that quietly tests event ordering, conditional aggregation, and cohort logic at once. Most candidates compute a funnel that double-counts, here's the version that survives interviewer scrutiny.

The warm-up that quietly filters out candidates who only know LLMs as a magic API. Nail the two-phase inference picture and you set up every later answer about latency, cost, and hallucinations.

Everyone can define it; few candidates know why 'advertised context' and 'usable context' aren't the same thing. The cost, latency, and attention caveats that make this answer stand out.

Modern LLMs share one architectural skeleton. Learn the five components and you can read almost any model card cold, plus the single distinction that proves to an interviewer you actually understand it rather than the buzzwords.

Tokens drive your customer's bill, their latency, and the weird failures like botched arithmetic. The rough numbers every FDE should rattle off, plus the trap answers interviewers listen for.

A deceptively simple sampling question with one trap (does temperature 0 fix hallucinations?) and one senior move interviewers reward. Know which workflows want zero.

Sierra's signature design exercise. The grading hinges on three things most candidates underweight: tool API shape, the read/write trust boundary, and eval cases written before the agent. Walkthrough inside.

This is a flight-risk filter, not small talk, FDE teams lose hires back to pure SWE roles constantly. The answer that passes is built on evidence, not enthusiasm. Here's what it sounds like.