Scale AI LLM & GenAI Fundamentals interview questions

Every FDE will face an angry customer asking this. The mechanistic explanation interviewers want (why hallucination is the default behavior, not a bug) plus the framing that defuses customer escalations.

A classic metrics question with a domain twist. The answer isn't 'recall'; it's a cost-of-errors argument plus a threshold strategy, and interviewers grade the reasoning, not the metric you pick.

The single most-reported FDE conceptual question across every AI lab. Interviewers don't want definitions; they want the decision framework, the cost math, and the recommendation order that reads senior.

Anyone can list mitigations; the question is really about costs: latency, dollars, and lost recall. The three-layer answer with honest price tags that AI-lab interviewers score highest.

The data-volume thresholds, the break-even math, and the regression-prevention checklist that turn 'just fine-tune it' into a defensible engineering decision. Includes the crossover rule interviewers listen for.

JSON mode, strict schemas, constrained decoding, validate-and-retry: the escalation ladder for structured output, plus the catch nobody mentions: schema-valid can still be factually wrong.

Prompts are production code that nobody treats like code, until an innocent one-line tweak tanks accuracy. The version-eval-canary-rollback pipeline that AI-native interviewers expect by default.

Evals are the #1 probing area in AI-lab FDE loops. The full suite design, golden sets, component metrics, safety red-teaming, CI gates, and the launch-blocking thresholds, in interview-ready form.

Position bias, verbosity bias, self-preference, and the agreement-rate workflow that turns a sloppy judge into eval infrastructure you can defend. AI labs probe this hard; here's the calibrated answer.

The classic FDE escalation. Six hypotheses, silent model updates, input drift, prompt entropy, sampling noise, perception shift, and the eval-replay test that splits them in an afternoon.

Provider model upgrades ship faster than your release cycle and quietly shift behavior. The design that turns 'we hope it's fine' into a blocking gate: a frozen golden set, pairwise diffing, segmented scoring, and a shadow-traffic canary.

The escalation ladder is settled; the at-scale tradeoff is not. Why retries quietly multiply your bill and tail latency, why constrained decoding has its own taxes, and the hybrid that most high-volume pipelines actually run.

Two failure modes, one architecture. Why no single guardrail holds, the defense-in-depth layers that assume the model will be broken, and the design rule that limits the blast radius when prompt filters inevitably fail.

Three stages, one KL leash. The reward-model-plus-PPO loop, the beta*KL term that keeps the policy from drifting off the reference, and how DPO collapses it into a single classification loss by treating the reward as beta*log(pi/pi_ref). The objectives, written correctly.

Win-rate up, facts down is the signature of reward hacking: the policy is over-optimizing the preference proxy and drifting off the reference. The diagnosis (Goodhart on the reward), the immediate lever (tighten the KL/beta anchor), and the held-out factual gate that stops shipping the regression.

PPO needs a separate value network to estimate the baseline for its advantage. GRPO samples a group of answers per prompt and uses their mean reward as the baseline, so the advantage is just how a sample compares to its peers, no critic required.

Budget covers maybe 5M of 100M LLM calls. Run a cheap first stage on everything, route only low-confidence items to the LLM, and use those LLM labels to distill a student that absorbs most of the volume. The cost-vs-coverage math that decides where the threshold goes.

The warm-up that quietly filters out candidates who only know LLMs as a magic API. Nail the two-phase inference picture and you set up every later answer about latency, cost, and hallucinations.

Everyone can define it; few candidates know why 'advertised context' and 'usable context' aren't the same thing. The cost, latency, and attention caveats that make this answer stand out.

Modern LLMs share one architectural skeleton. Learn the five components and you can read almost any model card cold, plus the single distinction that proves to an interviewer you actually understand it rather than the buzzwords.

A deceptively simple sampling question with one trap (does temperature 0 fix hallucinations?) and one senior move interviewers reward. Know which workflows genuinely want zero.

Tokens drive your customer's bill, their latency, and the weird failures like botched arithmetic. The rough numbers every FDE should rattle off, plus the trap answers interviewers listen for.

The concept underneath every RAG system you'll ever deploy. How to explain vectors, cosine similarity, and the one limitation of embeddings that costs real deployments accuracy.