Google LLM & GenAI Fundamentals interview questions

The warm-up that quietly filters out candidates who only know LLMs as a magic API. Nail the two-phase inference picture and you set up every later answer about latency, cost, and hallucinations.

Modern LLMs share one architectural skeleton. Learn the five components and you can read almost any model card cold, plus the single distinction that proves to an interviewer you actually understand it rather than the buzzwords.

Everyone can define it; few candidates know why 'advertised context' and 'usable context' aren't the same thing. The cost, latency, and attention caveats that make this answer stand out.

TTFT vs tokens-per-second, the output-length lever everyone forgets, and why streaming is the highest-ROI fix that changes no latency at all. The five-lever answer with real numbers.

The KV-cache mechanics behind the discount, the prefix rule that silently breaks caching for most teams, and the workloads where caching cuts bills 50-90% versus the ones where it does nothing.

Zero retention, private endpoints, redaction pipelines, or open weights in their VPC, the four-rung ladder for PII-sensitive deployments, and the discovery questions that pick the right rung.

10x doesn't come from one trick. It's multiplicative: routing x caching x prompt diet x batching x distillation. The stacked-levers playbook with rough multipliers, and the eval gate that keeps quality honest.

In high-stakes domains a wrong answer is worse than no answer, but a system that always abstains is useless. The selective-prediction design, the layered abstention triggers, and the two metrics that govern the tradeoff.

Million-token windows didn't kill RAG; they changed when you reach for it. The decision rule that holds up in production, and the silent failure each option hides behind a confident answer.

Three transformer families, one distinction that decides everything: which tokens are allowed to see which. The attention mask is the whole answer, and it explains why GPT generates and BERT does not.

Rotary embeddings encode position as a rotation of the query and key vectors, so attention scores depend only on the gap between two tokens. That relative property is why RoPE generalizes past its training length where learned absolute tables fall off a cliff.

Where you put the LayerNorm changes whether gradients survive a hundred layers. Pre-norm keeps a clean residual highway and well-scaled gradients so deep models train without warmup heroics; post-norm normalizes the output but fights vanishing gradients at depth.

Attention is bottlenecked by moving the N x N score matrix to and from HBM, not by FLOPs. FlashAttention tiles the computation in fast on-chip SRAM and fuses softmax with an online running statistic, so it computes the exact same output while never materializing the full matrix.

Each chat turn resends the whole conversation, so the model re-prefills the same prefix every time. Server-side KV caching and client-side cache breakpoints kill that redundant compute, but exact-prefix matching and stale entries are where teams silently get it wrong.

PPO needs a separate value network to estimate the baseline for its advantage. GRPO samples a group of answers per prompt and uses their mean reward as the baseline, so the advantage is just how a sample compares to its peers, no critic required.

Without a KV cache, generating token N re-attends over all N-1 prior tokens from scratch every step, turning decoding quadratic. The cache stores each layer's past keys and values so each new token costs one forward step, and its size is what caps your batch and context.

A fixed 1000-token budget forces the search choice. Step-level beam search guided by a process-reward model beats best-of-N and MCTS on accuracy per token for math, and you collect the PRM labels automatically with Monte-Carlo rollouts, no human step annotation.

A deceptively simple sampling question with one trap (does temperature 0 fix hallucinations?) and one senior move interviewers reward. Know which workflows genuinely want zero.

Tokens drive your customer's bill, their latency, and the weird failures like botched arithmetic. The rough numbers every FDE should rattle off, plus the trap answers interviewers listen for.

JSON mode, strict schemas, constrained decoding, validate-and-retry: the escalation ladder for structured output, plus the catch nobody mentions: schema-valid can still be factually wrong.

The concept underneath every RAG system you'll ever deploy. How to explain vectors, cosine similarity, and the one limitation of embeddings that costs real deployments accuracy.

Every FDE will face an angry customer asking this. The mechanistic explanation interviewers want (why hallucination is the default behavior, not a bug) plus the framing that defuses customer escalations.

The same operation scores attention and ranks your RAG results. Knowing exactly what the dot product measures, and when its magnitude term quietly breaks your retrieval, is what separates a working pipeline from a mysteriously bad one.