Hugging Face AI & ML Engineer interview questions

Batching inputs is how you keep a GPU fed, but the source is an iterator you cannot index or len(). The warm-up that screens whether you can write a lazy generator that handles the ragged final batch.

Every chat model decodes with top-p, but few candidates can build it from logits: softmax, sort, take the smallest set of tokens whose cumulative probability crosses p, renormalize, sample. The screen for whether you understand decoding, not just call an API.

The loss every language model trains on, built from logits with no torch in sight. The screen for whether you understand log-softmax and the numerical-stability trick, not just import nn.CrossEntropyLoss.

BPE is how GPT and most LLMs turn text into tokens, and building a tiny trainable version from a corpus is a favorite deep-dive. The screen for whether you understand the merge loop, deterministic tie-breaking, and round-trip decode, not just tiktoken.encode.

The 'works on my GPU box, CUDA error 803 in prod' bug, solved at the source. What the host driver actually pins, why the toolkit version is fungible, and the multi-stage build that ships a lean reproducible image.

The dataset is too big to fit on disk, so you stream it. The components that keep the GPU fed, the throughput math that says whether you'll be I/O-bound, and the failure handling that makes a multi-day run resumable.

Three serving runtimes that look interchangeable on a benchmark slide and aren't. The axes that actually separate them (scheduler, kernels, operability) and the one-line rule for which to reach for.

Do the weight math first, then choose your poison: shrink the weights, split them across GPUs, or spill to host memory. Each buys you the fit at a different cost, and one of them is almost always wrong for production.

The warm-up that quietly filters out candidates who only know LLMs as a magic API. Nail the two-phase inference picture and you set up every later answer about latency, cost, and hallucinations.

Everyone can define it; few candidates know why 'advertised context' and 'usable context' aren't the same thing. The cost, latency, and attention caveats that make this answer stand out.

Modern LLMs share one architectural skeleton. Learn the five components and you can read almost any model card cold, plus the single distinction that proves to an interviewer you actually understand it rather than the buzzwords.

Tokens drive your customer's bill, their latency, and the weird failures like botched arithmetic. The rough numbers every FDE should rattle off, plus the trap answers interviewers listen for.

Every FDE will face an angry customer asking this. The mechanistic explanation interviewers want (why hallucination is the default behavior, not a bug) plus the framing that defuses customer escalations.

A deceptively simple sampling question with one trap (does temperature 0 fix hallucinations?) and one senior move interviewers reward. Know which workflows want zero.

The most common screen opener, and interviewers use it to check whether hashmap thinking is reflexive. Here's the one-pass answer, the narration that earns points, and the duplicate-handling edge case most candidates fumble.

The concept underneath every RAG system you'll ever deploy. How to explain vectors, cosine similarity, and the one limitation of embeddings that costs real deployments accuracy.

A classic metrics question with a domain twist. The answer isn't 'recall'; it's a cost-of-errors argument plus a threshold strategy, and interviewers grade the reasoning, not the metric you pick.

JSON mode, strict schemas, constrained decoding, validate-and-retry: the escalation ladder for structured output, plus the catch nobody mentions: schema-valid can still be factually wrong.

Every FDE loop opens with this, but the definitions are table stakes. What gets scored is whether you can map each metric to a dollar cost the customer recognizes, here's the framing that does it.

The same operation scores attention and ranks your RAG results. Knowing exactly what the dot product measures, and when its magnitude term quietly breaks your retrieval, is what separates a working pipeline from a mysteriously bad one.

The reported ElevenLabs live round, written in a shared Google Doc, no execution, no autocomplete. Inherited folder permissions plus permission-change timestamps, and the two-axis resolution rule that keeps the code to thirty lines.

Anyone can list mitigations; the question is really about costs: latency, dollars, and lost recall. The three-layer answer with honest price tags that AI-lab interviewers score highest.

The interval pattern shows up everywhere in FDE loops: calendars, flight segments, log windows. Master the sort-then-sweep idiom here and three other interview questions fall out for free.

Why a raw base model answers your question with three more questions, what RLHF actually buys, and the niche cases where customers should pick base. A tidy answer most candidates fumble.