NVIDIA MLOps & ML Engineering interview questions

An experience probe with a depth gauge: 'we used Kubernetes' fails, and so does reciting the KServe README. The stack-process-role-warstory structure, the resource and probe details that prove hands-on work, and where KServe earns its complexity.

CPU-based HPA on GPU inference never fires, the trap half of all candidates fall into within a minute. The signals that actually track load, the anatomy of a five-minute cold start, and which mitigations are worth their cost at each layer.

The cluster shows eight free GPUs and your four-GPU pod still won't schedule, the fragmentation puzzle GPU-platform rounds open with. Bin-packing vs. spreading, MIG/MPS/time-slicing for the small-model problem, and the pod-per-model math that breaks at scale.

The GPU-platform capstone: dozens of models, spiky traffic, a fixed monthly GPU budget, and a p99 SLO that scale-to-zero would wreck. Token-based autoscaling, the three layers of LLM cold start, and the residency tiering that makes the budget math work.

The 'works on my GPU box, CUDA error 803 in prod' bug, solved at the source. What the host driver actually pins, why the toolkit version is fungible, and the multi-stage build that ships a lean reproducible image.

The dataset is too big to fit on disk, so you stream it. The components that keep the GPU fed, the throughput math that says whether you'll be I/O-bound, and the failure handling that makes a multi-day run resumable.

The standard AWS ML Engineer opener. Most candidates recite 'data and model versioning' and stall, the answer interviewers score highest explains why ML tests can't be pass/fail and what continuous training adds as a third loop.

An experience probe that sinks more candidates than any design question, because a vague answer ends the interview early. The four-plane structure that proves you're an operator, not a data scientist with MLOps on the resume.

A bank-round system design where the budget math does the screening: ~10ms for features inside a 100ms decision. Streaming aggregates, the hot-key problem during an active attack, and the parity discipline that makes training data honest.

They answer different questions, and mixing them up costs you the round: one tests your engineering, the other tests your model's business impact. What each can and cannot catch, and the order a strong candidate always runs them in.

Databricks-stack interviewers use this to date your knowledge: describing Staging→Production stage transitions marks you as two years behind. The alias-based promotion flow (@champion/@challenger), gate by gate, with the access-control detail most answers miss.

Logs, traces, and model-quality signals from hundreds of LLM products at firehose volume. The storage split that keeps queries fast, the alerting that survives drift and toxicity at scale, and why sampling has to be cost-aware and tail-biased.

A textbook screen with a non-textbook scoring rubric. Naming Tracking, Projects, Models, and Registry gets you a pass, knowing which component nobody actually uses, and why, gets you the offer conversation.

The trap is that S3 versioning sounds like a complete rebuttal, it isn't, and interviewers at banks know exactly why. What a registry adds that storage never can, and the audit question that decides regulated-industry offers.

The AWS ML Engineer staple. The answer that scores is a clean division of labor, CodePipeline owns code, SageMaker Pipelines owns the ML DAG, the Model Registry is the handoff point, plus the cross-account detail that separates real builds from doc reading.

The bug that makes offline metrics a lie: training on information that didn't exist at prediction time. A concrete fraud example with timestamps, the as-of join that fixes it, and the label-maturity trap automated retraining adds on top.

Reported from Capital One ML loops, and the symmetric 'both have pros and cons' answer fails it. Why model failures being statistical makes canary the default, the two cases where blue-green wins, and the entity-randomization detail that survives follow-ups.

A real customer-interview question that rewards scar tissue: the 29-second timeout wall, retry storms that triple your own load, and connection pools sized for web traffic meeting 30-second inferences. The issues worth claiming and the fixes that prove you were there.

The 45-minute MLOps capstone reported from Capital One and Uber loops. The component walkthrough is table stakes, what's graded is sequencing, the batch-vs-realtime call made early, and knowing which pieces you'd cut for v1.

A staff-level platform design where the constraint set, not the architecture, is the test. The controls SR 11-7 actually demands, how to make audit a byproduct of the pipeline instead of a quarterly fire drill, and the v1 you can ship without a compliance finding.

The incident with no onset spike and no pager: precision bled out over weeks while every dashboard stayed green. How to run a blameless postmortem when the incident has no clean start time, the root-cause classes that hide this long, and the action items that actually prevent the next one.