NVIDIA MLOps & ML Engineering interview questions

An experience probe with a depth gauge: 'we used Kubernetes' fails, and so does reciting the KServe README. The stack-process-role-warstory structure, the resource and probe details that prove hands-on work, and where KServe earns its complexity.

CPU-based HPA on GPU inference never fires, the trap half of all candidates fall into within a minute. The signals that actually track load, the anatomy of a five-minute cold start, and which mitigations are worth their cost at each layer.

The cluster shows eight free GPUs and your four-GPU pod still won't schedule, the fragmentation puzzle GPU-platform rounds open with. Bin-packing vs. spreading, MIG/MPS/time-slicing for the small-model problem, and the pod-per-model math that breaks at scale.

The GPU-platform capstone: dozens of models, spiky traffic, a fixed monthly GPU budget, and a p99 SLO that scale-to-zero would wreck. Token-based autoscaling, the three layers of LLM cold start, and the residency tiering that makes the budget math work.

The 'works on my GPU box, CUDA error 803 in prod' bug, solved at the source. What the host driver actually pins, why the toolkit version is fungible, and the multi-stage build that ships a lean reproducible image.

The dataset is too big to fit on disk, so you stream it. The components that keep the GPU fed, the throughput math that says whether you'll be I/O-bound, and the failure handling that makes a multi-day run resumable.

The standard AWS ML Engineer opener. Most candidates recite 'data and model versioning' and stall, the answer interviewers score highest explains why ML tests can't be pass/fail and what continuous training adds as a third loop.

A textbook screen with a non-textbook scoring rubric. Naming Tracking, Projects, Models, and Registry gets you a pass, knowing which component nobody actually uses, and why, gets you the offer conversation.

An experience probe that sinks more candidates than any design question, because a vague answer ends the interview early. The four-plane structure that proves you're an operator, not a data scientist with MLOps on the resume.

The trap is that S3 versioning sounds like a complete rebuttal, it isn't, and interviewers at banks know exactly why. What a registry adds that storage never can, and the audit question that decides regulated-industry offers.

They answer different questions, and mixing them up costs you the round: one tests your engineering, the other tests your model's business impact. What each can and cannot catch, and the order a strong candidate always runs them in.

Logs, traces, and model-quality signals from hundreds of LLM products at firehose volume. The storage split that keeps queries fast, the alerting that survives drift and toxicity at scale, and why sampling has to be cost-aware and tail-biased.

Candidates describe a feature store as 'a database for features' and fail, it's a consistency machine with two faces. The online/offline split, the latency numbers each side must hit, and when you genuinely shouldn't build one.

A real interview question designed to catch resume inflation: candidates say 'we promote to production' and can't name the API call. What actually moves when a model is promoted, and the answer that proves you've done it.

'We checked accuracy' is the answer that ends interviews. The five-layer test pyramid for ML pipelines, data contracts, transform unit tests, behavioral checks, serving tests, shadow validation, with the gates that actually block promotion.

The honest answer includes an admission most candidates are afraid to make: bit-exact GPU reproducibility is mostly a myth. The five things you version, the audit question banks actually ask, and why 'statistically reproducible' is the senior answer.

Databricks-stack interviewers use this to date your knowledge: describing Staging→Production stage transitions marks you as two years behind. The alias-based promotion flow (@champion/@challenger), gate by gate, with the access-control detail most answers miss.

The AWS ML Engineer staple. The answer that scores is a clean division of labor, CodePipeline owns code, SageMaker Pipelines owns the ML DAG, the Model Registry is the handoff point, plus the cross-account detail that separates real builds from doc reading.

Microsoft's scenario rounds reward one specific framing: two pipeline systems, two jobs, one handoff point. Which tool owns triggers and approvals, which owns compute and lineage, and the managed-endpoint detail that makes blue-green trivial.

Auto-retrain is easy; auto-promote is where teams ship incidents. The champion/challenger gate design, two eval sets, segment floors, guardrail metrics, and the defensible line on when a human stays in the loop.

'Retrain weekly' is the answer interviewers let you finish before dismantling it. The four trigger types, how to stop a drift alarm from causing a retraining storm, and why the trigger question is really a cost-and-staleness trade.