ml system design

A feed touches billions of candidate items and has tens of milliseconds to pick the next 10. The interview is about the two-stage architecture, what you optimize when engagement and integrity pull against each other, and the biases that quietly poison your training data.

A track lasts three minutes, a session lasts an hour, and a new song has zero plays the day it drops. The interview is about blending collaborative filtering with audio content embeddings, surviving cold start on both sides, and reading a skip as the loud negative it is.

Anyone can train a pCTR model. The hard part is proving a change is good before it touches revenue, when the model sits inside an auction, the logs only show ads that won, and a 1% calibration error is real money. This is an eval question, not a model question.

Watch time is the label everyone optimizes and almost nobody measures cleanly. You only observe minutes for videos people chose to play, the distribution is brutally skewed, and the position they saw it in moved the number. The interview is whether you can predict a biased label honestly.

The positive class is maybe 1 in 1,000, your labels arrive late and noisy, and the moment you ship a model the adversary probes it. This is the rare ML problem where the data is actively trying to beat you, so the design is as much about labels, latency, and enforcement cost as about the classifier.

This model's output is not a ranking, it is a price input. A miscalibrated CTR makes the auction bid wrong, so you overpay or underdeliver. Add conversions that land days after the click and advertisers with zero history, and calibration plus delayed feedback become the whole interview.

An ETA is a promise. The interview is whether you model it as a point estimate (and apologize when wrong) or as a distribution where p90 lets you under-promise, plus how you fold live traffic and finished trips back into the model.

There are millions of landmarks, most with a handful of photos, and the next photo might be of something not in your catalog at all. A flat classifier dies on the long tail and never says 'I don't know.' The interview is embeddings plus retrieval plus a confident refusal.

Fraud is well under 1% of transactions, the labels arrive weeks late as chargebacks, and the fraudsters adapt to your model on purpose. Optimizing accuracy gives you a model that approves everything. The interview is dollars-at-risk thresholding, label delay, and a review queue with a fixed headcount.