ML System Design (Product): the practice test

It is an economics theorem more than an architecture choice: a heavy ranking model spending even a millisecond per item cannot touch a hundred-million-item catalog inside a 200ms budget, so a cheap, recall-oriented stage (embedding retrieval, popularity, follows) narrows the field to a few hundred, and the precision-oriented ranker spends its capacity only there. Every serious feed, marketplace and video platform converges on this funnel, sometimes with a light pre-ranker between the stages. The design consequence interviewers probe next: the stages need different metrics, recall-at-k for generation, ranking quality for the ranker, because each can fail independently.

The architecture is shaped by a serving constraint: if user and item interact only through a final dot product, then item vectors depend on nothing about the user, so the entire catalog can be embedded offline into an approximate-nearest-neighbor index, and serving becomes one user-tower forward pass plus an ANN lookup, fast at any catalog size. The price is exactly the missing cross-attention: no early user-item feature crossing, which is why a heavier ranker sits downstream to capture what the towers cannot. Understanding that the dot-product bottleneck is the point, not a limitation someone forgot to fix, is what the question screens for.