GenRec: An LLM-Backed Recommendation Ranker at Netflix

Netflix's title ranker is a mature discriminative model built over many years on thousands of engineered features, special networks for feature interactions, sequence models and multi-task heads. It works, and it is expensive to change: a new content type such as live events or podcasts means new features, new architecture work and new pipelines. Ying Li and colleagues at Netflix describe GenRec, an attempt to replace that ranker with a large language model.

GenRec has two training phases. Phase 1, built elsewhere at Netflix and changed rarely, adapts an open-source LLM to Netflix data so it understands the catalog and member behavior. Phase 2, the subject of this paper, post-trains that foundation model for ranking and is refreshed often. A member's history, the titles' metadata and the request's context (device, time, surface) are written as text. The model reads that text once, a pooled hidden state is scored against a learned embedding for every catalog item, and a softmax over the catalog gives the ranking. Because the model never generates a title name, it cannot recommend something Netflix does not have.

Training mixes a ranking loss over the catalog with a language-modeling loss, and weights each example by reward models that estimate long-term satisfaction and rebalance content types. Serving runs on vLLM in prefill-only mode: one forward pass per request, no decoding.

The results: about 40 times fewer Phase-2 labels than production for a 1.6% relative offline gain in mean reciprocal rank, and a four-week online test on about 10% of traffic with statistically significant gains on a short-term engagement metric (+0.115%) and the long-term core metric (+0.006%).

Few FDEs will build a Netflix ranker. Many will be asked to put an LLM behind a ranking, routing, matching or scoring decision that a classical model already makes, and this paper is a rare production account of doing that against a strong incumbent.

The prompt becomes the feature vector. The engineering effort moves from building features to deciding what goes into a fixed token budget: which events to keep, which to drop, what to compress and how far back to go. GenRec cut its prompts from about 5,000 tokens to about 1,700 with no meaningful quality loss, and serving cost fell by about two thirds. Context engineering is the cost lever.

Score, don't generate. Generating title names token by token is slow and can produce items that do not exist. A scoring head over a fixed catalog ranks everything in one pass and makes invalid outputs impossible. The same move applies to any LLM picking from a known list: products, routes, categories, next actions.

Encode business rules as training weights. Instead of reinforcement learning, Netflix scales each example's loss by reward models for long-term value and content mix. It is simpler to run and maintain, which matters more than the extra gain GRPO-style training showed in their preliminary tests.

Read the online result with care. A long-term lift of 0.006% at p = 0.025 is a real result at Netflix's scale and a small one. Before quoting a vendor's "statistically significant improvement" to a customer, ask for the effect size and the interval.