On the Theoretical Limitations of Embedding-Based Retrieval

A single-vector retriever assigns each document one point and ranks documents by similarity to a query point. That arrangement can make some desired groups difficult or impossible to retrieve under the chosen scoring constraints. Writing a better query cannot always repair the geometry of a fixed document representation.

This paper studies that limitation mathematically and with a constructed retrieval benchmark called LIMIT. The benchmark asks simple attribute questions, but arranges relevance so many different document pairs must be recoverable. Several strong embedding models struggle, including on a small corpus. The simplicity of the text makes the failure useful to examine: difficult retrieval does not always require a complicated question.

Read the revised theory carefully. Its dimension lower bound assumes normalized vectors, every possible top-k subset and a positive separation margin. It is not a universal minimum dimension for an ordinary customer corpus. A later paper also challenges the claim that dimensionality alone explains the observed model failures.

The practical lesson is to test the relevance relationships the customer needs. Compare dense retrieval with lexical, structured and multi-vector alternatives on those cases, and measure whether the desired documents reach the candidate set before asking a reranker to fix the order.

An enterprise query may ask for an exact identifier, a combination of attributes or a relationship that differs from general semantic similarity. A retriever can return convincing neighbors while missing the records that satisfy that request. Increasing the language model's size does not repair missing evidence.

Use the paper to distinguish candidate generation from answer generation. First compare the embedding scores under exact search. If the desired document already ranks poorly there, changing ANN search parameters cannot restore it reliably. Then compare alternative representations or retrieval routes using the same relevance labels.

The hybrid search concept explains how complementary candidates can enter the pipeline. Exact constraints such as an approved status or tenant boundary belong in filters or structured queries, with access enforcement independent of similarity. The aim is a retrieval contract that matches the customer's definition of relevance.