Precise Zero-Shot Dense Retrieval without Relevance Labels

Dense retrieval embeds queries and documents into vectors and finds documents whose vectors are close to the query's. It works best after training on large sets of judged query-document pairs such as MS MARCO. Without them, an unsupervised encoder like Contriever, trained only to tell similar documents from dissimilar ones, often does worse than BM25.

HyDE (Hypothetical Document Embeddings) sidesteps the need to learn query-document relevance. Given a query, an instruction-following language model is asked to "write a passage to answer the question". The passage is not retrieved from anywhere; it may contain false details. An unsupervised contrastive encoder then embeds it. Because that encoder was trained on document-to-document similarity, the hypothetical document's vector lands near real documents that look like an answer. The authors argue the encoder's dense bottleneck acts as a lossy compressor that keeps the relevant pattern and drops much of the invented detail.

In practice, several hypothetical documents are sampled and their vectors averaged, optionally together with the query's own vector. The instruction is adjusted per task: a scientific passage for SciFact, a counter-argument for ArguAna, a financial article for FiQA, a news passage for TREC-NEWS, and a passage in the target language for multilingual search. No model is trained or fine-tuned.

With InstructGPT (text-davinci-003) and Contriever, HyDE outperformed Contriever and BM25 on web search (TREC DL19 and DL20), beat BM25 on five of six low-resource BEIR datasets by nDCG, and improved retrieval in Swahili, Korean, Japanese and Bengali. It came close to Contriever fine-tuned on MS MARCO in some settings and stayed behind it in others. Smaller generators (Flan-T5 11B, a 52B Cohere model) gave smaller gains.

It is a fast start for a new corpus. On day one with a customer's documents, there are no search logs and no relevance labels. HyDE gives a meaningful lift over an unsupervised encoder with only a prompt, and the authors suggest using it until enough usage data exists to train a supervised retriever.

It closes the gap between question and document language. Users ask short questions; documents are written as explanations, policies or reports. Writing a hypothetical answer moves the search into document language. In our toy, a question about "pulling a third molar" found the extraction document only after a hypothetical answer used the document's words.

It fails on what the model does not know. For internal names, codes and products, the generator writes about something else, and the search follows it. In our toy, a query about an internal flag name retrieved a document about a telescope that shares part of the name. Keep a keyword path (BM25 or hybrid search) for exact terms and identifiers.

It costs a generation per query. Each search becomes one or more LLM calls plus encoding, which adds latency and spend. Cache hypothetical documents for frequent queries, keep them short, and measure the gain against a cheaper query rewrite or a hybrid retriever before shipping.

Quality depends on the generator and the encoder. With an 11B generator the DL19 gain was 4.4 points instead of 16.8; on a fine-tuned encoder, the two smaller generators made results slightly worse. Measure on the customer's queries with the model you will actually deploy.