TL;DR: They fail on complementary query types, so production systems run both: BM25 for exact tokens (IDs, error codes, names), vectors for paraphrase, merged with Reciprocal Rank Fusion and a cross-encoder reranker on top.
How to approach it
This looks like a definitions question; answer it as a failure-modes question. The interviewer wants to know if you've watched real queries fail. Open with: "they fail on complementary query types, which is why production systems run both."
A strong answer
Keyword search (BM25 / lexical) ranks by term overlap with IDF weighting. It's exact: it nails queries containing identifiers (INV-20419, error 0x80070057), product and person names, rare jargon, and quoted phrases. It fails on vocabulary mismatch, "how do I get my money back" never matches a document titled "Refund Policy" if the words don't overlap, and it has no notion of meaning.
Vector search embeds queries and chunks into a shared space and ranks by cosine similarity. It handles paraphrase, synonyms, and cross-lingual matching: semantic recall. Its failure modes are subtler and more dangerous: it retrieves "related but wrong", a chunk about 2023 pricing for a 2024 pricing question, or the right topic from the wrong customer's contract, because embeddings compress away exactly the discriminating details (numbers, versions, entity names) that lexical search preserves. It also degrades on out-of-distribution jargon the embedding model never saw.
Both failure modes follow from the mechanisms, and being able to say why earns more than the definitions do. BM25's IDF weighting means a term is worth the rarity of the documents it appears in: "the" appears everywhere and counts for nothing, while INV-20419 appears in exactly one document and is worth nearly everything, which is precisely why lexical search is unbeatable on identifiers and precisely why it scores zero when the user's words and the document's words simply differ. The embedding side fails for a mirror-image reason: an embedding is a fixed budget of dimensions summarizing a whole chunk, and that compression spends its capacity on topic, not on particulars. "2023 pricing" and "2024 pricing" produce nearly parallel vectors because almost everything about them is the same; the one token that distinguishes them is exactly what the compression smooths over. Neither behavior is a bug to fix. They are what each mechanism is for, which is the real argument that running both is an architecture and not a hedge.
So the practical answer is hybrid: run BM25 and dense retrieval in parallel, merge with Reciprocal Rank Fusion (RRF) or a weighted score, and ideally pass the merged top-k through a cross-encoder reranker. Each leg rescues the other: BM25 rescues exact-match queries vectors fuzz over; vectors rescue natural-language queries BM25 can't bridge.
Engines like Elasticsearch, OpenSearch, Vespa, and Qdrant support both natively, so hybrid is cheap to adopt. As a rough prior from enterprise deployments: either method alone might land 60–75% recall@10 on a mixed query log; hybrid plus a reranker often reaches 85–90%.
If forced to pick one for an enterprise corpus, say so explicitly: lexical-first with vectors added is often the safer order, because enterprise queries skew heavily toward names, acronyms, and IDs. This is essentially Glean's public position on enterprise search.
Per query type, where each leg lands:
| Query type | BM25 (keyword) | Vector (semantic) |
|---|---|---|
| Exact IDs/codes | Nails them on exact token overlap | Fuzzes away discriminating numbers and versions |
| Paraphrase | Misses on vocabulary mismatch | Handles synonyms and cross-lingual matching |
| Rare jargon | Matches exactly | Degrades on terms the model never saw |
| Out-of-distribution | Preserves the literal tokens | Degrades on jargon outside training |
What interviewers probe next
"How do you combine the scores?" RRF needs no calibration, which is why it's the default; learned weighting needs labeled data. "When would pure vector be fine?" Homogeneous FAQ-style corpora with paraphrase-heavy queries. "What about filters?" Metadata filtering (date, source, ACL) happens alongside both and is frequently the real fix. "How would you prove hybrid helps for this customer?" Split your golden set by query type and show per-slice recall, not one blended number.
Common mistakes
Describing the mechanisms but never the failure modes: that's the difference between reading about retrieval and operating it. Claiming vectors strictly dominate ("embeddings understand meaning"); interviewers at search companies will hand you an ID-lookup query and watch. And forgetting the reranker exists, which is the cheapest large quality win in the whole retrieval stack.
Key takeaways
- BM25 wins on exact tokens; vectors win on paraphrase; their failure modes are complementary, so run both.
- Merge with RRF (no score calibration needed), then rerank with a cross-encoder for the cheapest quality jump.
- Prove the win per query-type slice, not with one blended recall number.