TL;DR: An embedding is a fixed-length vector where semantically similar text lands close together, so "find related text" becomes "find nearby vectors" (usually cosine, often via ANN like HNSW). The limitation worth naming: embeddings score topical similarity, not correctness or exact identifiers, so production retrieval goes hybrid.
How to approach it
Give the intuition first (meaning as geometry), then the mechanics (model, dimensions, similarity metric), then the operational details an FDE actually touches: which model, what it costs, where it breaks. Interviewers at Cohere in particular expect you to talk about embeddings like a practitioner, not a textbook.
A strong answer
An embedding is a fixed-length vector of numbers, typically 256 to 3,072 dimensions, produced by a neural network trained so that semantically similar text lands close together in vector space. "How do I reset my password?" and "I'm locked out of my account" share almost no words, but their embeddings are near neighbors. That is the whole trick: meaning becomes geometry, and "find related text" becomes "find nearby vectors."
A toy makes the geometry concrete. Pretend the model only had three dimensions, roughly "account trouble," "asking for help," and "finance" (real models have thousands of unlabeled ones, but the arithmetic is identical). "How do I reset my password?" might embed as [0.9, 0.3, 0.1] and "I'm locked out of my account" as [0.8, 0.5, 0.2]: cosine similarity 0.97, near neighbors despite sharing no keywords. "Q3 revenue by region" at [0.1, 0.2, 0.9] scores 0.27 against the password query. Those numbers are computed from the toy vectors, not invented, and the point of the exercise is that no string matching happened anywhere; similarity fell out of position alone. Everything a vector database does is this comparison, scaled up.
Semantic search is then a three-step pipeline. At indexing time, you chunk your documents and embed each chunk, storing vectors in a vector index (pgvector, Pinecone, Elasticsearch, FAISS). At query time, you embed the user's query with the same model and run nearest-neighbor search, usually cosine similarity, to get the top-k chunks. At scale you use approximate nearest neighbor algorithms (HNSW is the common one) because exact search over tens of millions of vectors is too slow; ANN trades a sliver of recall for orders-of-magnitude speed.
Practical fluency points worth volunteering: embedding models are separate from and far cheaper than generation models (fractions of a cent per thousand chunks); you must embed queries and documents with the same model and version, because mixing models gives garbage similarity scores; and re-embedding an entire corpus is the hidden migration cost when you switch embedding models, which is why teams pin versions.
Being able to size an index out loud is the same kind of fluency. Ten million chunks at 1,536 dimensions, four bytes per float, is 10M x 1,536 x 4 ≈ 61 GB of raw vectors before the ANN graph's own overhead. That one line of arithmetic explains several production decisions at once: why serious deployments care about dimension count (a 3,072-dim model doubles that bill for a quality gain you should demand evidence of), why quantization to int8 or binary exists (a 4x to 32x memory cut for a small recall cost), and why "just put it in memory" stops being an answer somewhere between one million and one hundred million vectors. Interviewers rarely ask for this directly, but dropping it while discussing scale is how practitioners sound different from textbooks.
The key limitation: embeddings capture topical similarity, not correctness or exactness. They will happily retrieve a passage about "2023 pricing" for a "2024 pricing" query: related but wrong. They are also weak on exact identifiers (part numbers, error codes, names) where old-fashioned keyword search wins. That is why production retrieval is usually hybrid (vectors plus BM25 keyword search plus a reranker) rather than embeddings alone.
What interviewers probe next
- "Why cosine similarity?" It measures angle, not magnitude, so document length doesn't dominate; many models normalize vectors so cosine and dot product coincide.
- "How do you evaluate retrieval quality?" A labeled set of query-to-relevant-chunk pairs; recall@k and MRR; don't tune chunk size or k blind.
- "What else are embeddings used for?" Clustering and dedup, classification, recommendation, anomaly detection on logs and tickets; showing breadth here reads well.
- "Same words, different meaning, 'java' the island vs the language?" Context within the chunk helps; metadata filters and rerankers do the rest.
Common mistakes
- Explaining the math (dot products, dimensions) but unable to name a single vector store, ANN algorithm, or rough cost: the FDE failure mode.
- Claiming embeddings "understand" text. Overselling leads directly into the related-but-wrong trap and the interviewer knows it.
- Forgetting the same-model-for-query-and-docs requirement; it is a real production bug interviewers have seen candidates miss.
- Presenting pure vector search as sufficient. Not mentioning hybrid search or rerankers signals you have never debugged retrieval in production.
Key takeaways
- Embeddings turn meaning into geometry; nearest-neighbor (ANN/HNSW) makes the search tractable at scale.
- Embed queries and documents with the identical model and version, or every score is garbage.
- Pure vectors miss exact identifiers and recency; production retrieval is hybrid (vectors plus BM25 plus reranker).