Embeddings & Vector Representations
An embedding turns a piece of text into a list of numbers positioned so that similar meanings land near each other in space, which lets you search by meaning instead of by keyword. Embeddings are the engine under RAG, semantic search, clustering, and deduplication, so FDE loops expect you to explain cosine similarity and the pitfalls that quietly break a vector index.
TL;DR: An embedding is a vector of numbers that captures the meaning of text, placed so that semantically similar items sit close together and you can rank relevance by the angle between vectors (cosine similarity). The rule that breaks most first attempts: you must index and query with the exact same embedding model, or the geometry no longer lines up and your search returns noise.
The intuition
Keyword search matches strings. Search "car" and you miss every document that said "automobile." Embeddings fix this by mapping text to a point in a high-dimensional space (often hundreds to a couple thousand dimensions) where position encodes meaning. "Car" and "automobile" land near each other; "car" and "banana" land far apart. The model learned this geometry from enormous amounts of text, so directions in the space roughly correspond to semantic relationships.
Once your text is a vector, "find similar things" becomes "find nearby vectors," a math problem with fast, well-understood solutions.
Measuring similarity
The standard measure is cosine similarity: the cosine of the angle between two vectors. It ranges from -1 to 1, where 1 means the same direction (very similar), 0 means unrelated, and negative means opposed. Cosine looks at direction, not length, which is why most pipelines normalize vectors to unit length first. After normalization the dot product equals the cosine directly, and ranking by Euclidean distance gives the same order (the nearest by distance is the most similar by cosine), so a vector index can use whichever is cheaper.
| Measure | What it compares | When the ranking is safe |
|---|---|---|
| Cosine similarity | The angle between vectors, length ignored | Always, for vectors from one model |
| Dot product | Angle and length together | Only after normalizing to unit length, when it equals cosine |
| Euclidean (L2) distance | Straight-line distance | After normalization, when nearest by distance is nearest by cosine |
In the flowchart all three sentences pass through the same box, and that box is the contract. Swap it for a different model and the "shared space" on the right stops being shared, with no error to tell you so.
Where you use them
- Semantic search and RAG. Embed your documents once, embed the query at request time, return the nearest neighbors. This is the retrieval half of RAG.
- Clustering. Group similar items (support tickets, product reviews) without predefined labels by clustering their vectors.
- Deduplication and near-duplicate detection. Two near-identical paragraphs sit almost on top of each other; a similarity threshold catches them even when the wording differs.
- Classification and recommendation. Nearest-neighbor over labeled examples gives a cheap, retrain-free classifier, and "items near this one" gives recommendations.
Read the diagram as two clocks running at different speeds. The top half runs once, when you build the index; the bottom half runs on every query. Step 7 is where most production failures start, because nothing errors when the two halves disagree.
The pitfalls that bite
A few mistakes recur in production:
- Mismatched models. Indexing with one embedding model and querying with another produces vectors in incompatible spaces. The numbers are still numbers, so nothing errors; results just quietly become garbage. If you upgrade the model, you must re-embed the entire corpus.
- Domain mismatch. A general-purpose model may not separate the fine distinctions in legal, medical, or internal-jargon text. Evaluate on your own data before trusting it. That is the measured conclusion, not caution: Muennighoff et al. (2022, 'MTEB: Massive Text Embedding Benchmark') ran 33 models across 8 task types, 58 datasets and 112 languages and found no single embedding method dominated across all tasks.
- Forgetting normalization. Mixing normalized and unnormalized vectors, or assuming the database uses cosine when it defaults to L2, gives subtly wrong rankings.
- Treating similarity as relevance. Cosine measures semantic closeness, not correctness for the task. Two sentences can be near-identical in meaning yet neither answers the user. This is why rerankers exist on top of vector search.
Why interviewers probe this
The answer that loses is "we'd upgrade to the better embedding model and re-index new documents as they come in." Half the corpus is now in one space and half in another, nothing errors, and search quietly returns noise. It tests whether you understand the layer beneath RAG rather than just calling an API. The screening follow-up is usually "you re-indexed half your documents with a new embedding model, what happens?" The answer they want: the two halves are in different spaces, similarity scores across them are meaningless, and you must re-embed everything with one model. A candidate who knows that has actually run an index in anger.
Common misconceptions
- "Embeddings understand text the way a person does." They capture statistical co-occurrence of meaning learned from training data, not grounded understanding. Out-of-domain text degrades them.
- "Any two embeddings can be compared." Only if they came from the same model. Cross-model comparisons are not meaningful.
- "Higher cosine similarity always means a better answer." It means semantically closer, which is not the same as relevant or correct.
- "More dimensions are always better." Larger vectors cost more storage and compute per query; the right size is a measured trade-off, not a maximum.
Key takeaways
- An embedding places text in a vector space where proximity means semantic similarity, enabling search by meaning.
- Rank with cosine similarity; normalize to unit vectors so direction is what counts.
- Always index and query with the same model, and re-embed the whole corpus if you change models.
- Semantic closeness is not the same as relevance or correctness, which is why reranking sits on top of raw vector search.
- When you change embedding models, re-embed everything in one job and cut over at once. A half-migrated index has no error message.