Vector Databases
A vector database stores embeddings alongside metadata and answers nearest-neighbor queries fast using approximate indexes. The real interview question is not how they work but when you actually need one instead of a library or plain Postgres with pgvector.
Transcript
You've embedded your documents, and somebody on the team says that the next step, obviously, is a vector database. How they work is the easy part. The question interviewers actually ask is when you need one, instead of a library or plain Postgres with pgvector.
It holds three things per record: the vector itself, a float array of, say, 768 or 1536 dimensions, then the metadata you filter and display on, like the document id, the tenant, a timestamp and the original text, and an index that makes search fast. The vector is the search key. The metadata is how you filter and cite.
The query is: give me the k vectors closest to this one. A brute-force scan compares the query against every stored vector. At ten thousand rows, that's fine. At fifty million vectors of 1536 dimensions, it's about 77 billion multiply-adds for a single query, before anything has been ranked. That's hopeless.
So these systems build an approximate nearest neighbor index, usually HNSW. It keeps a few layers of graph, sparse at the top and dense at the bottom. A query walks greedily from a coarse entry point, at each hop moving to whichever neighbor is closer, and works its way down through the layers. It visits a few hundred vectors instead of fifty million.
Approximate means it trades a little recall for a large speedup. Recall is the share of the true nearest neighbors the index actually returns. Find nine of the real top ten, and that's 90 percent recall on that query. Widen the search and it visits more and finds more, at the cost of latency. That's the knob you tune.
Now the call. If you already run Postgres and have under a few million vectors, add pgvector and skip the new dependency: one database to back up, joins with your business tables, and transactional filters for free. FAISS is a library, with no persistence, no metadata and no concurrency, so it suits a notebook. A dedicated system earns its place past a few million vectors, under fifty milliseconds at p99, or with heavy hybrid and filtering per tenant.
A query is rarely nearest globally. It's nearest among documents this user can see. Naive filtering runs the search first, then drops what's not allowed, which can leave you with two hits when you asked for ten. Good systems apply the filter during the graph walk. Get tenant scoping wrong and you have a data-leak bug.
Then they probe updates, because documents change. You need an update-or-insert path keyed on document id, and a plan for deletes, since a stale vector means a confidently wrong citation. New embedding model? Reindex. And before you trust the speed, measure recall against a brute-force baseline.
So in the room, name the crossover point: Postgres with pgvector until a few million vectors, strict latency or heavy filtering push you past it. Then name the cost of a second store you have to keep consistent. The full decision is on fdeinterviews.com.
TL;DR: A vector database stores embeddings plus their metadata and serves approximate nearest-neighbor search, so you can find the chunks closest to a query vector in milliseconds. Most teams do not need a dedicated one on day one: Postgres with pgvector covers you to a few million vectors, and you reach for Pinecone or Weaviate when scale, recall tuning, or filtered search outgrow it.
What it actually stores
A vector database holds three things per record: the vector itself (a float array, say 768 or 1536 dimensions from your embedding model), the payload or metadata you filter and display on (document id, tenant, timestamp, the original text), and an index that makes search fast. The vector is the search key. The metadata is everything you need to filter results and cite sources.
The query is "give me the k vectors closest to this one," measured by cosine similarity or dot product. A brute-force scan compares the query against every stored vector, which is O(n) and fine for ten thousand rows but hopeless at fifty million. Put numbers on it: fifty million vectors of 1,536 dimensions is about 77 billion multiply-adds for a single query, before anything has been ranked. So these systems build an approximate-nearest-neighbor (ANN) index, usually HNSW, a navigable graph that finds near neighbors by hopping between nodes instead of scanning everything. ANN trades a little recall for a large speedup, and that recall knob is the thing you tune. Recall here means the share of the true nearest neighbors the index actually returns; an index that finds nine of the real top ten has 90 percent recall on that query. HNSW keeps a few layers of graph, sparse at the top and dense at the bottom, and a query walks greedily from a coarse entry point down through the layers, at each hop moving to whichever neighbor is closer, so it visits a few hundred vectors instead of fifty million. The parameters that widen the search visit more and find more, at the cost of latency, which is the trade the recall knob exposes.
When you actually need one
This is the decision interviewers care about, so make the call.
Steps 1 to 4 happen once per document and steps 5 to 8 happen on every query. Which half you are paying for decides whether you need a dedicated store or a table you already have.
If you already run Postgres and have under a few million vectors, add the pgvector extension and skip the new dependency. One database to back up, one place for joins between vectors and your business tables, transactional metadata filters for free. FAISS is a library, not a database: it gives you a fast in-memory index with no persistence, no metadata, no concurrency, so it suits a notebook or a read-only batch job, not a live service. Dedicated systems (Pinecone managed, Weaviate or Qdrant self-hosted) earn their place when you cross a few million vectors, need sub-50ms p99 under load, or want first-class hybrid search and per-tenant filtering without hand-tuning Postgres.
Match the option to your scale, latency, and filtering needs.
| Option | Best fit |
|---|---|
| Postgres + pgvector | Already on Postgres, under a few million vectors; one store, joins, transactional filters |
| FAISS | Notebook or read-only batch job; no persistence, metadata, or concurrency |
| Hosted (Pinecone, Weaviate) | Past a few million vectors, sub-50ms p99, or heavy hybrid and per-tenant filtering |
Metadata filtering is where it gets hard
A query is rarely "nearest globally." It is "nearest among documents this user can see, from the last 90 days." Naive filtering runs ANN first and then drops disallowed results, which can leave you with two hits when you asked for ten because the index returned neighbors you were not allowed to see. Good systems do filtered ANN, applying the predicate during the graph walk. Ask how a candidate scopes by tenant: getting this wrong is a data-leak bug, not a relevance bug.
Why interviewers probe this
They want to catch the reflex of bolting on a trendy database when a Postgres extension would do. The strong answer names the crossover point and the operational cost: a separate store means a second system to keep consistent with your source of truth, a reindex whenever you change embedding models, and recall that silently degrades as you push HNSW for speed. They also probe updates. Vectors are not static; documents change. You need an upsert path keyed on document id and a plan for deletes, because a stale vector returns a confidently wrong citation.
Common misconceptions
- "A vector database is a kind of magic search." It returns geometric neighbors of an embedding. If your embedding model is weak or your chunks are bad, faster ANN just returns the wrong things faster.
- "FAISS and Pinecone are interchangeable." FAISS is an index library with no persistence or metadata; the others are services. Different jobs.
- "More dimensions means better results." Dimension is fixed by your embedding model and costs memory and latency. It is not a quality dial.
- "Recall is 100%." ANN is approximate by design. Measure recall against a brute-force baseline before you trust the speed.
Key takeaways
- A vector database = embeddings + metadata + an ANN index (usually HNSW) for fast nearest-neighbor search.
- Default to Postgres with pgvector under a few million vectors; reach for Pinecone or Weaviate at larger scale, strict latency, or heavy filtered search.
- FAISS is an in-process library, not a database: no persistence, no metadata, no concurrency.
- Metadata filtering and per-tenant scoping are correctness concerns, not features; get them wrong and you leak or lose results.
- Plan for upserts, deletes, and reindexing on model changes, and verify ANN recall against a brute-force baseline.