Vector Databases
A vector database stores embeddings alongside metadata and answers nearest-neighbor queries fast using approximate indexes. The real interview question is not how they work but when you actually need one instead of a library or plain Postgres with pgvector.
TL;DR: A vector database stores embeddings plus their metadata and serves approximate nearest-neighbor search, so you can find the chunks closest to a query vector in milliseconds. Most teams do not need a dedicated one on day one: Postgres with pgvector covers you to a few million vectors, and you reach for Pinecone or Weaviate when scale, recall tuning, or filtered search outgrow it.
What it actually stores
A vector database holds three things per record: the vector itself (a float array, say 768 or 1536 dimensions from your embedding model), the payload or metadata you filter and display on (document id, tenant, timestamp, the original text), and an index that makes search fast. The vector is the search key. The metadata is everything you need to filter results and cite sources.
The query is "give me the k vectors closest to this one," measured by cosine similarity or dot product. A brute-force scan compares the query against every stored vector, which is O(n) and fine for ten thousand rows but hopeless at fifty million. So these systems build an approximate-nearest-neighbor (ANN) index, usually HNSW, a navigable graph that finds near neighbors by hopping between nodes instead of scanning everything. ANN trades a little recall for a large speedup, and that recall knob is the thing you tune.
When you actually need one
This is the decision interviewers care about, so make the call.
Steps 1 to 4 happen once per document and steps 5 to 8 happen on every query. Which half you are paying for decides whether you need a dedicated store or a table you already have.
If you already run Postgres and have under a few million vectors, add the pgvector extension and skip the new dependency. One database to back up, one place for joins between vectors and your business tables, transactional metadata filters for free. FAISS is a library, not a database: it gives you a fast in-memory index with no persistence, no metadata, no concurrency, so it suits a notebook or a read-only batch job, not a live service. Dedicated systems (Pinecone managed, Weaviate or Qdrant self-hosted) earn their place when you cross a few million vectors, need sub-50ms p99 under load, or want first-class hybrid search and per-tenant filtering without hand-tuning Postgres.
Match the option to your scale, latency, and filtering needs.
| Option | Best fit |
|---|---|
| Postgres + pgvector | Already on Postgres, under a few million vectors; one store, joins, transactional filters |
| FAISS | Notebook or read-only batch job; no persistence, metadata, or concurrency |
| Hosted (Pinecone, Weaviate) | Past a few million vectors, sub-50ms p99, or heavy hybrid and per-tenant filtering |
Metadata filtering is where it gets hard
A query is rarely "nearest globally." It is "nearest among documents this user can see, from the last 90 days." Naive filtering runs ANN first and then drops disallowed results, which can leave you with two hits when you asked for ten because the index returned neighbors you were not allowed to see. Good systems do filtered ANN, applying the predicate during the graph walk. Ask how a candidate scopes by tenant: getting this wrong is a data-leak bug, not a relevance bug.
Why interviewers probe this
They want to catch the reflex of bolting on a trendy database when a Postgres extension would do. The strong answer names the crossover point and the operational cost: a separate store means a second system to keep consistent with your source of truth, a reindex whenever you change embedding models, and recall that silently degrades as you push HNSW for speed. They also probe updates. Vectors are not static; documents change. You need an upsert path keyed on document id and a plan for deletes, because a stale vector returns a confidently wrong citation.
Common misconceptions
- "A vector database is a kind of magic search." It returns geometric neighbors of an embedding. If your embedding model is weak or your chunks are bad, faster ANN just returns the wrong things faster.
- "FAISS and Pinecone are interchangeable." FAISS is an index library with no persistence or metadata; the others are services. Different jobs.
- "More dimensions means better results." Dimension is fixed by your embedding model and costs memory and latency. It is not a quality dial.
- "Recall is 100%." ANN is approximate by design. Measure recall against a brute-force baseline before you trust the speed.
Key takeaways
- A vector database = embeddings + metadata + an ANN index (usually HNSW) for fast nearest-neighbor search.
- Default to Postgres with pgvector under a few million vectors; reach for Pinecone or Weaviate at larger scale, strict latency, or heavy filtered search.
- FAISS is an in-process library, not a database: no persistence, no metadata, no concurrency.
- Metadata filtering and per-tenant scoping are correctness concerns, not features; get them wrong and you leak or lose results.
- Plan for upserts, deletes, and reindexing on model changes, and verify ANN recall against a brute-force baseline.