Vector Databases

A vector database stores embeddings alongside their metadata (L180) and answers "what's near this query?" with approximate nearest-neighbor search — ANN indexes like HNSW and IVF that trade a small, tunable recall loss for orders-of-magnitude speed — the index layer of RAG (L175), chosen as a database decision (L186).

The one-sentence interview answer: "A vector DB is a search index with a storage engine (L182). It stores each chunk's embedding (L181) plus its metadata (L180), and answers nearest-neighbor queries. Exact search is O(N) — fine for a demo, hopeless at scale. Production uses ANN: HNSW (a graph that walks to neighbors — great quality, memory-hungry) and IVF (clusters, then searches the nearest clusters — memory-lean, slightly less precise). Both trade a tunable recall loss for speed (L189). The choice is a database decision (L186): pgvector in Postgres (L183) first, specialists (L184–185) when scale or filters demand it."

Think of the vector DB as a well-organized city directory, not a phone book you read cover to cover. A phone book (exact search) checks every name to find a match — O(N), fine for a small town, hopeless for a city. A city directory (ANN) is organized: neighborhoods (IVF clusters) or a street map of nearby points (HNSW graph). To find "who lives near this address?" you go to the neighborhood, or follow the map — you don't read the whole book. You might miss one person (recall loss), but you find everyone relevant, instantly.

   EXACT (O(N))                    ANN — HNSW                  ANN — IVF
   ┌─────────────────┐             ┌─────────────────┐        ┌─────────────────┐
   │ check every     │             │ a graph: walk   │        │ clusters first: │
   │ vector, keep    │             │ to neighbors,   │        │ find the nearest│
   │ the nearest k   │             │ nearest first   │        │ clusters, then  │
   │ = full scan     │             │ (good recall,   │        │ search inside   │
   │ (demo only)     │             │ memory-hungry)  │        │ (lean, fast)    │
   └─────────────────┘             └─────────────────┘        └─────────────────┘

The mental model is directory, not phone book: ANN organizes the space so "nearby" is found fast, with a tunable recall/speed trade (L189).

   a query embedding arrives (L181)
        │
        ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 1 · FILTER (L180)                                        │
   │     metadata filters applied first where possible:       │
   │     tenant (L320) · source · date — scoped search        │
   └──────────────────┬───────────────────────────────────────┘
                      ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 2 · ANN SEARCH (L182)                                    │
   │     HNSW: walk the graph from entry points               │
   │     IVF: pick nearest clusters, scan their vectors       │
   │     → candidate set (more than k, for recall)            │
   └──────────────────┬───────────────────────────────────────┘
                      ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 3 · SCORE + RANK (L189)                                  │
   │     distance/similarity computed on candidates           │
   │     → top-k, optionally reranked (L190)                  │
   └──────────────────┬───────────────────────────────────────┘
                      ▼
   top-k chunks with their metadata (L180) → context (L191)

The flow is the query path: filter → ANN → score/rank — the vector DB's three jobs, and the tunables (recall vs speed, L182) live in the middle.

• What it stores. Every chunk's embedding (L181) + its metadata (L180) — text is referenceable but not the search target. The store is a database: inserts, updates, deletes, and the ANN index maintained with them.
• Exact vs ANN. Exact nearest neighbor is O(N) — every query scans everything. ANN (approximate) organizes the space so queries visit a fraction: it's approximate — it may miss some true neighbors — but recall is tunable (L189), and the speed win is orders of magnitude (L151).
• HNSW (Hierarchical Navigable Small World). A multi-layer graph; queries start at the top, walk down to nearer neighbors. Excellent recall, fast, but memory-hungry — the index lives in RAM. The default for quality-first retrieval.
• IVF (Inverted File). Vectors are clustered (k-means); a query finds the nearest clusters and scans only their members. Memory-lean (fits disk), slightly less precise, fast. Good for large corpora with memory limits.
• Filters (L180). The store applies metadata filters — tenant (L320), source, date — to the search. How it does this (pre-filter vs post-filter, L189) is a quality and performance decision.

• RAG at scale (L175). Millions of chunks, sub-100ms search (L151) — ANN is what makes the shop fast while the index grows.
• Multi-tenant platforms (L320). The vector store filters by tenant (L180) — the filter's efficiency is a platform requirement, and part of the L186 decision.
• Semantic search products. "Find docs like this" — ANN search over a growing corpus, with metadata filters for recency (L140) and source.
• Deduplication and clustering. Near-duplicate detection over embeddings — the same store, different query shape.
• Hybrid search (L187). The vector store plus a keyword index (BM25) — the store is half of the hybrid.

The through-line: the vector DB is where the representation becomes a service — the L175 index, organized for fast, filtered, tunable retrieval.

Say it in four moves:

1. The definition. "A vector DB stores embeddings plus metadata (L180) and answers nearest-neighbor queries with ANN — approximate, tunable, fast (L182)."
2. The indexes. "HNSW — a graph, great recall, memory-hungry. IVF — clusters, memory-lean, slightly less precise. Exact O(N) is the demo."
3. The dials. "efSearch and nprobe trade recall for speed (L189) — set for the latency budget (L151), measured on the golden set (L195)."
4. The decision. "It's a database decision (L186): pgvector in Postgres (L183) first; specialists (L184–185) when scale, filters, or ops demand."

• ANN is the production dial (L182). The senior answer explains the recall/speed trade with the dials — efSearch, nprobe — and the measurement (L195). The demo answer says "it searches vectors fast".
• Filters are a first-class query feature (L180). Tenant isolation (L320) and scoped search (L189) depend on efficient metadata filtering — pre-filter vs post-filter (L189) is a quality/performance decision that belongs in the L186 store choice.
• The store is a database, with database concerns (L182). Upserts, consistency, backup, and the index rebuild on model change (L181, L341) — the vector store has the same operational surface as any database.
• The index choice is a memory/speed/recall triangle (L182). HNSW buys quality with RAM; IVF buys memory with a little recall (L189); the choice is bounded by the corpus size and the latency budget (L151).
• The store choice composes with everything (L186). Embedding dimensionality (L181), filter load (L180), corpus size, and ops maturity all feed the decision — L186 is the synthesis, and this lesson is its vocabulary.

• Linear scan "vector search". O(N) over a list (L182) — the demo that dies at scale.
• No ANN. A library that computes exact distances on every query (L189) — correct but unusable (L151).
• HNSW for everything. Memory cost on a huge corpus (L182) — IVF is the lean answer.
• No filters. The store can't scope by tenant (L320) — the L186 choice must include filter efficiency.
• Default dials, forever. efSearch/nprobe never tuned (L195) — recall or speed left on the table.
• The index as a black box. No rebuild story when the embedding model changes (L181, L341) — a database without a migration plan.

• Start exact, move to ANN with the dials (L182) — correctness first, then the speed dial, measured (L195).
• Choose the index by memory and recall needs (L182) — HNSW for quality, IVF for lean scale.
• Tune the dials to the latency budget (L151) — efSearch/nprobe set for TTFT, measured on the golden set (L195).
• Check filter support before choosing the store (L180, L186) — tenant filters (L320) are non-negotiable.
• Plan the rebuild (L341) — embedding model changes (L181) are index migrations.
• Re-evaluate on growth (L186) — the store that fits 100k chunks may not fit 10M (L182).

Q: What is a vector database?

A: A store for embeddings and their metadata (L180) with approximate nearest-neighbor search (L182). Exact nearest neighbor is O(N) — fine for a demo, hopeless at scale. ANN — HNSW or IVF — organizes the space so queries visit a fraction of it, trading a tunable recall loss for orders-of-magnitude speed (L189). It's the index layer of RAG (L175), chosen as a database decision (L186).

Q: How does ANN work?

A: Two main index families (L182). HNSW is a multi-layer graph — a query starts at the top and walks to nearer neighbors; excellent recall, memory-hungry. IVF clusters the vectors — a query finds the nearest clusters and scans only them; memory-lean, slightly less precise. Both have dials — efSearch, nprobe — that trade recall for speed, set for the latency budget (L151) and measured on the golden set (L195).

Q: Why not exact search?

A: Exact nearest neighbor scans every vector — O(N) per query (L182). At 1M chunks that's a full read on every question — the TTFT budget (L151) dies. ANN's "approximate" is the feature: it organizes the space so the query visits a fraction, with recall loss that's small, tunable, and measurable (L195). Production retrieval is approximate by design (L189).

Q: How do filters work in a vector DB?

A: Metadata filters (L180) scope the search — tenant (L320), source, date. The store either pre-filters (applies the filter, then searches — precise but can miss neighbors near the boundary) or post-filters (searches, then filters — can return too few after the filter). How efficiently a store does this is part of the L186 decision, and critical for multi-tenant isolation (L320).

• HNSW vs IVF — when is each right (L182)?
• How do you tune the ANN recall/speed dials (L195)?
• What does a model-change rebuild look like (L341)?
• How do pre-filter and post-filter differ (L189)?
• How does the vector store scale (L186)?

The senior read: the dials row is the production answer — ANN is a tunable trade, not a compromise.

// The vector store: index, ANN search with filters (L182, L180, L189).
import { index, search } from './vector-store';   // pgvector, Qdrant, Pinecone…

// INGEST — store the vector + metadata (L180).
await index.upsert({
  id: chunk.id,
  vector: chunk.embedding,                        // from L181
  metadata: {
    source: chunk.source, tenant: chunk.tenant,
    date: chunk.date, section: chunk.section,     // L180
  },
});

// QUERY — ANN with filters, dials tuned for the budget (L151, L195).
const results = await search({
  vector: queryEmbedding,                         // same model (L181)
  filter: { tenant: session.tenant },             // L320 — by construction
  topK: 5,                                        // L189
  // THE ANN DIALS — recall vs speed (L182):
  hnswEfSearch: 128,        // higher = better recall, slower
  // or ivfNprobe: 16       // for IVF stores
});

// Each result carries its metadata — the citation source (L192).
for (const r of results) {
  console.log(r.metadata.source, r.score);        // per §7, contracts/…
}

What the reader must SEE — the three jobs in code:

  upsert(vector, metadata)   → store (L180, L182)
  filter: { tenant }         → scoped by construction (L320)
  hnswEfSearch / ivfNprobe   → the ANN dials (L182, L195)
  r.metadata.source          → the citation backbone (L192)

  Store, filter, dial — and metadata comes back for citations.

• ANN is the latency lever (L151). The index keeps search sub-100ms at scale — the TTFT budget (L145) depends on the ANN dials (L182).
• Memory is the HNSW tax (L182). Graph indexes live in RAM — corpus size × dims (L181) bounds the choice; IVF trades memory for a little recall (L189).
• Filters cost performance (L180). Pre-filtering can miss boundary neighbors (L189); post-filtering can return too few — the store's filter implementation is part of the L186 decision.
• Rebuilds are migrations (L341). Embedding model changes (L181) and index config changes are rebuild projects — scheduled, not ad hoc.

• Vector DB = store + ANN search: embeddings + metadata (L180).
• Exact = O(N) — the demo; ANN = the production dial (L182).
• HNSW = graph, recall, RAM. IVF = clusters, lean, slightly less recall.
• The dials: efSearch, nprobe — recall vs speed, measured (L195).
• Filters scope the search (L180) — tenant by construction (L320).
• The choice is a database decision (L186) — pgvector (L183) first.

VECTOR DATABASES = ANN search over coordinates

THE STORE (L182)
  embeddings (L181) + metadata (L180) — upserts, filters, search

EXACT vs ANN (L189)
  exact   O(N) scan — the demo that dies at scale
  ANN     organize the space — visit a fraction — tunable recall

THE INDEXES (L182)
  HNSW    graph walk — excellent recall — memory-hungry (RAM)
  IVF     cluster + scan — lean (disk) — slightly less precise

THE DIALS (L182, L195)
  efSearch (HNSW) · nprobe (IVF) — recall ↔ speed
  set for the latency budget (L151), measured on the golden set

THE FILTERS (L180, L320)
  tenant · source · date — scope by construction
  pre-filter vs post-filter (L189) — part of the L186 decision

THE DECISION (L186)
  pgvector in Postgres (L183) first — specialists when scale demands

INTERVIEW, 4 MOVES
  1 define  "embeddings + metadata + ANN (L182)"
  2 indexes "HNSW vs IVF — the memory/recall trade"
  3 dials   "efSearch/nprobe — measured, not defaulted (L195)"
  4 decide  "a database decision (L186)"

You now hold the store that makes retrieval fast: embeddings plus metadata, organized by HNSW or IVF, scoped by filters, and dialed for the latency budget. The vector DB is where the L181 space becomes a service — and the L186 decision is which service.

Next: the boring default — PostgreSQL + pgvector (L183), vectors in the database you already have.