Hybrid Search

Hybrid search fuses two retrievers — keyword search (BM25) for exact-token precision (codes, names, IDs) and semantic search (embeddings, L181) for meaning-based recall — into one ranking via score fusion (RRF), so retrieval catches both what the words say and what the query means (L187), measured against each retriever alone on the golden set (L195).

The one-sentence interview answer: "Hybrid search combines the two retrievers' strengths (L187). Keyword — BM25 — catches exact tokens: part numbers, error codes, product names, anything an embedding blurs (L188). Semantic — embeddings — catches meaning: synonyms, paraphrase, concepts (L181). I run both and fuse the rankings — reciprocal rank fusion (RRF) is the standard — so a result that ranks well in either channel ranks well overall. The senior move is measuring it: the golden set (L195) shows hybrid beating vector-only on precision and recall for exact-match-heavy content — which is most real corpora."

Think of the two retrievers as two librarians with different strengths. The keyword librarian (BM25) is a stickler for exact words: ask for "model 7A-220" and it goes straight to the shelf labeled 7A-220 — it notices exact labels. The semantic librarian (embeddings, L181) is a conceptual thinker: ask "what's the return policy for damaged goods?" and it finds the policy page even if it never uses those words. Each alone misses things the other catches. Hybrid search asks both, then merges their shortlists — a document on both lists ranks higher than one on either.

   KEYWORD (BM25, L188)          SEMANTIC (L181)
   ┌────────────────────┐        ┌────────────────────┐
   │ exact tokens       │        │ meaning, synonyms  │
   │ "7A-220" → shelf   │        │ "damaged goods" →  │
   │ precision on codes │        │ the policy page    │
   │ names, IDs         │        │ recall on concepts │
   └─────────┬──────────┘        └─────────┬──────────┘
             │         FUSE (RRF, L187)    │
             └────────────┬───────────────┘
                          ▼
                  one ranking — both strengths

The mental model is two librarians, one merged shortlist: each finds what the other can't, and the fusion (RRF) ranks a document higher when both agree.

   a query arrives (L189)
        │
        ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 1 · TWO RETRIEVERS, ONE QUERY                            │
   │     keyword: BM25 over the text index (L188)             │
   │     semantic: embedding → ANN over the vectors (L181-182)│
   └──────────────────┬───────────────────────────────────────┘
                      ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 2 · TWO SHORTLISTS                                       │
   │     BM25 top-50 (ranked by term frequency)               │
   │     ANN top-50 (ranked by similarity)                    │
   └──────────────────┬───────────────────────────────────────┘
                      ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 3 · FUSE (RRF) (L187)                                    │
   │     score = Σ 1/(k + rank) across both lists             │
   │     a doc on both lists ranks above either alone         │
   └──────────────────┬───────────────────────────────────────┘
                      ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 4 · top-k → rerank (L190) → context (L191)               │
   └──────────────────────────────────────────────────────────┘

The flow is the hybrid: two retrievers → two shortlists → RRF fusion → one ranking — and the quality gate (L195) measures whether it beats either alone.

• Why vector-only misses (L181, L187). Embeddings capture meaning, not exact tokens. A part number, an error code, a product name, an ID — these have no semantic neighborhood; the embedding blurs "7A-220" into a general "product" region. Exact-match content is the vector blind spot.
• BM25 (L188). The keyword baseline: term frequency × inverse document frequency — exact tokens, weighted by rarity. It catches the codes, names, and IDs the embedding loses, and it's cheap, well-understood, and available in Postgres (tsvector, L183) and every specialist (L184–185).
• The fusion — RRF (L187). Reciprocal rank fusion: each retriever returns a ranked list; a document's fused score is Σ 1/(k + rank) across the lists. No score calibration needed — just ranks — and a document ranked well in both channels outranks one ranked well in one. Robust and simple.
• When it wins (L195). Exact-match-heavy content — product catalogs, code, legal terms, IDs — hybrid beats vector-only measurably. For pure conceptual prose with no exact tokens, the semantic retriever alone may be enough — the golden set decides (L195).

• Product catalogs (L189). "Model 7A-220 specs" — the code is the query's spine; BM25 finds it, semantic finds the spec's context (L187).
• Codebase Q&A. "How do I use parseChunk?" — the function name is exact; the semantic channel finds the doc, the keyword channel pins the name (L188).
• Support knowledge bases. Error codes ("ERR_429"), product names, and policy concepts in one corpus — the hybrid covers both query shapes (L189).
• Legal research. Exact clause references ("§7") plus conceptual searches ("early termination rights") — both channels earn their place (L195).
• E-commerce search. "Waterproof hiking boots" — semantic for the concept, keyword for the exact brand/model terms (L187).

The through-line: real corpora are mixed — exact tokens and meaning in the same documents — and the hybrid is the retriever that covers both.

Say it in four moves:

1. The blind spot. "Vector-only misses exact tokens — codes, names, IDs have no semantic neighborhood (L187)."
2. The two retrievers. "BM25 catches the exact terms (L188); embeddings (L181) catch the meaning. I run both."
3. The fusion. "RRF — reciprocal rank fusion: Σ 1/(k + rank) across both lists — a doc on both ranks above either alone (L187)."
4. The measurement. "The golden set (L195) shows hybrid beating vector-only on exact-match-heavy content — and tells me when it doesn't."

• The blind spot is the design insight (L187). The senior answer names why vector-only fails — exact tokens have no semantic neighborhood — rather than "hybrid is better". The mechanism is the argument.
• BM25 is the complement, not a rival (L188). Keyword precision and semantic recall are orthogonal strengths — the fusion (RRF) is how they compose, with no score calibration between incompatible scales (L187).
• RRF's simplicity is its robustness (L187). Rank-based fusion needs no score tuning — it's the senior default because it works across retriever families and scales (L189).
• The hybrid is measured, not assumed (L195). Golden set precision/recall, hybrid vs each alone — the second retriever must prove it earns its latency (L151) and index cost (L150). The L195 gate applies to retrieval upgrades like any change (L341).
• The hybrid composes with the stack (L183–185). Postgres does BM25 + pgvector in one query (L183); Qdrant and Pinecone offer sparse-dense natively (L184–185) — the L186 store choice includes hybrid support.

• Vector-only retrieval (L187). The blind spot — exact tokens silently missed, and no one measures it (L195).
• Keyword-only retrieval (L188). The synonym/paraphrase miss — meaning-blind search (L181).
• Score averaging instead of RRF (L187). Averaging incompatible score scales — the semantic channel's magnitude swamps the keyword channel. RRF fuses ranks.
• Hybrid everywhere without measurement (L195). The second retriever's latency (L151) and index cost (L150) without the golden set proving the win.
• No keyword index at all (L188). The BM25 side missing — the store choice (L186) should include it (tsvector, L183; sparse vectors, L184–185).
• Fusion before filtering (L180). Fusing unfiltered lists — tenant isolation (L320) applies before ranking, not after.

• Add the keyword channel first (L188) — BM25 beside the vectors (L182), before any reranker (L190).
• Fuse with RRF (L187) — ranks, not scores; no calibration.
• Measure on the golden set (L195) — hybrid vs each alone, precision and recall.
• Filter before fusing (L180, L320) — tenant and date scope each retriever first.
• Check store support (L186) — tsvector + pgvector (L183), or sparse-dense (L184–185).
• Rerank after fusing (L190) — the hybrid's top-k is the reranker's input.

Q: What is hybrid search?

A: Two retrievers, one fused ranking (L187). Keyword — BM25 — catches exact tokens: codes, names, IDs (L188). Semantic — embeddings — catches meaning: synonyms, concepts (L181). I run both, fuse with RRF (Σ 1/(k + rank) across the lists), and rank a document higher when both channels agree. The golden set (L195) measures whether it beats either alone — for exact-match-heavy corpora, it does.

Q: Why doesn't vector search alone work?

A: Because embeddings capture meaning, not exact tokens (L187). A part number, an error code, a product name — these have no semantic neighborhood; "7A-220" embeds into a generic "product" region, and the exact match is lost. BM25 finds the exact token; the embedding finds the context. Real corpora have both — so real retrieval needs both.

Q: How do you fuse the two rankings?

A: Reciprocal rank fusion (RRF) (L187). Each retriever returns a ranked list; a document's score is Σ 1/(k + rank) across the lists. It needs no score calibration — the semantic channel's similarity and the keyword channel's BM25 score have incompatible scales, but ranks compose. A document ranked well in both channels outranks one ranked well in one — which is exactly the hybrid's value.

Q: When is hybrid not worth it?

A: When the golden set says so (L195). Pure conceptual prose with no exact tokens — a corpus of essays, say — may be served by the semantic retriever alone, and the keyword channel's latency (L151) and index cost (L150) don't pay for themselves. The rule: measure hybrid vs each alone; the second retriever earns its place only when the numbers show it.

• How does RRF work under the hood (L187)?
• What exactly does BM25 catch that vectors miss (L188)?
• How does the golden set measure the hybrid (L195)?
• How do the stores support the keyword channel (L183–185)?
• How does filtering compose with the hybrid (L180)?

The senior read: the two single channels are each half a retriever — the hybrid is the sum, fused by RRF (L187).

// Hybrid search: BM25 + embeddings, fused by RRF (L187-188).
import { bm25Search } from './keyword';       // L188 — the keyword channel
import { vectorSearch } from './vector-store'; // L181-182 — the semantic channel

// RRF — reciprocal rank fusion over ranks, not scores (L187).
function rrf(...lists, k = 60) {
  const scores = new Map();
  for (const list of lists) {
    list.forEach((doc, i) => {
      scores.set(doc.id, (scores.get(doc.id) ?? 0) + 1 / (k + i + 1));
    });
  }
  return [...scores.entries()].sort((a, b) => b[1] - a[1]).map(([id]) => id);
}

// THE HYBRID — both channels, scoped, fused (L180, L320, L189).
async function hybridSearch(query, { tenant, topK = 5 }) {
  // Scope each retriever BEFORE fusing (L320, L140).
  const keyword = await bm25Search(query, { filter: { tenant }, limit: 50 });   // L188
  const semantic = await vectorSearch(query, { filter: { tenant }, topK: 50 }); // L181

  // Fuse the ranked shortlists (L187).
  const fused = rrf(keyword, semantic).slice(0, topK);

  // The fused top-k is the reranker's input (L190) → context (L191).
  return fused;
}

What the reader must SEE — the three parts of the hybrid:

  bm25Search + vectorSearch  → both channels, scoped (L188, L181, L320)
  rrf(keyword, semantic)     → rank fusion, no score calibration (L187)
  filter: { tenant }         → isolation before fusing (L320)

  Keyword precision + semantic recall, one ranking.

• Two retrievers, two latencies (L151). The hybrid runs BM25 and ANN — parallelize them (L222) and fuse; the added latency is one max, not a sum, if scheduled well.
• The keyword index costs storage (L150). BM25's inverted index (L188) and the sparse vectors (L184–185) add index size — the L150 cost of the second channel.
• RRF is O(n log n) (L151). Fusing two shortlists of 50 is trivial — the fuse is never the bottleneck; the retrievers are.
• The cache still applies (L171). Repeats skip both retrievers — the response cache (L171) is the hybrid's best friend.

• Hybrid = keyword (BM25) + semantic (embeddings), fused (L187).
• The blind spot: exact tokens have no semantic neighborhood (L188).
• The fusion: RRF — Σ 1/(k + rank) — ranks, not scores (L187).
• Scope before fusing (L180, L320); rerank after (L190).
• Measure it (L195) — hybrid vs each alone; the second channel earns its place.
• Store support: tsvector + pgvector (L183), sparse-dense (L184–185).

HYBRID SEARCH = keyword precision + semantic recall, one ranking

THE BLIND SPOT (L187)
  embeddings capture meaning, not exact tokens (L181)
  codes, names, IDs have no semantic neighborhood

THE TWO CHANNELS (L188, L181)
  BM25        exact tokens — precision on terms (L188)
  embeddings  meaning, synonyms — recall on concepts (L181)

THE FUSION (L187)
  RRF: score = Σ 1/(k + rank) across the lists
  ranks, not scores — no calibration
  a doc on both lists ranks above either alone

THE DISCIPLINE
  scope each retriever BEFORE fusing (L320, L180)
  fuse, then rerank (L190) → context (L191)
  measure on the golden set (L195) — hybrid vs each alone
  the second channel earns its place only if the numbers say so

THE STORES (L186)
  Postgres: tsvector + pgvector, one query (L183)
  Qdrant / Pinecone: sparse-dense natively (L184-185)

INTERVIEW, 4 MOVES
  1 blind spot "exact tokens have no semantic neighborhood"
  2 channels   "BM25 precision + embedding recall (L188, L181)"
  3 fusion     "RRF — ranks, not scores (L187)"
  4 measure    "the golden set decides (L195)"

You now hold the upgrade that fixes vector-only's blind spot: BM25 for the exact tokens, embeddings for the meaning, RRF to fuse them, and the golden set to prove it. It's the highest-leverage retrieval change in most RAG stacks — and the numbers, not fashion, decide when it pays.

Next: the two channels, understood deeply — keyword vs semantic search (L188), what each finds that the other misses.