Contextual Retrieval

Contextual retrieval fixes the bare-chunk failure — a chunk embedded alone is a context-free fragment ("the rate is 4.2%" means nothing alone) — by giving each chunk its context: surrounding-context chunks store a window around the fragment, and contextual embeddings embed the chunk with a model-generated context sentence (L194) — both lifting retrieval on hard queries, at embedding cost (L150), measured on the golden set (L195).

The one-sentence interview answer: "Contextual retrieval fixes chunks that mean nothing alone (L194). The failure: chunking (L178) cuts 'the rate is 4.2%' from its section — embedded bare, it's generic finance text, and hard queries miss it (L196). Two fixes. Surrounding context: store a window — the section heading and neighbors — with each chunk, so retrieval and the context carry the meaning (L194). Contextual embeddings: at ingestion, the model writes one context sentence per chunk — 'this rate appears in the 2024 loan terms, section on interest' — and I embed chunk + sentence together (L194). Both cost embedding/model calls (L150) and lift recall on hard queries — measured on the golden set (L195)."

Think of a chunk as a single sentence of a conversation you walked into. "The rate is 4.2%." — you have no idea what rate, whose, or why it matters. You need the conversation's context: who's speaking, the topic, the previous lines. Surrounding-context retrieval hands you the previous lines (the window around the chunk, L194). Contextual embeddings hand you a summary of the conversation so far (the generated context sentence, L194). Either way, the fragment becomes meaningful — and retrievable.

   the bare chunk (L196)              the context fixes (L194)
   ┌──────────────────────┐           ┌──────────────────────────────┐
   │ "the rate is 4.2%"   │           │ surrounding: the section +   │
   │ means nothing alone  │  ────►    │   neighbors, stored with it  │
   │ embedded bare →      │           │ contextual: "…in the 2024    │
   │ generic finance text │           │   loan terms, interest §" +  │
   └──────────────────────┘           │   the chunk, embedded        │
                                      └──────────────────────────────┘

The mental model is the fragment, given its conversation: the window or the summary restores what chunking (L178) cut away.

   ingestion (L176)
        │
        ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 1 · CHUNK + ITS CONTEXT (L178, L194)                     │
   │     each chunk is stored WITH a context:                 │
   │     surrounding window (section + neighbors) OR          │
   │     a generated context sentence (L194)                  │
   └──────────────────┬───────────────────────────────────────┘
                      ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 2 · EMBED (L181, L194)                                   │
   │     bare: embed the chunk alone (L196 — the failure)     │
   │     contextual: embed chunk + context together (L194)    │
   └──────────────────┬───────────────────────────────────────┘
                      ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 3 · RETRIEVE (L189)                                      │
   │     the query finds the MEANINGFUL chunk                 │
   │     (hard queries now match — L195)                      │
   └──────────────────┬───────────────────────────────────────┘
                      ▼
   → rerank (L190) → context with the window (L191, L192)

The flow is the fix: chunk with context → embed with context → retrieve by meaning — the context that was cut by chunking (L178) is restored before embedding.

• The failure (L196). Chunking (L178) cuts fragments — a sentence, a figure, a spec — from their section. Embedded bare, the fragment is context-free: "the rate is 4.2%" is generic finance text in the vector space (L181). Hard queries — "what's the interest rate on the 2024 loan terms?" — fail to match (L189), and the chunk is a retrieval miss (L196).
• Surrounding-context chunks (L194). Store each chunk with a window: the section heading and neighbors, as context. Two benefits: retrieval matches the richer text (L189), and the retrieved chunk carries its context into the prompt (L191) — the model reads the fragment with its section.
• Contextual embeddings (L194). At ingestion, the model generates a context sentence per chunk — "this 4.2% rate appears in the 2024 loan terms, section on interest rates" — and the chunk is embedded with that sentence (L181). The embedding now lives near "interest rate", "loan terms", "2024" — the query's vocabulary (L188).
• The choice (L150). Surrounding context is nearly free (a storage choice, L180); contextual embeddings cost a model call per chunk at ingestion (L150) but embed the meaning. The corpus and the golden set (L195) decide.

• Financial docs. "The rate is 4.2%" — contextual embedding places it in the loan terms (L194); the surrounding window renders the section (L191).
• Product manuals. A torque spec cut from its table — the window restores "Table 4: wheel torque specs" (L194).
• Legal contracts. A clause fragment — the section context restores "§7 Termination" (L180, L192).
• Research papers. A methods sentence — the window restores "the study used…" (L194).
• Any dense, technical corpus (L195). Where fragments dominate and hard queries miss — contextual retrieval is the recall fix (L196).

The through-line: chunking (L178) cuts context; contextual retrieval restores it — on the retrieval side (L189), the generation side (L191), or both (L194).

Say it in four moves:

1. The failure. "A bare chunk means nothing — 'the rate is 4.2%' is generic text alone (L196). Chunking cut its context (L178)."
2. The first fix. "Surrounding context — store a window with each chunk, so retrieval matches it and the context renders (L194)."
3. The second fix. "Contextual embeddings — a generated context sentence embedded with the chunk, so the vector lives near the query's vocabulary (L194, L181)."
4. The measure. "Both cost embedding/model calls (L150) — the golden set (L195) shows which fix pays for which corpus."

• The failure is a chunking artifact (L178, L196). The senior answer traces the miss to chunking (L178): fragments lose their section, embeddings lose their meaning. Contextual retrieval is the repair — the senior design plans for it when choosing the chunking strategy (L179).
• The two fixes hit different stages (L194). Surrounding context improves retrieval and generation (L191); contextual embeddings improve retrieval only (L181). The senior answer separates the stages and applies each where it pays (L195).
• Contextual embeddings are an ingestion-time model cost (L150). A context sentence per chunk — the corpus-wide model spend (L150) is the trade for recall (L195), amortized at ingestion (L176), cacheable (L171).
• The context sentence is a metadata cousin (L180). Storing it per chunk (L194) is a metadata field (L180) — versioned with the chunking config (L341), re-generated on re-ingestion (L176).
• The gain is measured, like any retrieval change (L195). Golden-set recall on hard queries, before/after each fix (L341) — contextual retrieval is an empirical upgrade, not a fashion.

• Bare chunks forever (L196). The context-free fragment embedded alone — the hard-query miss that contextual retrieval exists to fix (L194).
• Context only in the prompt (L191). The retrieval misses because the embedding was bare — the window helps generation (L191) but not the match (L189).
• Contextual embeddings without verification (L196). A hallucinated context sentence (L141) embeds the chunk in the wrong neighborhood — verify the generated context (L195).
• Context stored, never rendered (L191). The window in the index but not in the prompt — generation still reads a bare fragment (L192).
• Paying for both blindly (L150). Both fixes everywhere — the golden set (L195) should show each one's contribution per corpus.
• Re-ingesting without regenerating (L341). The context sentence is a field — re-ingestion (L176) and chunking changes (L179) must regenerate it (L341).

• Store a surrounding window with each chunk (L194) — the section and neighbors, as context (L180).
• Generate a context sentence per chunk at ingestion (L194) — embedded with the chunk (L181), at model cost (L150).
• Render the window in the context (L191) — generation reads the fragment with its section (L192).
• Verify the generated context (L195) — a wrong context sentence embeds the chunk wrongly (L196).
• Measure each fix separately (L195) — the golden set shows what pays (L341).
• Regenerate on re-ingestion (L176, L341) — the context is a field, versioned with the config.

Q: What is contextual retrieval?

A: The fix for bare chunks (L194). Chunking (L178) cuts fragments from their context — "the rate is 4.2%" means nothing embedded alone (L196). Two fixes: surrounding context — store a window (section + neighbors) with each chunk, so retrieval matches it and generation reads it; and contextual embeddings — a generated context sentence embedded with the chunk, so its vector lives near the query's vocabulary (L181). Both lift hard-query recall (L195).

Q: Why does a bare chunk retrieve poorly?

A: Because the embedding is context-free (L181). "The rate is 4.2%" is generic finance text in the vector space — the hard query "what's the interest rate on the 2024 loan terms?" shares almost no vocabulary with it (L188). The chunk's meaning lived in its section, which chunking (L178) cut away. Contextual retrieval restores that meaning — in the embedding (L194) or the window (L194).

Q: Surrounding context vs contextual embeddings?

A: They hit different stages (L194). Surrounding context — the window — improves retrieval and generation: the chunk matches richer text (L189) and renders with its section (L191). Contextual embeddings improve retrieval only: the context sentence shapes the vector (L181), but doesn't need to be shown. Surrounding is nearly free (storage, L180); contextual embeddings cost a model call per chunk (L150). Many systems do both — and the golden set (L195) measures each.

Q: What's the cost of contextual embeddings?

A: A model call per chunk at ingestion (L150) — the corpus-wide cost is the trade for recall (L195). It's an ingestion-time cost (L176), amortized and cacheable (L171), not a query-path cost (L151). The alternative — surrounding context only — is nearly free but doesn't fix the match (L189), only the rendering (L191). The golden set decides which pays for your corpus (L341).

• How does the window affect the context budget (L191)?
• How do you verify the generated context sentence (L195)?
• How does this compose with chunking strategies (L179)?
• What's the ingestion cost at scale (L150)?
• How do you version the context field (L341)?

The senior read: the two fixes are complementary — one for the match (L189), one for the rendering (L191); both measured (L195).

// Contextual retrieval: chunk with context, embed with context (L194).
// 1 · SURROUNDING CONTEXT — store a window with each chunk (L194).
function chunkWithWindow(doc, chunk, i) {
  return {
    text: chunk.text,
    window: `${doc.sectionHeading}\n\n${doc.chunks[i - 1]?.text ?? ''}\n${chunk.text}\n${doc.chunks[i + 1]?.text ?? ''}`,
    metadata: { source: doc.path, section: doc.sectionHeading },  // L180
  };
}

// 2 · CONTEXTUAL EMBEDDING — a context sentence, embedded with the chunk (L194).
async function contextualEmbed(chunk, doc) {
  const context = await generateContextSentence(chunk.text, doc);   // L150 — model call
  //   → "this 4.2% rate appears in the 2024 loan terms, §interest"
  return {
    vector: await embed(`${context}\n\n${chunk.text}`),             // L181 — context in
    context,                                                        // stored (L180)
  };
}

// 3 · RENDER — the window goes in the context for generation (L191, L192).
const prompt = retrieved.map((r) => `[${r.metadata.source} §${r.metadata.section}]\n${r.window}`).join('\n\n');

What the reader must SEE — the context restored on both sides:

  chunkWithWindow()          → the window stored with the chunk (L194)
  contextualEmbed()          → the sentence shapes the vector (L181)
  generateContextSentence()  → the model cost, at ingestion (L150)
  prompt uses r.window       → generation reads the context (L191)

  The context chunking cut away is restored before embed and render.

• The window is storage and tokens (L150). Storing a window per chunk grows the index (L182) and the context (L191) — the window size is a budget decision (L149).
• Contextual embedding is an ingestion-time cost (L150). One model call per chunk (L176) — parallelizable (L222), cacheable (L171), and amortized over every query the chunk serves.
• The match improvement is on the recall side (L189). Better embeddings (L181) mean better ANN matches (L182) — no query-path latency added (L151).
• The golden set measures the gain (L195). Hard-query recall before/after — the only honest way to know if the fix pays (L341).

• The failure: a bare chunk means nothing alone (L196) — chunking cut its context (L178).
• Fix 1: surrounding context — a window stored with the chunk (L194).
• Fix 2: contextual embeddings — a context sentence embedded with the chunk (L194).
• Fix 1 helps retrieval + generation (L189, L191); fix 2 helps retrieval only (L181).
• Cost: window storage + model calls at ingestion (L150).
• Measure: hard-query recall on the golden set (L195, L341).

CONTEXTUAL RETRIEVAL = restore what chunking cut away (L178, L196)

THE FAILURE (L196)
  "the rate is 4.2%" embedded bare = generic finance text
  hard queries share no vocabulary with the fragment (L188)

THE TWO FIXES (L194)
  surrounding context   store a window (section + neighbors)
                        retrieval matches richer text (L189)
                        generation renders the window (L191)
  contextual embeddings  generate a context sentence per chunk
                        embed chunk + sentence (L181)
                        the vector lives near the query's words

THE COMPOSE (L194)
  contextual embeddings for the MATCH (L189)
  surrounding windows for the RENDER (L191)
  both measured on the golden set (L195)

THE COST (L150)
  window = storage + context tokens (L149)
  context sentence = a model call per chunk, at ingestion (L176)
  amortized, cacheable (L171) — not a query-path cost (L151)

INTERVIEW, 4 MOVES
  1 failure  "bare chunks mean nothing (L196)"
  2 fix 1    "the window — context stored and rendered (L194)"
  3 fix 2    "the sentence — context embedded (L181)"
  4 measure  "the golden set proves the gain (L195)"

You now hold the repair for chunking's collateral damage: the window that carries a fragment's section into the match and the render, and the context sentence that embeds its meaning. The chunk no longer means nothing alone — and the hard queries find it.

Next: proving the whole pipeline — RAG evaluation (L195), where retrieval and answers are measured against the golden set.