Embeddings & Vector Semantics

An embedding is a fixed-length vector of numbers that represents the meaning of a piece of text — assigned by a model trained so that texts with similar meanings end up with similar vectors, letting you measure semantic similarity as geometric distance.

The one-sentence interview answer: "An embedding maps text to a point in a high-dimensional space — typically 1,000–3,000 dimensions — where meaning is geometry: similar sentences land near each other, unrelated ones land far apart. You measure similarity with cosine of the angle between vectors. Retrieval becomes 'find the nearest neighbours' — which is exactly what RAG (L174) does to find relevant context."

Think of embeddings as a map of meaning, where distance is similarity.

Imagine every possible sentence as a point on a map. "The cat sat on the mat" and "A feline rested on the rug" are nearby — similar meaning, different words. "The stock market rose today" is far away. The embedding model is the cartographer: it has read so much text that it learned where each meaning lives. The map has not two dimensions but thousands — but the rule is the same: close = similar, far = unrelated.

   meaning space (shown in 2D, really ~1536D)

   "cat sat on mat" •        • "dog lay on bed"        (animals, furniture)
                    │
   "feline on rug"  •        • "stock market rose"     (far away — different meaning)
                    │
                    └───────────────• "bonds rallied"
                    (pet/household region)   (finance region)

   cosine similarity("cat sat on mat", "feline on rug")  ≈ 0.9
   cosine similarity("cat sat on mat", "stock market")   ≈ 0.1

The magic — and the trap — is that the map was learned, not hand-built. The model inferred "meaning" from co-occurrence in text, so the geometry reflects linguistic similarity, not factual truth.

   Your documents                          Your query
   "The refund policy says…"               "Can I return a course?"
        │                                        │
        ▼                                        ▼
   ┌────────────────┐                    ┌────────────────┐
   │ CHUNK the docs │                    │ EMBED the query│
   │ (L178: split   │                    │                │
   │  into pieces)  │                    │                │
   └───────┬────────┘                    └───────┬────────┘
           ▼                                    ▼
   ┌───────────────────────────────────────────────┐
   │ EMBEDDING MODEL  (same model for both sides)  │
   │   text → vector[1536]                         │
   └───────┬───────────────────────────────┬───────┘
           ▼                               ▼
   ┌────────────────┐                     │
   │ STORE vectors  │                     │
   │ in a vector DB │◀────────────────────┘
   │ (L182)         │   cosine similarity:
   │                │   find the chunk vectors
   │                │   nearest to the query vector
   └────────────────┘
           │
           ▼
   the nearest chunks → the RAG context (L174, L191)

Two facts make this pipeline the backbone of RAG:

1. The same embedding model embeds documents and queries. The space is shared, so distances are comparable. Mixing models is like comparing two different maps.
2. Retrieval is a nearest-neighbour search, not a keyword search. "Can I return a course?" finds the refund-policy chunk even though it shares no words — that's the semantic win, and it's what L174 builds on.

• An embedding is produced by a model trained on contrastive examples: texts that mean the same thing are pulled together in the space, texts that don't are pushed apart. The result is a vector where direction encodes meaning.
• Dimensions: typically 384 (small) to 3,072 (frontier). More dimensions = more capacity to distinguish meanings, but more storage and slower search. The model family (L148) picks this for you.
• Cosine similarity measures the angle between vectors (ignoring length): cos(θ) = (A·B) / (|A||B|), in [-1, 1]. 1 = same direction (similar), 0 = orthogonal (unrelated), -1 = opposite. It's the standard because it's length-invariant and cheap.
• Normalisation matters. Most pipelines normalise vectors before storing, so cosine similarity becomes a simple dot product — faster to compute and index (L182).
• The trap: cosine similarity measures direction in the learned space, not factual agreement. "The sky is blue" and "The sky is not blue" are near-perfectly similar in direction — embeddings capture topic, not truth. That's why RAG still needs reranking (L190) and groundedness evals (L337).

• RAG retrieval (L174–L197). The whole knowledge-system phase is this pipeline: chunk (L178) → embed → store (L182) → nearest-neighbour query → context (L191).
• Semantic search. "Find documents about X" instead of "containing the word X" — for support, legal, internal docs.
• Deduplication and clustering. Near-duplicate tickets, articles, or code — embeddings group them by meaning.
• Recommendation-ish "similar items". "Show me articles like this one" — nearest neighbours of the item's embedding.
• LLM-as-judge groundwork (L343). Comparing an answer to a reference by embedding distance is a cheap, early evaluation signal — before you spend tokens on a judge model.
• Anomaly detection. Points far from every cluster = unusual documents (L317 touches this for abuse).

The through-line: whenever the question is "what's most related to this?", embeddings are the answer. They are the semantic index the whole knowledge phase is built on.

Say it in four moves:

1. The definition. "An embedding maps text to a point in a high-dimensional space — meaning becomes geometry. Similar meanings, nearby points."
2. The measure. "Similarity is cosine of the angle between vectors — direction, not distance. Close to 1 means the same topic; near 0 means unrelated."
3. The pipeline. "Chunk the documents, embed each chunk, store the vectors in a vector database, then at query time embed the question and fetch the nearest neighbours. That's RAG's retrieval (L174)."
4. The honesty. "Embeddings capture aboutness, not truth. 'The sky is blue' and 'the sky is not blue' are almost identical as vectors — which is why retrieval is followed by reranking and groundedness checks."

• Embedding quality is a retrieval quality, measured by recall@k (L195, L338). The dimension count and model family (L148) matter less than "does it retrieve the right chunk for my domain?" — that's an eval question, not a spec question.
• The same model for indexing and querying is non-negotiable. A document embedded with model A and a query embedded with model B live in different spaces — the similarity is meaningless. Changing embedding models means re-indexing everything (L176, L183).
• Hybrid search (L187) beats pure semantic search on keyword-heavy queries — names, IDs, exact phrases. Semantic vectors miss exact matches; keyword search misses paraphrases. Production retrieval is usually both, fused.
• Embeddings are a cost and storage line you can compute exactly (L150). N chunks × dimensions × 4 bytes is a concrete number; re-embedding on every document update is a real pipeline cost (L176).
• The vector is a lossy compression of the text. Retrieving by vector finds the neighbourhood; it never proves relevance. That's why the retrieval phase ends with reranking (L190) and the synthesis phase begins with the actual chunk text (L191) — never the vector alone.

• Using embeddings as a fact store. The vector says "this chunk is about refunds" — it doesn't say whether the refund policy is true, current, or applicable. Facts live in the text, not the vector.
• Mixing embedding models between index and query. Two different spaces — the similarity score is meaningless. Re-index when you change models.
• Forgetting to chunk (L178). Embedding a whole document produces a mushy average vector that retrieves poorly. Chunks are the retrieval unit.
• Trusting cosine for truth. "The sky is not blue" scores ~1.0 against "the sky is blue". Topic similarity ≠ factual agreement (L337).
• Ignoring normalisation. Unnormalised vectors make "longer text = longer vector" a confounder in similarity. Normalise, or use cosine explicitly.
• Choosing dimensions by fashion. 3,072 dimensions is not "better" than 768 for your task — it's more storage, slower search, and often no retrieval gain (L186).

• Pick one embedding model per corpus and keep it stable — re-indexing is expensive (L176, L183).
• Chunk before embedding (L178): the chunk is the retrieval unit, so the chunking strategy and the embedding model are designed together.
• Normalise vectors at write time so query-time similarity is a fast dot product (L182).
• Use hybrid search for production (L187): semantic vectors + keyword exact-match, fused — the two cover each other's blind spots.
• Evaluate retrieval with recall@k (L195, L338) on your real queries, not the model's marketing.
• Never retrieve by vector alone — pass the chunk text to the LLM (L191), and verify groundedness (L337).

Q: What is an embedding?

A: A fixed-length vector that represents a text's meaning, learned by a model trained so similar meanings produce similar vectors. Similarity is measured as cosine of the angle between vectors. It turns "is this about the same thing?" into geometry.

Q: How do embeddings power RAG?

A: The documents are chunked and embedded into a vector store; at query time the question is embedded with the same model, and the store returns the nearest chunks by cosine similarity. Those chunks become the LLM's context (L174, L191). Retrieval is a nearest-neighbour search over meaning, not a keyword match.

Q: What are the limits of cosine similarity?

A: It measures direction in a learned space — aboutness, not truth. Contradictory statements can be nearly identical as vectors. So cosine is a retrieval signal, never a correctness signal; it needs reranking (L190) and groundedness checks (L337) above it.

Q: How do you choose an embedding model?

A: On retrieval quality for my domain, measured with recall@k (L195, L338) — not on dimension count. I also weigh multilingual support, cost per token (L150), and storage: higher dimensions cost more to store and search without always retrieving better (L186).

• What's the difference between embeddings and tokens (L137)?
• How do you store and index vectors — pgvector, Pinecone, Qdrant (L182–L186)?
• Why is chunking (L178) a retrieval-quality decision, not a storage decision?
• What is hybrid search, and when is it better than pure semantic (L187)?
• How do you evaluate whether your embeddings retrieve the right things (L195)?

The senior read: tokens are what the model reads; embeddings are how you find what to show it. RAG (L174) is exactly the seam between the two — retrieval in vector space, synthesis in token space.

// The embedding pipeline, minimal: embed chunks → embed query → nearest neighbours.
const { OpenAI } = require('openai');
const openai = new OpenAI();

async function embed(text) {
  const res = await openai.embeddings.create({
    model: 'text-embedding-3-small',   // the small, cheap embedding model
    input: text,
  });
  return res.data[0].embedding;        // vector[1536]
}

// 1 · embed the chunks (at write time)
const chunks = [
  'Refunds are available within 30 days.',
  'Digital courses are non-refundable after download.',
  'Contact support for billing issues.',
];
const vectors = await Promise.all(chunks.map(embed));

// 2 · cosine similarity (normalised → dot product)
function cosine(a, b) {
  return a.reduce((sum, v, i) => sum + v * b[i], 0);   // assumes normalised
}

// 3 · embed the query with the SAME model, find the nearest chunk
const query = await embed('Can I get my money back for a course?');
const ranked = vectors
  .map((v, i) => ({ i, score: cosine(query, v) }))
  .sort((a, b) => b.score - a.score);

console.log(ranked[0].i, chunks[ranked[0].i]);   // → 1, "Digital courses are non-refundable…"

What the reader must SEE — the pipeline in code:

  embed(chunks)  → store the vectors        (write time)
  embed(query)   → cosine vs every vector   (query time)
  nearest chunk  → the RAG context          (L174, L191)

  The query never shares a word with the best chunk —
  it matched on MEANING, not keywords.

• Embedding is cheap relative to generation. A few ms per chunk, no streaming, no output tokens — it's the "index" operation of the AI stack.
• Storage is computable. N chunks × D dimensions × 4 bytes (float32). 1M chunks at 1,536 dims ≈ 6 GB — before vector-index overhead (L182, L183).
• Normalisation turns cosine into a dot product — the fastest similarity, and what vector indexes optimise (L182).
• Search speed is the index's job, not the model's. At scale, brute-force cosine over millions of vectors is too slow; that's what HNSW/IVF indexes and dedicated vector DBs solve (L182–L186).
• Re-embedding on change is a pipeline cost (L176): a document edit invalidates its chunk vectors, and hybrid-search (L187) and caching (L171) decisions affect the real per-query cost (L150).

• Embedding = text → vector, meaning becomes geometry (similar = nearby).
• Similarity = cosine of the angle (direction, not distance); normalised → dot product.
• Pipeline: chunk → embed → store → nearest-neighbour query — the backbone of RAG (L174).
• Same model for index and query — different spaces are incomparable.
• Aboutness ≠ truth — "not blue" ≈ "blue" as vectors; retrieval ≠ correctness (L337).
• Embedding quality = recall@k on your domain (L195), not dimension count.

EMBEDDING = meaning as geometry
  text → vector[384..3072]
  similar meanings → nearby points
  distance = cosine of the angle

COSINE
  cos(A,B) = (A·B) / (|A||B|)      ∈ [-1, 1]
  1 = same topic   0 = unrelated   -1 = opposite
  normalised vectors → dot product (fast)

THE PIPELINE (RAG's retrieval, L174)
  chunk (L178) → embed → store (L182)
  query → embed (same model) → nearest neighbours → context (L191)

WHAT IT IS / ISN'T
  ✓ semantic "aboutness" retrieval
  ✗ a fact store, ✗ a truth signal, ✗ keyword replacement

RULES
  one embedding model per corpus (re-index on change, L176)
  hybrid search for exact strings + semantics (L187)
  recall@k on your domain decides quality (L195)
  pass chunk TEXT to the LLM, never the vector alone (L191)

INTERVIEW, 4 MOVES
  1 definition "meaning as coordinates"
  2 measure   "cosine, direction not distance"
  3 pipeline  "chunk → embed → store → nearest neighbours"
  4 honesty   "aboutness, not truth → rerank + verify"

You now hold the second foundation of the AI stack: the token stream the model reads (L135–L146), and the meaning-space you retrieve from (this lesson). Every RAG system (L174–L197), every knowledge product, every "chat with your data" — they're all this pipeline wearing different clothes. Keep the honesty rule close: embeddings find the neighbourhood; the text, the reranker, and the groundedness check decide the truth.

Next: the architect's decision layer — model selection and frontier families (L148), then the numbers that make selection real: token budgeting (L149), cost (L150), and latency (L151).