Embeddings and Representation Learning

An embedding is a versioned scoring interface. It turns text, its role, and an optional task instruction into a fixed-length vector. A retrieval system can then compare one query vector with millions of stored document vectors without running a language model over every pair. The useful promise is narrow: text pairs judged relevant by the training process should receive higher scores than irrelevant pairs. Nearness does not by itself mean that two texts are synonyms, factually correct, authorized for the same user, or useful for every task.

Chapter Chapter 44 treated dense retrieval as one candidate-generation channel. This chapter opens that channel. The central engineering problem is not to discover a universal geometry of meaning. It is to define one ranking contract, learn it from imperfect pairs, preserve it through indexing, and know when a model or index change has broken it.

Start with the scoring contract

An embedding has meaning only together with the transformation that produced it. Two arrays of 768 floating-point numbers are not necessarily comparable. They may come from different tokenizers, pooling rules, task prompts, model checkpoints, or normalization conventions. Record those choices as data:

EmbeddingSpec {
  model_id, model_revision, tokenizer_revision
  query_role, document_role, instruction_version
  pooling, source_layer, max_tokens, truncation_policy
  dimension, normalization, similarity, dtype, quantization
}

EmbeddingRecord {
  tenant, document_id, chunk_id, source_version, content_hash
  embedding_spec_hash, vector, acl_version, index_generation
}

The spec is part of the index schema. A query must use a compatible query-side spec, and every stored vector must use the corresponding document-side spec. For an asymmetric retrieval model, those two roles may add different prefixes or use different encoder weights. Omitting a required query instruction is not a harmless formatting change; it asks the model to apply a different scoring function.

Let $E_q$ denote the query encoder and $E_d$ the document encoder. They may share weights, but they need not. For query text $x$ and document text $y$, define

$$u = E_q(x), \qquad v = E_d(y), \qquad s(x,y) = u^\mathsf{T}v.$$

Here $u,v\in\mathbb{R}^{d}$ are $d$-dimensional vectors, $u^\mathsf{T}v$ is their inner product, and $s(x,y)$ is the score used for ranking. If the contract applies $\ell_2$ normalization,

$$\widehat{u}=\frac{u}{\lVert u\rVert_2}, \qquad \widehat{v}=\frac{v}{\lVert v\rVert_2}, \qquad \widehat{u}^{\mathsf{T}}\widehat{v} = 1-\frac{1}{2}\lVert\widehat{u}-\widehat{v}\rVert_2^2.$$

The hats denote unit-length vectors and $\lVert\cdot\rVert_2$ is Euclidean norm. On unit vectors, cosine similarity, inner product, and squared Euclidean distance induce the same ranking, with distance ordered in the opposite direction. Without normalization, vector length affects the inner product. Normalization must therefore match training and indexing; it is not a generic post-processing improvement. Dense Passage Retrieval, for example, trained and searched with an unnormalized inner product (Karpukhin et al. 2020).

Pooling decides what one vector contains

A transformer produces one hidden state per input token. Retrieval usually needs one vector per query or chunk, so a pooling rule must compress those token states. For masked mean pooling,

$$z(x)= \frac{\sum_{t=1}^{T} a_t h_t} {\sum_{t=1}^{T} a_t}.$$

Here $x$ contains $T$ token positions, $h_t\in\mathbb{R}^{d}$ is the hidden state at position $t$, and $a_t\in\{0,1\}$ excludes padding or any other token the contract says not to pool. A special-token pooler instead chooses one $h_t$; max pooling takes a coordinate-wise maximum; learned attention assigns data-dependent weights. The chosen layer, inclusion of special tokens, padding mask, and normalization after pooling all affect the result.

Sentence-BERT showed why this extra training and pooling interface matters. Its siamese encoder produced sentence vectors that could be compared directly, avoiding pairwise BERT inference for semantic search. Mean pooling was its default and worked well in its reported ablations, but that experiment does not make mean pooling universal (Reimers and Gurevych 2019). Decoder-based embedders may use the last token, bidirectional attention, or a learned pooler. Those are separate designs, not stages in a single historical progression.

Unadapted language-model states are not a retrieval metric by default. The pretraining objective supervises token prediction, not the ordering of pooled query-document scores. Early contextual models also exhibited anisotropy: randomly sampled token representations often had a positive mean cosine rather than spreading evenly around the origin (Ethayarajh 2019). That result concerned contextual token occurrences in particular BERT, ELMo, and GPT-2 checkpoints. It is a geometric warning, not proof that anisotropy causes every pooling failure or that an isotropic space must retrieve well. Later analyses found that correcting anisotropy alone does not necessarily restore semantic isometry (Fuster Baggetto and Fresno 2022). The operational test is still ranking quality on the intended query and corpus distribution.

Contrastive learning shapes the ranking

Contrastive training makes the scoring contract explicit. Suppose a batch has $B$ labeled pairs $(x_i,y_i)$, where $y_i$ is relevant to query $x_i$. Let $u_i=E_q(x_i)$ and $v_j=E_d(y_j)$. A common in-batch loss for query $i$ is

$$\mathcal{L}_i = -\log \frac{\exp\!\left(s(u_i,v_i)/\tau\right)} {\sum_{j=1}^{B}\exp\!\left(s(u_i,v_j)/\tau\right)}.$$

Here $s$ is the contracted similarity function, $v_i$ is the labeled positive, the other $v_j$ are candidate negatives for query $i$, and $\tau>0$ is the temperature. The batch loss is $B^{-1}\sum_{i=1}^{B}\mathcal{L}_i$. The softmax compares the positive with the particular negative set presented during training. It does not compare with every document the model will meet in production.

InfoNCE was introduced in contrastive predictive coding (Oord et al. 2018). Under its specific sampling construction, one candidate comes from the conditional distribution $p(d\mid q)$ and the others are independent draws from the marginal $p(d)$. The expected objective then gives a mutual-information lower bound whose ceiling grows as $\log B$. With an unrestricted scorer, the optimal logit differs by a query-only constant from $\log\!\left[p(d\mid q)/p(d)\right]$. If negatives instead come from a mining proposal $\nu(d)$, that ratio changes to $p(d\mid q)/\nu(d)$; query-dependent hard mining may invalidate the standard mutual-information interpretation entirely. More candidates raise the bound's ceiling, but they do not guarantee a tighter bound or a better retriever (Poole et al. 2019).

The geometric view is often more useful in practice. On normalized vectors, contrastive training rewards alignment of labeled positive pairs and uniformity of the overall representation distribution on the sphere. Wang and Isola proved an asymptotic relationship under a symmetric sampling model and showed these quantities were useful diagnostics on their evaluated vision and language tasks (Wang and Isola 2020). Query-document retrieval can be asymmetric, so this is a diagnostic analogy rather than a theorem about every dual encoder. SimCSE demonstrated the mechanism for sentence similarity: dropout created two views of one sentence in its unsupervised setting, while entailment and contradiction pairs supplied positives and hard negatives in its supervised setting (Gao et al. 2021).

Temperature controls how the model distributes gradient across candidates. A high-scoring negative receives more softmax mass and therefore more pressure. Lowering $\tau$ concentrates relative weight on the highest-scoring negatives, but it also amplifies mislabeled negatives and can destroy useful local neighborhoods. There is no task-independent best temperature. Tune it together with normalization, batch composition, duplicate masking, and the negative miner; a numerical value copied from another model has no stable meaning when the score scale changes.

Interaction placement fixes the cost envelope

The moment at which query and document tokens can interact determines what can be precomputed.

A dual encoder is the usual first-stage dense retriever because the corpus side is independent of the query. A cross-encoder is more expressive because it reads the pair jointly, but that expressiveness does not guarantee the best result on every workload. Its common production role is reranking a bounded candidate set, where the number of pairwise passes is affordable.

ColBERT occupies the middle. It encodes query and document tokens independently and applies late interaction through MaxSim:

$$S_{\mathrm{LI}}(q,d) =\sum_{i=1}^{T_q} \max_{1\le j\le T_d} \widehat{q}_i^\mathsf{T}\widehat{d}_j.$$

Here $T_q$ and $T_d$ are the query and document token counts, and $\widehat{q}_i,\widehat{d}_j\in\mathbb{R}^{d}$ are normalized token vectors. For each query token, MaxSim keeps the best document-token match and then sums those matches. Document vectors remain precomputable, but the index stores many vectors per document. ColBERTv2 reduced the footprint of an uncompressed late-interaction index by six to ten times in its experiments through residual compression and denoised supervision; that comparison was not against a single-vector index (Santhanam et al. 2022).

Training data defines relevance

The loss cannot decide what “related” should mean. The positive-pair policy does that. A search click, a question with an answer-bearing passage, two duplicate records, a translation pair, and two sentences with similar meaning express different relations. Combining them without task labels asks one vector space to satisfy incompatible orderings. Every training row therefore needs the query and document roles, task, languages, source and license, source version, label origin and confidence, deduplication group, split assignment, and any teacher or miner revision.

In-batch negatives reuse the other $B-1$ documents in the matrix above. Dense Passage Retrieval used this efficiently and added BM25-mined candidates in its open-domain QA experiments (Karpukhin et al. 2020). Reuse is not free of semantics: two batch rows may share a relevant document, duplicate one another, or contain different valid answers to the same query. Known positives and duplicate groups must be masked before the loss treats off-diagonal cells as negatives. Larger batches add compute, communication, and false-negative exposure as well as more candidates.

Once random negatives become easy, a frozen checkpoint can retrieve high-scoring unjudged documents from the corpus. ANCE refreshed such an index asynchronously so the miner tracked the model over training (Xiong et al. 2021). Those candidates are unjudged, not known to be irrelevant. RocketQA reported that sparse labels made many top retrieved passages false negatives and used a cross-encoder to filter candidates (Qu et al. 2021). A teacher is still fallible. Preserve known positives, near duplicates, answer-bearing passages, and multi-positive labels; send uncertain high-impact examples to human judgment rather than converting a teacher score directly into truth.

import numpy as np

rng = np.random.default_rng(0)

def unit(vector):
    norm = np.linalg.norm(vector)
    return vector / norm if norm else vector

def infonce(query, positive, negatives, temperature=0.08):
    candidates = [positive, *negatives]
    logits = np.array([query @ item for item in candidates]) / temperature
    logits -= logits.max()
    probabilities = np.exp(logits) / np.exp(logits).sum()
    return -np.log(probabilities[0])

query = unit(rng.normal(size=32))
positive = unit(query + 0.4 * rng.normal(size=32))

for hardness in [0.0, 0.3, 0.6]:
    negatives = [
        unit(hardness * query + (1 - hardness) * rng.normal(size=32))
        for _ in range(16)
    ]
    print(hardness, round(infonce(query, positive, negatives), 3))

Weak supervision expands positive coverage at the cost of label noise. E5 filtered naturally occurring web pairs before contrastive pretraining (Wang et al. 2022). Later work generated diverse tasks and pairs with a language model, then trained an embedder on the synthetic data (Wang et al. 2024). Synthetic generation can cover rare intents and languages, but it also copies the generator's factual errors, style, and task assumptions. Keep prompts, generator revisions, filters, and source rights with the dataset. Split by source and time before mining so near duplicates cannot leak across train and evaluation.

Distillation is another label source. A cross-encoder can score a bounded set of query-document pairs, and the dual encoder can learn its pairwise margins rather than its raw score scale. Distillation inherits the teacher's bias and is limited to the candidate set the teacher sees. It cannot teach a distinction that mining never presents.

One model can expose several contracts

General-purpose embedders often support task instructions. INSTRUCTOR trained a single model across a large mixture by embedding each input with a description of its task and domain (Su et al. 2023). This does not mean arbitrary prose is understood as a policy. FollowIR found that many tested retrievers used detailed instructions largely as extra keywords, while targeted fine-tuning improved constraint following (Weller et al. 2025). Treat the exact instruction template as a versioned part of the spec and test paraphrases, negation, exclusions, and conflicting constraints.

Dimension can also be a trained interface. Matryoshka representation learning attaches losses to selected vector prefixes. For a retrieval adaptation, write

$$z^{(m)}= \frac{z_{1:m}}{\lVert z_{1:m}\rVert_2}, \qquad \mathcal{L}_{\mathrm{MRL}} =\sum_{m\in\mathcal{M}} c_m\, \mathcal{L}_{\mathrm{NCE}}^{(m)}.$$

Here $z\in\mathbb{R}^{d}$ is the full embedding, $z_{1:m}$ is its first $m$ coordinates, $\mathcal{M}\subseteq\{1,\ldots,d\}$ is the set of trained prefix dimensions, $c_m\ge0$ is the weight for prefix $m$, and $\mathcal{L}_{\mathrm{NCE}}^{(m)}$ applies the contrastive loss using independently normalized $m$-dimensional query and document prefixes. Only trained dimensions are promised. Slicing a normalized full vector does not leave a normalized prefix, and arbitrary intermediate dimensions are not guaranteed to work.

The original Matryoshka work learned coarse-to-fine representations across vision and language evaluations and reported strong size and retrieval trade-offs on its tested datasets (Kusupati et al. 2022). Its main systems benefit is one model forward pass that can feed indexes of several supported dimensions. The multiple training losses are not literally free, and a shorter prefix is not always strictly worse or always equal to a separately trained model. Measure every offered dimension on the deployment workload.

Indexes make compatibility operational

For $N$ stored vectors, dimension $d$, and $b$ bytes per coordinate, the raw vector payload is

$$S_{\mathrm{raw}} = N d b \quad\text{bytes}.$$

Here $S_{\mathrm{raw}}$ excludes document metadata, graph or inverted-index overhead, replicas, centroids, and temporary migration copies. Lower dimension, reduced precision, product quantization, and late-interaction compression change storage, memory bandwidth, latency, and neighbor recall in different ways. A smaller payload is useful only if the resulting end-to-end retrieval quality is still acceptable.

Changing any EmbeddingSpec field creates a new index generation. Do not compare query vectors from one model with document vectors from another, even when the dimensions match. Build the replacement beside the current index, re-embed the complete authorized corpus, and dual-read representative traffic. Compare exact vector rankings first, then approximate-index rankings, so representation drift is not confused with ANN loss. Cut traffic over explicitly and retain a rollback window. Tombstones, source updates, and ACL changes must reach every live generation and every embedding cache.

Embeddings are derived content, not anonymized content. They inherit the source's tenant, authorization, retention, residency, and deletion policy. A vector index must not become a path around the authorization invariant in Chapter 44. Cache keys include the spec hash, input content hash, role, instruction version, and tenant or other isolation boundary. Provider logs and batching paths need the same data-handling review as any other model call.

Evaluation must match the use

An aggregate embedding leaderboard is useful for screening, not deployment selection. MTEB demonstrated that models which perform well on semantic textual similarity do not necessarily dominate clustering, classification, reranking, and retrieval (Muennighoff et al. 2023). MMTEB expanded the language and task coverage and showed why a single English average is even less informative (Enevoldsen and others 2025). These benchmarks compare many workloads efficiently; they cannot reproduce a private corpus, its permissions, its update pattern, or its exact definition of relevance.

For a retrieval embedder, freeze the corpus version, chunking, query set, relevance judgments, query and document roles, spec hash, and index parameters. Then evaluate these boundaries separately:

• Representation: exact-search recall@k and nDCG@k, hard-negative error, calibration by score band, and performance against BM25 and the current model.
• Approximation: ANN recall against exact neighbors, empty-result rate, latency, memory, and index-build time at each compression or dimension setting.
• Slices: query intent, document source, length and truncation, language, script, cross-lingual direction, code switching, freshness, and permission cohort. Report the worst material slice as well as the mean.
• Operations: query encoding latency, document throughput, accelerator and CPU cost, raw and total index bytes, cache hit rate, generation age, migration duration, and rollback success.
• End to end: evidence recall after packing, answer quality, citation support, abstention, latency, and cost with the generator and reranker held fixed.

Use paired per-query comparisons and bootstrap confidence intervals. Include queries with multiple valid passages and unanswerable queries. Perturb task instructions, shorten and lengthen documents, add near duplicates, change a fact, delete a source, revoke access, and shift the query language. A model that wins a clean average but fails deletion, one language, or a high-value query cohort is not the better production embedder.

The resulting boundary is precise. An embedder produces a versioned score under one task contract. Retrieval uses that score to select evidence; it still owns authorization, freshness, fusion, and abstention. The next chapter, Chapter 46, starts after retrieval has selected evidence and asks how instructions, history, tool results, and retrieved material should share a bounded working context.

Further reading