The Whole Stack in One Pass

This book is long, and its table of contents can make it look like a list of unrelated techniques. Start instead with one thread that can be followed again and again: an ordinary request moving from raw text on the open web to the deployed agent that finally answers it, with each step landing in a layer and a chapter. The point is orientation, not depth. Every mechanism here is a forward pointer, deliberately left for its own chapter.

Pick one task and refuse to let go of it: a user asks an assistant to find the bug in a small repository, propose a fix, and explain it. That one request touches every layer this book covers. Rather than survey the layers abstractly, this chapter follows the request as if it were a traveler, and treats each layer it must pass through as a station on the route. Following it once, in order, is the fastest way to see why the stack has the shape it does. Two running models ride alongside the request the whole way, chosen because they answer the same questions with opposite engineering: Llama 3, a dense Transformer (the standard model architecture, defined in Chapter 8) (Grattafiori and others 2024), and DeepSeek-V3, a mixture-of-experts (MoE) model that activates only a few expert sub-networks for each token (DeepSeek-AI 2024). Both lineages have since converged on sparse designs: Llama 4 (April 2025) is Meta's first mixture-of-experts family (Meta AI 2025), and DeepSeek-V4 followed in April 2026 (DeepSeek-AI 2026); that convergence is itself the verdict of the constraint arrow drawn later in this chapter.

Station one: from web bytes to tokens

The task begins long before the user types it, in a corpus. The model that will eventually read the repository learned what code looks like from trillions of tokens scraped, filtered, and deduplicated from the web and from public repositories. Llama 3's largest model was pre-trained on 15.6 trillion tokens (Grattafiori and others 2024); DeepSeek-V3 on 14.8 trillion (DeepSeek-AI 2024). Those numbers are not the interesting part. The interesting part is everything done to the raw bytes before they counted as a token at all: source selection, quality filtering, near-duplicate removal, and decontamination against the very benchmarks the model will later be graded on. That work belongs to Chapter 6. The step that turns a filtered byte stream into the integer tokens the model actually consumes, and the vocabulary that decides how a variable name or a Chinese sentence is split, belongs to Chapter 7.

Station two: base-model formation turns compute into a base model

With a token stream in hand, the first large phase is pre-training: compute is spent to minimize next-token loss and produce a base model. The model guesses the next chunk of text, and the error in that guess is the loss it is trained to reduce. How large a model to train on how many tokens, for a fixed budget, is the question of Chapter 5, first posed as a set of power laws relating loss to model size, data, and compute (Kaplan et al. 2020) and later sharpened into a compute-optimal balance between parameters and tokens (Hoffmann et al. 2022). What the network itself is built from, attention and residual streams and normalization, goes back to the original Transformer (Vaswani et al. 2017) and is the subject of Chapter 8.

Here the two running examples already diverge, and the divergence is the book in miniature. Llama 3's 405B model is a dense Transformer: every parameter participates in every token (Grattafiori and others 2024). DeepSeek-V3 is a mixture of experts, 671B total parameters but only 37B activated per token, paired with multi-head latent attention (MLA) (Multi-head Latent Attention), an attention design that shrinks the inference-time cache, (DeepSeek-AI 2024). That choice, dense versus sparse, is Chapter 9. Running either training job across thousands of accelerators without the run diverging or the hardware idling is Chapter 10, where DeepSeek-V3's FP8 mixed-precision recipe and its 2.788 million H800 GPU-hours of total training live (DeepSeek-AI 2024).

Many modern runs do not jump directly from this broad base to post-training. They add a mid-training phase: late high-quality annealing, specialist code or math mixtures, instruction-formatted next-token data, or long-context extension. That bridge is Chapter 11. It still belongs to base-model construction, because it changes the distribution the model has internalized before adaptation starts changing behavior.

Station three: adaptation teaches the model to want the task

A base model predicts plausible continuations of text. It does not yet want to fix your bug. Turning a next-token predictor into something that follows instructions, respects a behavior specification, refuses harmful requests, and uses verifiable rewards where the task permits them is adaptation, the second great expense after base-model formation. Supervised fine-tuning on demonstrations is Chapter 17, and the data layer underneath preferences is Chapter 18. To learn from human or automated preference signals, a model either trains a reward model and optimizes a policy against it or takes the direct-preference route that skips the explicit reward model: that fork is Chapter 19 and Chapter 20. Safety tuning and instruction hierarchy belong to Chapter 22. And teaching a model to spend tokens thinking before it commits to an answer, then deciding when that thinking should be prompted, searched, verified, distilled, or paid for at inference time, is the arc of Part IV from Chapter 24 through Chapter 30.

Station four: serving turns weights into a service

Now the model exists and the user has typed the request. Serving it is its own engineering discipline, because a trained model is a pile of weights, not a service. The core tension is that generation is memory-bound and the key-value cache (the key-value cache) grows with every token, which is where Chapter 31 starts. Batching many users and paging that cache falls to Chapter 32. Two further levers cut the bill directly: decoding more than one token per forward pass (Chapter 33), and lowering precision so each weight and each cache entry gets cheaper (Chapter 34).

Station five: the agent loop closes

Finally the loop closes. The assistant does not answer the bug report from memory alone. It reads files, runs the test suite, sees the failure, edits, and re-runs, one observation-and-action step at a time. That control structure is Chapter 38, the runtime that drives it is Chapter 41, and the retrieval that pulls the right files into the context window is Chapter 44. Deciding what to keep in that finite window, and what to summarize or drop, is Chapter 46.

The slice, drawn as one pass through the stack, is Figure 1.2.

What the journey hides: three nested loops

The stations above are arranged in space, one after another, but the stack also runs in time, and the timing hides a structure worth naming once because it recurs in every part of the book. The stack is built from three nested loops, each a control structure that consumes compute to reduce a different kind of error.

The training loop runs over a corpus, adjusting weights by gradient descent to lower a loss. Paid once, it runs for weeks and leaves an artifact behind. Over a prompt runs the inference loop, generating one token at a time from frozen weights, and this one is paid on every request, forever, which is why a fraction of a cent per token decides the economics of a product. Outermost is the agentic loop, turning over an environment, alternating model calls with tool calls until a task is done; it wraps many inference loops inside one unit of useful work, and it is where a model stops being a text predictor and starts being infrastructure.

The containment is the point, and Figure 1.3 draws it: the artifact that training freezes once sits at the center, each inference loop reads it and pays per request, and the agentic loop folds many such inference loops into a single unit of useful work. Reading the payment cadence off the figure, once, then per request, then per task, is what makes the constraint arrow below legible.

These loops are not independent, and the most useful insight in this book is where one reaches up to dictate another's design.

Try this to make the constraint arrow concrete: charge each design its training FLOPs (the count of arithmetic operations) once plus its inference FLOPs per served token. Both bills scale with activated parameters, so the sparse design is cheaper in both loops at every serving volume; the roughly $3.3\times10^{24}$ training FLOPs charged to DeepSeek-V3 here are consistent with the 2.788 million H800 GPU-hours station two quoted. What the MoE pays is not FLOPs but memory, routing, and communication: all 671B parameters stay resident so that each token can activate its 37B, the complexity the callout above names.

import numpy as np
import matplotlib.pyplot as plt
# Lifetime FLOPs for two designs that reach comparable capability.
# Both loops scale with activated parameters: training ~ 6*N_act*D (paid
# once); inference ~ 2*N_act per served token. Total parameters N set the
# memory bill, not the FLOPs bill: every expert must stay resident.
designs = {
    "Llama 3 405B dense":  dict(N=405e9, N_act=405e9, D=15.6e12),
    "DeepSeek-V3 MoE 37B": dict(N=671e9, N_act=37e9,  D=14.8e12),
}
served = np.logspace(11, 16, 200)
for name, d in designs.items():
    train = 6*d["N_act"]*d["D"]
    plt.plot(served, train + 2*d["N_act"]*served, label=name)
    print(f"{name}: train ~{train:.1e} FLOPs, "
          f"serve ~{2*d['N_act']:.1e} FLOPs/token")
plt.xscale("log"); plt.yscale("log")
plt.xlabel("tokens served over lifetime"); plt.ylabel("total FLOPs")
plt.legend(); plt.tight_layout(); plt.show()

Judging any station: capability, efficiency, trust

The three loops describe how the stack runs. A second motif describes how to judge any decision inside it. Every technique in this book buys some mix of three things, and usually trades one against another.

Capability is whether the system can do the task at all: can the model find the bug, follow the multi-step plan, reason through the fix. Efficiency asks what that capability costs in compute, memory, latency, and dollars. Trust is whether you can deploy the result and rely on it: whether it is correct, safe, honest about uncertainty, and measurable. DeepSeek-V3 and Llama 3 are both bids to push capability up while holding cost down, and a mixture-of-experts model is a pure efficiency play, more capability per activated parameter (DeepSeek-AI 2024). Trust is the layer the hype cycle forgets and the operator cannot. Knowing whether the agent actually fixed the bug, rather than producing a plausible patch that breaks a different test, is measurement: Chapter 47 for what a static score can and cannot tell you, Chapter 48 for whether the observed gap is real, Chapter 51 for whether fluent output is supported, Chapter 52 for grading a multi-step trajectory, and Chapter 55 for keeping a capable system inside its bounds. The economics that tie capability and efficiency to a price a user will pay are Chapter 76.

Figure 1.5 places the lens over the two running models. Both bid for capability, but a mixture-of-experts model spends fewer activated FLOPs to reach it, pulling toward the efficiency vertex, while trust sits at the bottom for both until measurement has graded the patch each one produced.

Across the book, chapter endings return to the same judgment: what did this technique do to capability, efficiency, and trust? Hold that lens up to the running task and the trade-off is visible. A dense model and a sparse model can reach the same capability, the sparse one does it for fewer activated FLOPs, and neither is trustworthy until something has measured the patch it produced.

How to read from here

The next chapter, Chapter 2, lays out the whole territory this slice cut through, naming the parts and how they connect, so the forward references above resolve into a map. After that the book follows the slice in order: data and architecture, then training, adaptation, and reasoning, then serving, then agents, then the evaluation, governance, and infrastructure that hold it all up. A reader can follow that order straight through, or jump to the chapter that owns whichever layer is on fire today and use the cross-references to climb back out. Either way, this first end-to-end pass keeps the layer in front of you from looking like the whole story.

Further reading