Part IX: Infrastructure, Compute, and Frontiers

"The free lunch is over."

Herb Sutter, "Welcome to the Jungle"

The stack can be drawn as software, but its lower layers are stubbornly physical. A chip has only so many memory pins, a cluster only so many network tiers. Checkpoints have to cross storage systems, and power arrives on a schedule measured in years. At this scale, failure is not an exceptional event but a statistical certainty. This part goes beneath the model and asks what the machine makes possible before the math begins.

Chapter 62 starts with bandwidth, the hidden shape behind many training and serving decisions. Chapter 63 turns to the software substrate that maps model mathematics onto that hardware: automatic differentiation, the frameworks built around it, and how sharding entered the programming interface. Chapter 64 follows that substrate down to device code, where fusion, IO-aware kernels, and the compiler stack decide how much of the hardware floor is reachable, and where the industry's deepest software moat sits. Chapter 65 shows how long runs survive failure, keep accelerators fed, and remain observable. Chapter 66 and Chapter 67 move from racks to packages, HBM, supply chains, export controls, and the geography of compute. Chapter 68 adds the constraint that often arrives last in conversation and first in deployment: power. Chapter 69 follows the failures that become inevitable at scale. Chapter 70 asks what happens when data, feedback, and learnable signal run thin. Chapter 71 then returns to measurement, where the frontier is less a single score than a horizon whose instruments keep saturating. Chapter 72 closes the part with the other side of the frontier: once models can produce claims faster than people can inspect them, proof, replication, oversight, and acceptance become infrastructure too.

Read this part as a reminder that intelligence runs on infrastructure. Every elegant modeling decision eventually becomes traffic, heat, memory pressure, a supply chain, a power contract, a geography, or an operating procedure. The frontier is not only where capability grows. It is where constraints become visible.