Part IV: Reasoning and Test-Time Compute

"... generating a chain of thought ... significantly improves the ability of large language models to perform complex reasoning."

Jason Wei et al., "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models"

Reasoning in modern models is easiest to misunderstand when it is treated as a new organ inside the network. This part takes the more useful view: reasoning is a way of spending computation before the answer is fixed. That extra computation, spent while the model answers rather than during training, is what the title calls test-time compute. Sometimes that spending happens in the prompt. It can be organized as search, or delegated to a program, solver, or proof checker. In other systems the model is trained to reason; in still others the work stays at inference, where it samples, checks, branches, or routes by difficulty before committing.

Chapter 24 begins with the smallest move, giving the model room to think and refusing to trust a single sample. Chapter 25 then makes the control problem explicit: a chain is one path, a tree keeps alternatives alive, and a graph reuses partial work. Chapter 26 splits reasoning into translation and execution, where the model writes a program, query, symbolic chain, or proof script and an external runtime computes or checks it. Chapter 27 studies the verifier itself, from unit tests and answer checkers to process reward models and generative verifiers.

Chapter 28 moves the cost into training, where verifiable rewards (reward signals taken from answers that can be checked automatically) and reinforcement learning try to make useful reasoning habitual rather than ad hoc. In Chapter 29 the traces themselves become data: self-generated rationales, rejection sampling, small curated sets, distillation, and long-to-short transfer. Chapter 30 then treats computation at test time as a budgeted resource: parallel attempts, sequential refinement, verification, routing by difficulty, and the production problem of overthinking.

The practical test for any reasoning claim is therefore concrete: where is the extra work paid for, who checks it, what data records it, and does more computation improve coverage, selection, or both. Once those questions are separated, the difference between a prompting method, a reasoning-trained model, and a test-time scaling system becomes an engineering choice rather than a mystery.