Context Engineering

A frontier model does not act on the world. It acts on its context window, the bounded sequence of tokens it can attend to in a single forward pass. Everything the model knows about a task at the moment of generation, the instructions, the retrieved documents, the conversation so far, the tool results, must be assembled into that window by something outside the model. The field renamed this work from prompt engineering to context engineering in 2025 because the unit of work had changed from a single string to the assembly of the whole window. A longer context window does not by itself solve the problem; token budgets and tool protocols shape what an agent can actually do.

The argument turns on one stubborn fact: a bigger window does not make the problem go away. It moves the problem. Each section below pushes on that claim from a different side, from why the window misbehaves, to what lever is left, to how a running agent assembles its context and where that assembly breaks.

The window is the only channel

The context window is a fixed, scarce resource, and it is the only channel through which a model perceives a task. That one sentence is the key to the whole chapter, so it is worth getting clear.

The window is finite, and every token in it competes for the same space. A system prompt, a tool catalog, retrieved passages, and a long conversation history all draw from one budget. When the budget is exceeded, something must be dropped, summarized, or retrieved on demand, and each of those choices changes what the model can answer. There is no second channel. Whatever the model does not see in the window, it does not know.

A reader may reasonably expect that a larger window dissolves this pressure. The next section is the reason it does not.

Why a bigger window does not help

A model does not use the tokens in its window uniformly. Even when the relevant information is present, a model tends to attend most reliably to the beginning and the end of its input and least reliably to the middle. Liu et al. named this the lost-in-the-middle effect (Liu et al. 2023), measuring a characteristic U-shaped accuracy curve on multi-document question answering and key-value retrieval: performance is highest when the needed fact sits near the start or the end of the context and degrades, sometimes below the closed-book baseline, when it sits in the middle. A bigger window does not repeal this. It enlarges the middle.

Figure 46.2 sketches the consequence as a placement map. Reliability is high at the two boundaries and sags through the long middle, so where a fact sits in the window, not merely whether it is present, governs whether the model uses it.

Vary where the needed fact sits and watch the recall move: this simulates a U-shaped recall profile, then Monte-Carlo places one fact at the head, the middle, and a random slot to compare hit rates.

import numpy as np
import matplotlib.pyplot as plt

rng = np.random.default_rng(0)
pos = np.linspace(0, 1, 41)              # position in the window, 0=head 1=tail
recall = 0.45 + 0.5 * (2 * pos - 1) ** 2 # U-shape: high at edges, low in middle

def hit_rate(p, trials=20000):
    prob = np.interp(p, pos, recall)
    return rng.random(trials) < prob

for name, p in [("head", 0.0), ("middle", 0.5), ("random", None)]:
    where = rng.random(20000) if p is None else p
    print(f"{name:7s} recall = {hit_rate(where).mean():.3f}")

plt.plot(pos, recall); plt.xlabel("position (head to tail)")
plt.ylabel("recall probability"); plt.title("lost in the middle"); plt.show()

So the problem is not only "how do I fit everything in," it is "what do I put in, in what order, so the model uses it." That is a budgeting and placement problem, and it is the work context engineering takes over.

The lever: context as a programming surface

The model's weights are fixed at inference time. The only lever left is what enters the window and where, so the design idea is to treat the context window as a managed buffer, not a prompt.

The lever is real because of in-context learning. Brown et al. showed with GPT-3 that a sufficiently large autoregressive model can perform a task from demonstrations placed in its context, with no weight update at all (Brown et al. 2020). They distinguished three regimes by how many demonstrations the prompt carries: zero-shot, an instruction with no examples; one-shot, a single example; and few-shot, a handful, typically ten to one hundred, that fit in the window. The finding that mattered for everything since is that the context is a programming surface. You can change a model's behavior by changing its input, within one session, at the cost of the tokens the demonstrations consume.

Once the context is a programming surface, the assembly of it becomes the artifact worth engineering, which is the work a survey of the field now catalogs as a discipline in its own right (Mei et al. 2025). Andrej Karpathy made the argument for the name in mid-2025, proposing context engineering over prompt engineering on the grounds that a "prompt" connotes a short instruction, while real applications perform "the delicate art and science of filling the context window with just the right information for each step" (Karpathy 2025). Tobi Lutke framed it from the application side the same week: context engineering is "the art of providing all the context for the task to be plausibly solvable by the LLM" (Lütke 2025). The rename is not cosmetic. It moves the unit of work from a single string a human writes to a pipeline a system assembles: system prompt, retrieved documents, conversation history, distilled memory, and tool outputs, composed for each step of a running agent.

Three moves: budget, place, compress

Three design moves follow directly from the two facts above, that the window is finite and that the model underuses its middle.

Budgeting. Treat the window as a cost-accounted resource. Account for the fixed overhead first, the system prompt and the tool definitions, then divide what remains among history, retrieval, and headroom for the model's own output. Retrieval (Chapter 44) exists precisely so that not everything has to live in the window at once; the corpus stays outside and only the relevant slice is paged in per query.

Placement. Because the middle is the weak position, order matters. Put the instruction the model must follow and the highest-value evidence where attention is reliable, near the boundaries, not buried in a long undifferentiated dump. Lost-in-the-middle is a property of the model, and it dictates a choice in the layer above it.

Compression over accumulation. A conversation or an agent trajectory grows without bound, but the window does not. Old turns are summarized into distilled state, tool outputs are filtered before they re-enter the window, and durable facts are written to an external memory (Chapter 39) and retrieved when needed rather than carried forward verbatim.

From a string to a pipeline

The three moves did not arrive all at once. The lineage runs from a single string toward a managed pipeline, and tracing it shows why each move became necessary.

The first era was prompt engineering. With in-context learning established, practitioners learned to coax behavior from a model by wording: phrasing an instruction carefully, adding a few worked examples, appending "let's think step by step" to elicit a chain of thought (Chapter 24). The artifact was a prompt template, and the skill was writing it.

The limits of that era showed up as applications grew. A single hand-written string cannot hold a large knowledge base, a long conversation, or the output of tools the model calls. Retrieval-augmented generation (Chapter 44) answered the knowledge-base limit by fetching relevant passages at query time. For the action limit, agent harnesses (Chapter 41, Chapter 38) let the model call tools and fed the results back into the window. Both turned the context into something assembled at runtime from many sources, which is the moment prompt engineering became too small a word for the job. The rename to context engineering in 2025 was the field naming what it had already started doing: Karpathy and Lutke gave it a label, but the practice of budgeting, retrieving, ordering, and compressing the window predated the term.

The recursion: budgeting the tools themselves

Tools created their own integration problem and their own protocol, and that protocol then turned the budgeting problem back on itself.

Before late 2024, every application wired each model to each data source with bespoke glue, an N-times-M problem. Anthropic released the Model Context Protocol (MCP) in November 2024 as an open standard for that wiring (Anthropic 2024): a model speaks one protocol, and any MCP server, for a filesystem, a database, an issue tracker, exposes its tools and data through it. MCP standardized how external context and capabilities enter the window. Figure 46.4 contrasts the two wirings: the bespoke mesh where every model integrates every source directly, against the hub where each side speaks the protocol once.

The protocol then revealed the budgeting problem at the layer above it. As agents connected to dozens of MCP servers, the tool definitions themselves filled the context before the agent read a single request. In November 2025 Anthropic described code execution with MCP as the response (Anthropic 2025): rather than loading every tool definition upfront, present MCP servers as code APIs that the model discovers and calls by writing code, loading only the definitions a task needs and filtering tool results in the execution environment before they return to the window. Their illustrative example reduces a workflow from about 150,000 tokens of loaded definitions and intermediate results to roughly 2,000, a 98.7 percent reduction. The same lesson recurs: the constraint is the context budget, and the design moves to defend it.

Where the balances bite

Context engineering is a set of balances, each with a knee.

• Few-shot versus zero-shot. Demonstrations raise reliability on a task but cost tokens, and every demonstration token is a token unavailable for retrieval, history, or output. As instruction-tuned models improved, the break-even shifted: many tasks that once needed several examples now run zero-shot, and examples are worth their cost only where they pin down a format or a corner case the instruction cannot convey concisely.
• More context versus better context. Stuffing the window with everything plausibly relevant is the easy move and often the wrong one. It raises cost per call, and it pushes high-value evidence into the lost-in-the-middle dead zone where the model underuses it. A smaller, well-selected, well-ordered context frequently beats a larger indiscriminate one.
• Static prompt versus assembled pipeline. A fixed template is simple, debuggable, and cheap to reason about. A pipeline that retrieves, ranks, summarizes, and orders per step is more capable but introduces moving parts that fail in their own ways: a bad retrieval poisons the context, a faulty summary drops the fact the next step needed.
• Tool breadth versus token cost. Connecting many tools makes an agent more capable in principle, but loading all their definitions upfront spends the budget before work begins. Progressive disclosure, loading definitions on demand as in code execution with MCP, recovers the budget at the cost of an extra discovery step and a more complex runtime. The principle has since generalized beyond tools: Agent Skills hold only a name and description in context and load the full instruction file on demand (Anthropic 2025), and compaction, summarizing older turns as the conversation nears the window limit, is now a platform primitive rather than harness custom code (Anthropic 2026).

Building one context, and how it breaks

In practice the context for one model call is built, not written. A budget-aware assembler is the minimal shape:

def build_context(task, history, tools, retriever, window):
    budget = window - reserve_for_output(window)
    parts = []
    parts.append(system_prompt(task))          # fixed, goes first
    parts.append(tool_manifest(tools))         # load on demand if large
    evidence = retriever.search(task.query)    # only the relevant slice
    parts.append(rank_and_trim(evidence, budget))
    parts.append(summarize_old(history, budget))  # compress, do not accumulate
    return order_for_attention(parts)          # high-value near boundaries

Figure 46.5 traces the same assembler as a data flow. Many sources feed one budgeted buffer: the fixed overhead is accounted first, retrieval pages in only a slice of the corpus, history is compressed rather than carried forward, and a final ordering step lifts the highest-value parts to the boundaries where Figure 46.2 says the model will read them.

Three properties of this sketch carry the chapter's argument. The output reserve comes off the top, because a context that fills the window leaves the model no room to answer. Retrieval returns a slice, not the corpus, so the knowledge base lives outside the window and only its relevant part is paged in. And order_for_attention is not cosmetic: it places the instruction and the strongest evidence near the reliable boundaries.

The failure modes are worth naming because each bites at a different place. Context overflow truncates silently, dropping whatever fell off the end, often the very history the next turn needed. Lost-in-the-middle degradation hides behind a green build: the fact is present, the model just does not use it, and the only way to see the loss is to vary placement and watch the score move. Retrieval poisoning is cruder: a confidently wrong passage sits in the window, and the model treats it as evidence. Tool-definition bloat, finally, spends the budget before the agent even starts, the problem code execution with MCP exists to solve. Each of these is a budgeting or placement failure, which is why budgeting and placement, not wording, are the substance of the discipline.

Further reading

The sources cited inline above appear on the book's References page. The survey below is a fuller map of the field for a reader who wants the broader landscape.