Agent Architectures

An agent is a language model placed inside a loop that lets it act, observe the result, and act again. The loop has a standard four-part decomposition: planning, memory, tool use, and the action loop that ties them together. Its control flow interleaves reasoning with action because the next decision must see what the last action revealed. Tool use is the part that turns a text generator into something that changes the world.

A plan that meets the world

Imagine the obvious way to build an agent. Ask the model to read the task, think it through, and write the full plan: step one, run the tests; step two, open the failing file; step three, fix the off-by-one on line forty; step four, rerun. Then execute the plan, one step at a time, top to bottom.

This design is clean, legible, and wrong. The plan was written before any step ran, which means it was written before the agent knew anything the steps would reveal. Step one runs the tests and the failure is not the one the model assumed. The bug is not on line forty, and it is not an off-by-one. Every later step in the plan now refers to a world that does not exist. The agent is holding a script for a play whose first scene already went differently. It can follow the script into nonsense, or it can throw the script away. Either way, planning the whole task up front bought nothing, because the first observation invalidated it.

The structure of an agent is therefore a loop, not a plan. The shape of that loop explains why thinking and doing must be interleaved, and why the agent decomposes into planning, memory, tool use, and action.

Closing the gap

Step back to why a loop is needed at all. A base model maps a prompt to a continuation. That is enough to answer a question whose answer is already latent in the weights, and useless for a task whose answer depends on the state of the world: the current contents of a file, the result of a query, whether a test passes after an edit. The model cannot read that state, and it cannot change it. It emits text and stops.

The essential problem of an agent is to close that gap. The model needs a way to take an action in the world, see what the action produced, and feed that observation back into its next decision. It also needs to do this over many steps, because real tasks are not one action long: fixing a bug is read, hypothesize, edit, test, and very likely repeat.

Three constraints shape every answer to this problem, and they are worth naming before any architecture is drawn. The context window is bounded, so a long task cannot keep every observation in view at once. Each step also costs a model call, which gives the loop both a latency and a price. Third, the model is probabilistic: any single action can be wrong, so the architecture has to tolerate and recover from mistakes rather than assume them away. The plan-then-execute design failed precisely because it ignored that last point and assumed its first guess would survive contact with the world.

Try changing the per-step reliability and the task length below to see why a single up-front plan is so fragile: a plan with no recovery succeeds only if every step lands, so its success probability is the per-step reliability raised to the number of steps.

import numpy as np
n = np.arange(1, 21)               # task length: 1 to 20 steps
for p in [0.99, 0.95, 0.90, 0.80]: # per-step reliability
    survive = p ** n               # P(all n steps land, no recovery)
    import matplotlib.pyplot as plt
    plt.plot(n, survive, marker="o", ms=3, label=f"p={p}")
plt.xlabel("steps in the plan (n)")
plt.ylabel("P(plan succeeds with no recovery)")
plt.ylim(0, 1); plt.legend(); plt.title("Why plan-then-execute is fragile")
plt.show()
for p in [0.95, 0.80]:
    print(f"p={p}: 10-step plan succeeds {p**10:.1%} of the time")

The loop, and why it interleaves

The design that works does not separate reasoning from acting. It interleaves them. The model produces a short reasoning trace, then an action, observes the result, then reasons again with that observation in hand. This is the reason-and-act pattern, named ReAct by Yao et al. (Yao et al. 2023), and the synergy is the point: the reasoning lets the model decide and adjust its plan as it goes, and the actions let it pull in facts that the reasoning alone would have to hallucinate. The plan is never written all at once. It is revised at every turn against what the last action returned, which is exactly the failure mode of the previous section turned into a virtue.

The loop in Figure 38.3 is the heart of the architecture, and its specific shape is what separates an agent from a single model call. Everything else here is either a part the loop animates or a consequence of running it in production.

The runtime contract that implements the loop is small to state. Create a session bound to an agent definition. Send the context to the model, dispatch the tool calls it returns, append the results, and repeat until the model signals done or the loop is interrupted. That sequence is the reason-and-act pattern in operational form. The mechanism that makes this contract survive production, durability across crashes, scheduling, interruption, is the harness, and it is the subject of Chapter 41. The split is architecture here, engineering there.

Anatomy of the loop

The loop has four moving parts. The dominant framing decomposes an agent into these four around a model that acts as the controller. The decomposition is Lilian Weng's (Weng 2023), and it has become the common vocabulary: planning, memory, tool use, and the action loop that ties them together.

• Planning turns a goal into a sequence of steps. At its simplest this is the model deciding, one step at a time, what to do next. At its most structured it is explicit task decomposition into subgoals, with self-reflection on past steps to correct course.
• Memory is what survives across steps. Short-term memory is the context window itself, the running transcript of what has happened. Long-term memory is an external store the agent reads from and writes to, because the window is too small to hold a long task. Memory as a system is the subject of Chapter 39; here it is one of the four parts the loop relies on.
• Tool use is how the agent reaches outside its weights: a function it can call to fetch a page, run code, query a database, or edit a file. A tool call is the agent's only way to read or change external state.
• Action is the loop that drives the other three: send the current context to the model, let it choose a tool call, execute the call, append the result, and repeat.

It helps to see the parts laid out by where they live. Planning is reasoning the model does inside its own forward pass, and short-term memory, the context window, is inside that same view. Tool use is the single edge that crosses to the world outside the weights, with long-term memory as a store on the far side of it.

Of the four, tool use is the one that earns the name agent. Planning and memory shape how the model thinks; tool use is the only part that lets it reach past the weights and touch the world, the single edge in Figure 38.4 that crosses the boundary. Without it the loop spins on text alone.

How the pieces matured

The decomposition did not arrive fully formed. The order in which the parts became real explains why the synthesis took the shape it did.

The first capability to mature was tool use as a learned skill. Toolformer (Schick et al. 2023) showed that a model could teach itself, in a self-supervised way, when to call an API and how to splice the result back into its generation, which established that tool use is something a model can be trained to do rather than a feature bolted on at inference time.

In parallel, the reasoning side matured. Chain-of-thought prompting showed that letting a model write intermediate steps before an answer improved performance on multi-step problems, a thread Chapter 24 follows in depth. ReAct's contribution was to fuse that reasoning with action in one interleaved trace (Yao et al. 2023), which directly attacked chain-of-thought's weakness: a pure reasoning chain has no way to check itself against the world, so it propagates its own errors and hallucinations, while interleaving a Wikipedia lookup between reasoning steps lets the model ground each step in a fact it just retrieved. The four-part decomposition was the synthesis that named the pieces (Weng 2023), with planning and memory as first-class components alongside tool use rather than implicit features of a prompt.

What has changed most recently is where the reasoning lives. Early agents carried their reasoning in the prompt: the harness instructed the model to think step by step and to interleave thoughts with actions. Reasoning models trained to deliberate internally, covered in Chapter 28, move some of that work below the loop. The agent architecture is the same shape, a model in a reason-and-act loop, but the planning component is increasingly something the model does in its own trained reasoning rather than something the harness has to scaffold with prompting.

Living tensions

The architecture's choices are balances, and each has a cost worth naming.

• Explicit planning versus letting the model reason as it goes. Up-front task decomposition gives a legible plan a person can inspect, and it can keep a long task on track. It also commits to a structure before any observation exists, so a plan that meets a surprising result is a plan that has to be torn up. This is the opening failure in miniature. Step-at-a-time reasoning adapts to every observation but has no global view and can wander. Most production agents lean toward the reason-and-act middle: a light plan, revised every step against what the last action returned.

• Tool count versus model attention. Every tool the agent can call is a schema in the model's context and a candidate it must consider. A handful of well-chosen tools is easy for the model to select among. A large catalog costs context on every turn and degrades selection accuracy, because the model's attention is finite and irrelevant tools actively hurt. This is the pressure that forces a tool-selection strategy, discussed below and in Chapter 41.

• Short-term versus long-term memory. Keeping everything in the context window is simple and lossless until the window fills, at which point the task stalls. Externalizing memory lets the task run indefinitely but makes the agent's recall only as good as what it chose to write down and can retrieve. The boundary, what stays in the window and what moves to a store, is a design decision introduced here and expanded in Chapter 39.

• A tool call per turn versus code as action. The loop below dispatches one structured call per iteration, which keeps every action legible, gateable, and interruptible. CodeAct established the alternative: the model emits executable code that composes many tool invocations, loops, and filters inside a sandbox, collapsing several loop iterations into one action and raising success rates by up to 20% across 17 models (Wang et al. 2024). The trade is the loop's shape itself: fewer, bigger actions spend fewer tokens on round trips but are harder to inspect and to gate per operation, a tension Chapter 46 meets again when code execution wraps MCP tools.

What the loop dispatches

The loop itself is short to write and unforgiving in its details. In pseudocode it is a while over model calls:

loop:
    response = model(context)
    if response has no tool call:
        return response            # the model is done
    result = execute(response.tool_call)
    context = context + response + result

Everything hard is hidden in execute and in the management of context.

The tools a loop dispatches are not uniform, and a loop that treats them all alike mishandles most of them. Tools divide by statefulness into three kinds. Stateless tools hold no state on the agent's side: a web fetch, a sandbox command given a sandbox ID, a plain REST call. They are the easy case and scale trivially. Stateful-connection tools, the Model Context Protocol (MCP) in its 2025 revisions being the canonical one (Model Context Protocol 2026), carry negotiated capabilities and cursor state in a connection the loop must hold across calls. Stateful-resource tools keep their state in an external system, a database session or a long-running job, that the loop must correlate to across turns. The architectural point is that these three answer the question "what happens on restart?" differently: nothing, reconnect, or rediscover. The engineering of holding those connections alive through reconnects and token refreshes is the harness's job and lives in Chapter 41; the architecture's job is to know the three classes exist and not collapse them into one. The classes are more durable than any protocol's place in them: the 2026 MCP spec work removes the protocol-level session, moving the canonical stateful-connection example toward the stateless class while the taxonomy itself survives (Model Context Protocol 2026). Figure 38.5 lines the three up against the discriminating question.

One action space sits at the taxonomy's edge: computer use, where the tool is the whole graphical interface, actions are clicks and keystrokes, and observations are screenshots (Anthropic shipped it in October 2024, OpenAI's Operator followed in January 2025 (Anthropic 2024)). It stresses the loop hardest, long horizons and noisy pixel observations with no schema to validate a call against, which is why hybrid systems such as CoAct-1 route between GUI actions and generated code (Song et al. 2025); Chapter 52 grades these agents by final environment state.

The other recurring implementation question is which tools the model sees in a given turn. A static flat list, every tool in every prompt, is the simplest design and runs out at roughly thirty to fifty tools, because context cost and selection accuracy both degrade as the catalog grows. Past that point the options are to retrieve a relevant subset per turn, to show a pruned set per phase, or to let the model discover tools on demand. The selection mechanism is harness machinery; the reason it is forced is a model property, which is the constraint the next box names.

Drag the slider to see the constraint the next box describes: as the number of tools in context grows, function-calling accuracy falls off, which is why a flat catalog stops working past a few dozen tools.

Further reading