Agent Architectures

A fixed workflow is enough when every step and branch can be specified before execution. An agent becomes useful when the next operation depends on information that is not yet available: a test result, a search result, a user decision, or a changed environment. Its architecture is the closed decision loop that turns those observations into the next bounded action.

That loop needs more than a model and a list of tools. It needs a controller, an assembled context, an action interface, an environment, a state transition, and a termination rule. Tool calling is one action interface; executable code, graphical-interface operations, messages, and environment-native commands are others. Planning and memory are choices within the loop, not a universal list of boxes from which every agent must be built.

Two influential 2023 systems established parts of this design space. ReAct interleaved explicit verbal reasoning with task-specific actions so that later decisions could use new observations (Yao et al. 2023). Toolformer generated, executed, and filtered candidate calls to five fixed APIs, then fine-tuned a model on the retained calls (Schick et al. 2023). They showed that models can learn and use action interfaces, but neither paper defines the only valid agent architecture.

One turn is a state transition

At turn $t$, the runtime constructs a model input, the controller proposes a decision, and the runtime updates durable state:

$$\begin{aligned} x_t &= \mathcal{C}(g,u,s_t,A_t,b_t), \\ y_t &\sim \pi_\theta(\cdot\mid x_t), \\ d_t &= \mathcal{D}(y_t), \\ s_{t+1} &= \mathcal{U}(s_t,d_t,o_{t+1}). \end{aligned}$$

Here, $t$ is the turn index; $g$ is the task goal; $u$ contains user instructions and constraints; $s_t$ is the durable session state before the turn; $A_t$ is the set of action specifications exposed on that turn; $b_t$ is the remaining token, action, time, and cost budget record; and $\mathcal{C}$ is the context assembler. Its output $x_t$ is the bounded input sent to the controller. The controller $\pi_\theta$, with parameters $\theta$, samples output $y_t$; decoder $\mathcal{D}$ parses that output into decision $d_t$. The updater $\mathcal{U}$ combines the old state, the decision, and observation $o_{t+1}$ to produce the next state $s_{t+1}$. Every symbol in the equations refers to a runtime object that can be logged and tested.

The parsed decision must have an explicit type. A useful minimum is

$$d_t \in \{\operatorname{act}(a_t),\operatorname{ask}(q_t), \operatorname{final}(r_t),\operatorname{stop}(e_t)\}.$$

Here, $a_t$ is a requested action or an explicit action batch; $q_t$ is a question for the user, $r_t$ is a proposed final response, and $e_t$ is a structured stop reason. An action can produce a tool result, an execution error, or a policy denial as $o_{t+1}$. A user reply can also become an observation. A final response leaves the loop only after its completion condition passes. A stop reason records success, terminal failure, cancellation, budget exhaustion, or another declared termination condition.

The state $s_t$ and the model context $x_t$ are not the same object. The state may contain a full event log, approval records, handles to external resources, and data that should never enter a prompt. The context is a selected and formatted view of that state. Keeping the two separate makes redaction, summarization, replay, and model changes possible without rewriting the session record.

ReAct is one way to instantiate the controller in Figure 38.2. It places visible verbal reasoning between actions (Yao et al. 2023). The general loop does not require such a trace. A model may reason internally, an external planner may supply a plan, or a deterministic workflow may choose some transitions. What must remain observable is the decision record: selected branch, action arguments, observation provenance, approval result, budgets consumed, and completion evidence.

Architecture is a set of contracts

The familiar labels planning, memory, and tool use are useful topics, but they mix policy, state, and interface concerns. A production architecture is clearer when each runtime role has a contract:

This decomposition also separates architecture from harness implementation. The architecture says that state must survive and actions need a guard. The harness in Chapter 41 supplies queues, leases, retries, checkpoints, and crash recovery that make those contracts durable.

Choose the control pattern from the task

Agent architecture is plural because the controller need not use the same control pattern for every task.

A workflow is usually preferable when it can express the task. It is cheaper, easier to test, and easier to authorize. A model-controlled loop earns its extra cost when it must choose among actions using observations that were unavailable at design time.

Planning is a cadence, not a binary choice

An immutable plan can become stale, but that does not make planning useless. The design question is when a plan is made, how far it reaches, and what causes revision. ReAct demonstrated stepwise adaptation with explicit reasoning and actions (Yao et al. 2023). Reason for Future, Act for Now instead plans a future trajectory, executes the first action, incorporates feedback, and plans again (Liu et al. 2024). Plan-and-Act separates a trained high-level planner from an executor and reports results on long-horizon web tasks (Erdogan et al. 2025). These are different planning cadences, not successive versions of one correct architecture.

A written plan is state, not authority. Each step still passes through current preconditions, permissions, and budgets. Useful replan triggers include a failed precondition, an unexpected observation, a changed user constraint, a denied action, a timeout, a failed completion check, or a remaining-budget threshold. Plans should also record which observations support them; otherwise a summary can preserve the step while dropping the assumption that justified it.

A visible reasoning trace is not required for adaptive control. It can be useful in research systems such as ReAct, but operational inspection should rely on stable artifacts: the plan version, decision type, action arguments, verifier output, and state transition. Reflexion is a specific alternative that turns task feedback into textual reflection stored for later trials (Shinn et al. 2023). It demonstrates one way to update contextual state without updating model weights; it does not make self-critique a general correctness guarantee.

Choose an action representation

An action interface determines what the controller can express and what the runtime can validate. Three common representations make different trades:

Toolformer concerns how a model learned when and how to call a small fixed set of APIs (Schick et al. 2023). CodeAct concerns a different action representation: executable Python. Across 17 models on API-Bank and M³ToolEval, its authors report up to 20 absolute percentage points higher success than their compared text and JSON formats (Wang et al. 2024). That result is scoped to those models, interfaces, and benchmarks. It does not show that code is always safer or cheaper.

Interface design itself can change agent performance. SWE-agent introduced an agent-computer interface for repository navigation, editing, and testing, then evaluated it on software-engineering tasks (Yang et al. 2024). The broader lesson is not that every agent needs that interface. It is that action names, argument shapes, observation formats, and feedback latency are part of the architecture, not neutral plumbing.

Regardless of representation, the model proposes and a reference monitor decides whether execution is allowed. A side-effecting request needs a stable idempotency key so a transport retry does not repeat a purchase, message, or write. Long-running work needs a durable handle rather than a live connection as its identity. Parallel actions are safe only when their dependencies and effects do not conflict.

Make every branch explicit

The minimal loop is not “call a model until it emits no tool.” No call may mean a final answer, a clarification request, a refusal, malformed output, or a model failure. A safer architectural skeleton is:

state = initialize(task, user_constraints, permissions, budgets)

loop:
    stop = check_termination(state)
    if stop exists:
        return recorded_outcome(stop, state)

    context = assemble_context(state)
    raw = controller(context)
    decision = decode_typed_decision(raw)

    if decision is invalid:
        state = record_parse_error(state, raw)
        continue  # bounded by the parse-retry budget

    if decision is ask_user:
        suspend until reply or cancellation
        state = record_user_reply(state, reply)
        continue

    if decision is final_response:
        evidence = verify_completion(state, decision)
        if evidence passes:
            return recorded_outcome(success, state, evidence)
        state = record_failed_completion(state, evidence)
        continue

    if decision is requested_action:
        verdict = validate_authorize_and_budget(decision, state)
        if verdict denies:
            state = record_observation(state, verdict)
            continue
        request_id = idempotency_key(session_id, turn_id, decision)
        result = execute_with_timeout(decision, request_id)
        state = record_observation(state, normalize(result))

This skeleton can ask the user, request an action, return a final response, or stop. Parser errors, authorization denials, timeouts, and tool failures become typed observations rather than exceptions that silently erase a turn. The termination check covers success, failure, cancellation, and budget exhaustion. The harness may implement retries and resumption differently, but it must preserve these outcomes.

Separate autonomy from authority

Autonomy describes who chooses the next operation. Authority describes which operations the system may perform. They are independent axes. A controller can autonomously search a read-only corpus with little authority, while a fixed workflow can have broad authority to deploy software or transfer funds.

This distinction changes architecture reviews. Adding a planner, longer horizon, or dynamic tool discovery may increase autonomy without changing permissions. Mounting a write-capable credential, widening a filesystem scope, or removing an approval gate increases authority even if the control loop stays fixed. Blast radius follows authority, reachability, and reversibility, not an informal label such as “agentic.” Security and authorization controls are developed in Chapter 56; the architecture must expose the decision points at which those controls apply.

Lower-layer constraint: context is a shared budget

The context window must hold instructions, task data, working state, action schemas, and room for output. Its token accounting is

$$B_{\mathrm{ctx}} = B_{\mathrm{fixed}} + B_{\mathrm{task}} + B_{\mathrm{state},t} + B_{\mathrm{tools},t} + B_{\mathrm{out},t} \le W, \qquad B_{\mathrm{tools},t}=\sum_{a\in A_t}s(a).$$

Here, $B_{\mathrm{ctx}}$ is the total tokens reserved for turn $t$; $B_{\mathrm{fixed}}$ covers system and policy instructions; $B_{\mathrm{task}}$ covers the user goal and fixed task data; $B_{\mathrm{state},t}$ covers selected session state; $B_{\mathrm{tools},t}$ covers the mounted action schemas; $B_{\mathrm{out},t}$ reserves controller output; and $W$ is the model's supported context limit. The set $A_t$ contains actions exposed on turn $t$, and $s(a)$ is the serialized token length of action specification $a$. The equality is accounting, not a claim that all allocations have equal value.

Mounting another action therefore has a certain token cost. Its effect on action selection is empirical, not a universal exponential curve or a fixed tool-count threshold. It depends on the model, task, schema similarity, descriptions, and prompt. BFCL evaluates serial and parallel calls, abstention, and stateful multi-turn behavior (Patil et al. 2025). ACEBench adds ambiguous and incomplete instructions plus multi-turn agent scenarios (Chen et al. 2025). These benchmarks expose several failure modes, but they do not establish one catalog size at which every model fails.

Choose among a flat catalog, phase-based mounting, retrieval, or explicit discovery by measuring the actual workload. Measure selection quality with the needed action present, add irrelevant tools and similar schemas, test abstention when no action applies, and report retrieval recall when schemas are selected dynamically. Tool retrieval can save context while also hiding the only action that would solve the task.

Verify the architecture as a system

An architecture comparison must keep the same model checkpoint, same task distribution, same tool implementations, same permissions, and same budgets. Otherwise a stronger model, broader credential, or larger token allowance can be mistaken for a better loop.

Inspect failures by transition, not only by final score. Determine whether the needed fact never entered context, the controller chose the wrong decision type, the action schema was ambiguous, authorization rejected a valid operation, the environment returned a misleading observation, the state updater dropped it, or the completion checker accepted too early. Each cause belongs to a different contract and requires a different fix.

The architecture now has a precise state boundary, but it has not yet decided what should survive for one turn, one session, or many sessions. That is the subject of Chapter 39.

Further reading