The Harness

The harness is the runtime around the model: the layer that turns model output into tool calls and tool results back into model input, manages the context window, routes tools, sandboxes execution, and exposes the verbs that let a person pause, redirect, fork, or kill a running loop. A loop that looks trivial as a flowchart needs this machinery as soon as it meets production, and the shape of that machinery can move an evaluation score as much as a model swap does.

A loop that refuses to stay simple

The contract is small. Create a session bound to an agent definition. Drive its loop: send context to the model, dispatch the tool calls it returns, append the results, repeat until done or interrupted. Resume after a crash or a wait. Interrupt mid-flight. Fork at a checkpoint. Drawn as a flowchart the loop is a box and an arrow back to itself.

It stops being trivial the moment it meets production. A tool that holds a connection across turns leaves the loop no longer stateless between steps. As the context window fills, something has to decide what to forget. A user hits cancel while a sandbox command is mid-execution, and now the loop must stop work it does not directly control. The sharpest case is two replicas that both think they own the same scheduled tick, where the loop has to coordinate with a copy of itself it cannot see. The harness is where these frictions are absorbed or leaked. It owns mechanism, not policy: it does not decide who may call (identity, Chapter 56), where the model comes from (model access), or which actions require a human (oversight, Chapter 55). It decides how those decisions are carried out, and the most consequential mechanism it owns is the ability to stop the loop.

The loop drawn bare is one box and a back-edge. What Figure 41.2 adds is where each of those production frictions bites the loop, which is the whole argument in one picture: four unrelated-sounding problems land at four distinct points on a structure that looked like it had none.

The six mechanisms that follow absorb those frictions. Each began as a simple version, broke at a specific point under load, and acquired exactly the machinery that point demanded. Read in order, the six are the harness: where an instance comes from, how a cancel reaches the work, who is allowed to run, what the lock actually protects, what survives a full context window, and how the model learns a tool exists. Each carries the lineage that produced it, the knee where its trade-off bites, and the operational hardening that lets the loop survive a long night unattended.

Where an instance comes from

The first decision is when an agent definition, a prompt plus a tool set plus a model binding, becomes a live instance. The obvious starting point is a singleton: one runner shared across every session, isolated only by a session_id, zero allocation per request. The hidden cost is not performance, it is rollout granularity. With a singleton, the prompt is a deploy-time constant, so a bad prompt regresses 100% of sessions until rollback completes. OpenAI's GPT-4o sycophancy episode is the shape of this: the update rolled out April 25, 2025, rollback began April 28, and the full revert took roughly a day (OpenAI 2025). A runtime registry resolves a prompt version per request instead, which lets a regression touch a canary slice while the stable version keeps serving everyone else. The blast radius now equals the canary, not the fleet. The cost is a resolution hop, a cross-replica consistency story, and a new variable in every incident.

The knee is when this stops being optional. A singleton is fine at one or two agent types and starts hurting past three or four, when a prompt regression in one agent forces a rollback that reverts an unrelated fix in another because they shared a deploy. That shared-deploy incident is the moment versioning stops being a nicety, because the cost of not having it has become a regression in code nobody touched.

How a cancel reaches the work

Cancellation is exactly as effective as its most-blind hop. The harness can see an interrupt flag instantly, but the flag has to travel down every layer that might be mid-execution: the model call has to honor its context deadline, the tool dispatch layer has to propagate cancellation into the tool, the sandbox command has to respect SIGTERM, and a stateful protocol call has to respect RPC-level cancellation. A model call that ignores its deadline holds the session for tens of seconds past cancel. A sandbox command that ignores SIGTERM holds it for the full grace period. So "the runtime supports cancellation" is a claim about every hop in the cascade, and a single uncooperative layer makes the cancel cosmetic. This is why interruption is a primitive and not a feature: the pause a human-in-the-loop approval depends on is threaded through every hop, or it does not actually stop anything. The decision about what requires approval belongs to oversight; the pause that lets the approval happen is the runtime's verb, and frameworks that expose it as a first-class interrupt make that pause inspectable rather than implicit (LangChain 2025). Figure 41.3 traces the cascade and the latency each uncooperative hop adds, so the claim that cancel is only as real as its most-blind hop becomes a property of the chain rather than an assertion.

Try setting obeys to False for one hop and watch the time-to-stop jump to that hop's grace period: the cancel is only as fast as its most-blind layer.

# Each hop either obeys the interrupt fast, or ignores it and only
# stops when its own grace period expires. Time-to-stop is the slowest hop.
hops = [
    {"name": "model call",   "obeys": True,  "grace_s": 30.0},
    {"name": "tool dispatch","obeys": True,  "grace_s": 5.0},
    {"name": "sandbox cmd",   "obeys": False, "grace_s": 10.0},  # ignores SIGTERM
    {"name": "RPC call",      "obeys": True,  "grace_s": 8.0},
]
flag_latency = 0.01  # harness sets the interrupt flag almost instantly
stop_times = [flag_latency if h["obeys"] else h["grace_s"] for h in hops]
for h, t in zip(hops, stop_times):
    print(f"{h['name']:14s} stops after {t:6.2f}s")
print(f"\ntime-to-stop = max = {max(stop_times):.2f}s "
      f"(bounded by the most-blind hop)")

The verb has four shapes, and the quality of human control is bounded by which one the harness offers. Hard cancel is kill(session) with no state preservation, simple and brutal, making every correction a context-losing event. Cooperative pause checks an interrupt flag at step boundaries and serializes state. Queued steering appends user input to the next turn without pausing, fast but able to commit a stale decision before it sees the message. Resume-at-point combines steering with a rewind. The pause that an approval depends on is exactly as good as the strongest of these the runtime implements, and no better.

Who is allowed to run

When the loop runs on a schedule, every replica runs its own scheduler, so without coordination every replica fires every tick. The usual fix is an agent-level distributed lock acquired before the run. It prevents two replicas from firing the same tick concurrently. It does not make the tick's side effects exactly-once. If a tick posts to Slack, opens a ticket, then crashes before recording "done," recovery replays it and the Slack message posts twice. The lock coordinates who runs; exactly-once requires coordinating what already happened to the outside world. Idempotency keys at the side-effect call sites close that gap, and they are a different guarantee that a scheduler with only a lock does not have.

A lock prevents overlap, not duplication, and there is a second, sharper way the altitude is wrong. A harness almost always has session-level locking, which stops two Run() calls from mutating one session at once. It says nothing about two different sessions of the same agent touching the same external resource. Two coding sessions push conflicting commits to one branch. A user session and a scheduled tick target the same external state, neither aware of the other. The lock is in the harness; the conflict is at the resource, and the harness lock does not reach it. Figure 41.4 shows the mismatch directly.

Every session in green is individually locked; every shared resource in red is unlocked. The fix lives at the resource: a worktree per session that moves the conflict to a merge, or a resource-level lock keyed on the named resource, or optimistic reconciliation on the external system. The collision is not hypothetical; merge conflicts in coding-agent pull requests are now frequent enough to form a labeled dataset (Ogenrwot and Businge 2026). The right choice depends on whether the downstream system already has its own concurrency control to lean on. Git ref updates and database row locks are real coordination primitives; where they exist the harness can defer to them.

The scheduler itself climbs the same pressure curve. In-process and external-cron schedulers give way to durable execution frameworks (Temporal, Inngest, Restate) once a tick needs multi-step choreography with external side effects, because probabilistic model behavior makes naive retry insufficient, which is exactly the case durable journaling is built for. Whether to keep the session persistent or ephemeral across ticks is a parallel choice with its own knee. A persistent session accumulates context, so a monitoring agent remembers last week's baseline, but events grow without bound, which makes context management mandatory. An ephemeral session is fresh per tick and correct when each tick is independent. Persistent for a stateless job buys unbounded storage for nothing; ephemeral for a job that needs memory throws the memory away every tick.

What survives a full window

A long-running loop hits the context window, and the harness has to answer two questions that do not answer themselves: what gets preserved, and who decides. Automatic compaction summarizes older turns so the session runs past the hard limit, and its failure mode is structural: summarization discards detail and the agent has no signal about what it lost. Anthropic's cookbook measured compaction preserving three of three high-level facts and zero of three obscure specifics (Anthropic 2026). The alternative is externalization, where the agent writes what it wants to keep to a file (progress.md, plan.md) and reloads after compaction. That trades the summary's blindness for the agent's blindness about its own future needs. Neither is mechanical, which is the subject of the contested box below.

"Just use a bigger window" is the tempting third option, and it relocates the failure rather than removing it. A larger context window reduces compaction frequency but does not remove the degradation: accuracy drops as input length grows even with perfect retrieval (Du et al. 2025), and the same rot shows up across realistic retrieval settings (Hong et al. 2025). "Fit everything in" moves the failure from "context overflowed" to "context is full and the model is quietly worse."

How the model learns a tool exists

Dispatch is how a tool runs; the registry is how the model learns the tool exists. A static flat list sends every tool in every system prompt and runs out at roughly 30 to 50 tools, because function-calling accuracy falls as the catalog grows and robustness suffers specifically when irrelevant tools are present (Yan et al. 2025). Beyond that the runtime needs per-turn selection. The security-flavored failure is the sharpest: a malicious tool description at the catalog layer can inject instructions into the model through the system prompt, bypassing the user prompt entirely (Invariant Labs' mcp-injection-experiments). The registry is not a neutral lookup table, which is why catalog admission is a trust decision handed to Chapter 56. What the registry admits to the catalog, it admits to the model's instructions.

Selection evolved as a ladder, each rung raising the catalog ceiling and adding a cost the runtime must then own. The static flat list gave way to dynamic tool-RAG, which retrieves only the tools relevant to each turn (embedding their schemas and fetching the top-k matches), scaling to thousands of tools at the cost of a retrieval layer with its own error modes. Hierarchical namespacing with mount and unmount shows a pruned subset per phase, read-only tools while understanding a problem and write tools while fixing it, which requires the runtime to track phase. Model-driven discovery gives the model a list_available_tools tool and lets it ask, which raises the ceiling but makes discovery quality a function of model capability the runtime does not own. Figure 41.5 lays the four rungs against the scaling ceiling each one buys and the cost it adds.

What a stateful tool drags behind it

Tools fold into the design along a statefulness axis. Stateless tools (a web fetch, a sandbox exec given a sandbox ID) hold no harness-side state and scale trivially. Stateful-connection tools (MCP, gRPC streams) carry negotiated capabilities and cursors in a connection the harness holds for the session, and dispatching them through one mechanism shared with stateless calls mishandles one class or the other. The hardest variant is a single logical operation that mixes both, because its partial-failure space is the product of the two.

Stateful tool transport is the clearest lineage in the chapter. The simplest design holds connections in memory per replica, keyed by (session_id, server_url), opened lazily. It works until the platform scales horizontally, and then statefulness fights the load balancer: the MCP 2026 roadmap names stateful sessions as the primary scaling bottleneck (Model Context Protocol 2026). The concrete failure is the reconnect storm. When a replica holding N sessions, each connected to M servers, dies, recovery is N times M simultaneous reconnections against the surviving infrastructure, each renegotiating capabilities and losing cursor state. The mechanism dictated the mitigation: session affinity to keep a connection on one replica, reconnect backoff with jitter to desynchronize the herd, and resumable streams to make reconnection cheap. The MCP spec followed this curve directly, deprecating HTTP+SSE in favor of Streamable HTTP, where a reconnect replays from a cursor using Mcp-Session-Id plus Last-Event-ID instead of restarting (Model Context Protocol 2025; Model Context Protocol 2025). The spec also hardened a related hazard pooling alone does not touch: when several concurrent sessions for one user race to refresh a single-use OAuth token, the losers present an already-consumed token and fail, so the 2025-06-18 spec revision requires clients to implement RFC 8707 resource indicators to keep a token from being redeemed against the wrong server (Campbell et al. 2020; Model Context Protocol 2025). The 2026 spec work carries the lineage to its endpoint: the 2026-07-28 release candidate removes the protocol-level session entirely, dropping Mcp-Session-Id (SEP-2567) and the initialize handshake (SEP-2575), so any request can hit any replica and a server that needs state across calls mints an explicit handle the model passes back as an ordinary argument (Model Context Protocol 2026). The arc runs from held connections to resumable streams to no protocol session at all.

Multi-agent composition has a parallel arc, with OpenAI's experimental Swarm superseded by the Agents SDK, but composition itself belongs to Chapter 43 and the harness only owns whether the composed agents share or isolate session and sandbox state. That sharing choice is its own trade-off with a measured cost. Shared session and sandbox is cheapest and pays with races and prompt contamination. Full sandbox isolation is the most plumbing and the strongest containment: Geng and Neubig measured worktree isolation at 59.1% against soft instruction-level isolation at 56.1% on Commit0-Lite, and showed soft isolation falling below a single-agent baseline on PaperBench (Geng and Neubig 2026). Instruction-level constraints are advisory; real isolation does not depend on the model's compliance.

Hardening for the long night

The loop itself is small. The machinery around it is what the rest of this chapter has been about, and the last of it concentrates in a handful of operational choices that decide whether the loop survives running unattended.

The lock has two gotchas that come straight from the locking literature. Redis-based locks need fencing tokens to be safe under network partitions (Kleppmann's Redlock critique) (Kleppmann 2016). And with PgBouncer transaction pooling, pg_advisory_lock releases when the connection returns to the pool, so a caller that expects a session-scoped lock must use pg_advisory_xact_lock or a dedicated session pool, or the lock evaporates mid-tick.

A wall-clock timeout on every session, regardless of trigger, is the one limit that holds even when token accounting is wrong or lagging, because wall-clock time is the quantity the harness can measure locally without trusting a downstream meter. The evidence for why it is load-bearing is stark: a Claude Code recursion consumed about 1.67 billion tokens in five hours (anthropics/claude-code 2025), and a practitioner postmortem reports a LangChain pipeline whose agents, talking over agent-to-agent messaging (A2A, Chapter 43), looped for eleven days and produced roughly a $47,000 bill with neither agent carrying a budget ceiling (Waxell 2025).

Admission control belongs at the harness layer. Session creation that calls the Kubernetes API requests a sandbox rather than creating one, and the dangerous failure is not refusal but a hang: the pod sits Pending and the stream to the client never starts, with no error to surface because nothing errored. Checking capacity before promising the client a session turns a slow silent nothing into a fast legible "no" the client can retry against.

The remaining controls follow the same mechanical reasoning. Jitter prevents a thundering herd when many agents share a schedule. A circuit breaker disables an agent after N consecutive failures so it cannot burn tokens indefinitely: in the AWS Cost Explorer outage of December 2025, an internal coding agent deleted and recreated a production environment; AWS attributed the root cause to a misconfigured role, press reporting to the agent acting autonomously, and on either reading the permissions were broad and no breaker tripped (Singh 2025; Schuman 2026). And self-healing at startup replays ticks whose end was never recorded, which is safe only because the idempotency keys above absorb the replay. Sandboxing itself is thin from the harness's seat: it owns worktree isolation for correctness and admission control for capacity, and the sandbox primitive owns the rest, including keeping high-value secrets out of the box, by scoped-token exchange or egress substitution, so generated code cannot read or exfiltrate them (Chapter 56).

The harness matters because it is where the agent loop becomes operable. It buys capability by letting the loop run at full speed across stateful tools and composed agents. The price of that efficiency is context management and admission control, the machinery that keeps a runaway from burning the budget. Trust it earns in one place only: human authority over a running agent is exactly as real as the runtime's ability to pause it, which is a property designed in from the start or done without.

Further reading

Cited sources appear on the book's References page. One item is referenced inline only as an attribution and is listed here for the reader who wants the primary artifact.