The Tooling Ecosystem

The stack from earlier parts is realized by software others wrote: training frameworks, serving engines, agent frameworks, and the standards that let them interoperate. Each layer of the tooling ecosystem owns a different control point. Once the caller stops being an application and becomes an agent, every layer faces the same shift: the control point has to move back into the layer itself.

The caller that broke the tooling

A capability does not run on a paper. It runs on tooling, and the tooling ecosystem divides into a few layers, each owning one stage of the lifecycle this book has traced. Training frameworks turn FLOPs into weights and own the parallelism and memory of Chapter 10. From those weights, serving engines produce tokens under a latency and throughput budget, the subject of Chapter 31. Agent frameworks go one step further, making a model into something that plans, calls tools, and acts over many steps (Yao et al. 2023), the harness of Chapter 41 and the architectures of Chapter 38. Standards let a tool written once be reached by any of them.

The first generation of this tooling assumed a tidy caller: an application server, long-lived, holding its own credentials, calling a model once per request and logging the result. An agent breaks that assumption. It runs in a sandbox, invokes a CLI, writes a script, spawns a helper, hands work to another agent, and keeps going after the chat window that started it is gone. The important caller is no longer the application. It is a process the agent chose, a background task it spawned, a session that outlives the request.

When the caller moves like that, every control point that sat in the application is now in the wrong place. A provider key in a config file is a key the model itself can read. Compile a permission into the app and no reviewer can see it; leave a plan in the chat history and no one can edit it. These layers need to be treated together rather than buried inside the chapters that use them, because a single change in who calls them reshapes all four at once. The question is what each tooling layer does when the caller becomes an agent.

The one move

The answer, repeated four times, is a single move: pull the control point out of the caller and into the layer the caller cannot bypass. The layer must own its own control point, because the caller can no longer be trusted to hold it (Figure 75.2).

Protocols divide the ecosystem boundary

The new tooling ecosystem is not only a list of frameworks. It is becoming a pair of protocol boundaries. The Model Context Protocol (MCP) standardizes how an AI application or agent reaches external data, tools, prompts, and workflows (Anthropic 2024). MCP originated at Anthropic in November 2024, was adopted by OpenAI, Google, and Microsoft hosts, and since December 2025 is governed neutrally under the Linux Foundation's Agentic AI Foundation (Anthropic 2025), so the boundary no longer belongs to any one vendor. It is the host-to-tool boundary: this agent needs calendar events, a database query, a code search, or a workflow. Agent2Agent standardizes a different boundary: how one agent discovers another agent's capabilities, communicates over an existing enterprise protocol stack, coordinates long-running tasks, and exchanges artifacts across vendors or frameworks (Google Cloud 2025). It is the agent-to-agent boundary, Google-originated and, since June 2025, under Linux Foundation governance as well (The Linux Foundation 2025).

This split is useful because it prevents a false abstraction. A tool is not an agent. A tool exposes a bounded operation; an agent has its own task state, capabilities, and policy surface. Treating every collaborator as a tool hides delegation and accountability. Treating every tool as an agent adds coordination overhead where a simple call would do.

We use the Latere stack as the running case study because its four products map cleanly onto four primitives of the agent-facing tooling layer: model access, the execution sandbox, governance, and orchestration. They are an example of the move, not the only way to make it. The four sections that follow are the same move made on four different control points, and each one pays for its property with a matching cost.

Four primitives, one shape

Model access as a scoped front door

A gateway routes calls to OpenAI, Anthropic, Gemini, OpenRouter, or Ollama, tracks usage, and caps spend. The first generation stopped there. The agent-facing version, exemplified by Latere Lux, adds one rule: the real provider key lives in the gateway and never reaches the calling process. The agent connects to the gateway, not to the provider, so model choice becomes configuration and every call can be attributed to an organisation, person, product, session, and model. Revoking or narrowing the gateway credential stops a runaway task without rotating the provider key. The control point is model access itself, not a key in a file.

The cost is a hop for control. Routing every call through a gateway adds a network hop and makes the gateway a dependency on the critical path. What it buys is that the provider key never reaches the process and spend is attributable per session. For agent traffic, where the caller is untrusted, the hop is worth it; for a single trusted application server it can be pure overhead.

Execution as a durable-or-disposable sandbox

Cloud compute is sold in two shapes that do not fit agent work: a function that vanishes when it returns, and a machine you must babysit. The sandbox primitive, exemplified by Latere Cella, sits between them by separating what you keep from what you throw away. State persists in a named workspace; compute stops when idle and resumes in under half a second, so losing it costs nothing. Secrets stay in a locked store: the workspace sees only an opaque placeholder, and a TLS-terminating egress proxy swaps in the real value on the wire, but only toward the hosts a secret is scoped to, so the real secret never enters the box and a leaked placeholder is worthless (Chapter 56). Network egress runs deny-by-default behind that proxy's allowlist, which travels with the workspace, and every command, secret use, and start or stop is logged against whoever caused it. That last property is what the governance layer above will consume.

The cost is durability versus cost. Keeping a named workspace warm enough to resume in half a second is not free, and the deny-by-default allowlist and secret injection add friction that a throwaway function does not pay. The trade is sound exactly when the work is long, stateful, or touches real secrets, and wasteful for a stateless one-shot that a plain function would serve.

Governance as an agent-is-a-document substrate

Once more than one agent runs, autonomy bought visibility away: an answer and a bill arrive, the steps between them do not. The governance primitive, exemplified by Latere Topos, refuses that trade by making an agent a document: a name, a human owner, the model, the tools, a budget, and the approval steps, all in version-controlled plain text a reviewer can read before and after a run. Every action is an append-only log line with a timestamp and an actor, so a session that fans out across a dozen agents reconstructs as one family tree. Two rules give the document teeth. Permissions can only attenuate as work is delegated: a child agent inherits its parent's permissions minus whatever was withdrawn, and can never grant itself back what it lost. And a human can be placed at any step where a decision matters, so the agent pauses until a sign-off arrives.

The cost is overhead versus auditability. Agent-as-a-document and an append-only log cost write amplification and a review step in the loop. The return is that a compliance reviewer who has never seen the code can read the whole picture, and that no refused request goes unrecorded. The cost is justified once more than one agent runs unattended, and is mostly ceremony for a single supervised one.

Orchestration as a plan you steer

A model that plans and acts over many steps needs a structure between the request and the code. The orchestration primitive, exemplified by Latere Wallfacer, makes that structure an explicit artifact: turn an idea into a plan, the plan into scoped tasks, the tasks into agent execution, and the output into reviewable changes.

The plan, not the chat history, is the thing a human reads, edits, and pushes back on, and the tasks underneath stay within what the plan describes. Each arrow is a point where a person can pause and steer the work (Figure 75.5).

The cost is planning latency versus rework. Inserting a plan between the request and the code is slower to first output than a coding agent that jumps straight to code. It pays back by giving a human one artifact to steer instead of a chat history or a wall of diffs, and by keeping the tasks inside a reviewed scope. For a one-line change the plan is overhead; for multi-step work it is the only thing you can hold.

The four primitives share one shape. Each takes a control point that used to sit inside the caller, a key, a runtime, a permission, a plan, and moves it into a layer the caller cannot bypass. Standards are what keep those layers composable: a tool exposed once through the Model Context Protocol (MCP) (Anthropic 2024), a common protocol for connecting models to tool servers, can be reached by any harness, so the agent framework does not re-implement the tool for every model.

Where the layers came from

Each layer reached its current shape by superseding an earlier one, and the lineage is worth tracing because it shows the same caller-shift driving all of them.

Training frameworks came first and are the most settled. Megatron-LM established intra-layer tensor parallelism, splitting one layer across devices, in native PyTorch (Shoeybi et al. 2019), and ZeRO, the optimizer behind DeepSpeed, removed the memory redundancy of data-parallel training, which replicates the whole model on every device, so model size could scale with device count (Rajbhandari et al. 2019). Their internals belong to Chapter 10; here they matter as the layer whose caller never moved. A training run is still launched by an operator, so this layer felt the agent shift least.

Serving engines moved next. The decisive step was treating the key-value cache, the attention state a model retains for every token already in the session, like operating-system memory: PagedAttention, the algorithm behind vLLM, stores the cache in fixed-size blocks mapped to non-contiguous physical memory, which removed the fragmentation that capped throughput (Kwon et al. 2023). That story is Chapter 31 and Chapter 32. The serving layer's caller is shifting now, from an application that sends one prompt to an agent that holds a long, branching session, which is why session-aware routing and attribution are becoming serving concerns and not only gateway ones.

The agent-facing layers show the shift in the open, because they are new enough that their first generation is still visible. The gateway began as a cost-and-routing tool: one connection address for a messy multi-provider bill. That solved a software problem, and it assumed the key could safely sit in a config file because the caller was an app. Lux is the second generation, built once the caller became an agent and the key could no longer be trusted to the caller. The sandbox evolved from the serverless-versus-VM split toward a middle that keeps state and discards compute. And agent orchestration evolved through a sharper empirical detour, recorded in the goalless-agents experiment behind Wallfacer (Ou 2026): a structured crew of role-specialized agents kept shipping but stopped discovering, polishing trivia inside its lane, while a single goalless agent with full freedom ballooned a project past 60,000 lines and collapsed. The resolution was neither more rules nor fewer, but a deliberate rhythm, roughly 80% improving what exists and 20% exploring, with a human present at each change of direction.

Why the boundaries hold

The boundary between these primitives is itself a design choice. Lux is kept separate from Cella and Wallfacer on purpose: model access is not runtime lifecycle and not task orchestration, so the provider keys, routing, and spend policy live in one place rather than piling up inside the sandbox or the task runner. Clean layer boundaries are what let standards like the Model Context Protocol connect the layers without each one absorbing the others. They also create a new attack surface. Tool descriptors are now part of the prompt-like environment the model reads. MCPTox shows that malicious instructions hidden in MCP tool metadata can cause unsafe tool use across live servers (Wang et al. 2025), and subsequent work frames tool poisoning, shadowing, and descriptor rug pulls as protocol-level risks that require signing, semantic vetting, and runtime guardrails (Jamshidi et al. 2025). Interoperability lowers integration cost, but it also means untrusted tools can arrive through a standard door. The standard does not remove the need for identity, provenance, permissions, and audit; it makes those controls portable.

How the layers compose

The implementation reality is that these layers compose, and the composition is where the value is. An agent under Topos runs inside a Cella workspace, reaches models through Lux, and is owned by a real identity, so a single session links a permission document, a runtime log, a model-usage trail, and a named owner (Figure 75.6). Each layer cites the one below: governance consumes the sandbox's audit log (Cella start/stop and secret-use records), and the gateway's attribution feeds the same session id the governance log keys on.

A reader implementing any one layer should hold two invariants. First, the control point stays in the layer, never in the caller: the key in the gateway, the secret behind the egress proxy, the permission in the document, the scope in the plan. Second, every privileged action is logged against an actor at the moment it happens, because an append-only record is what makes oversight possible after the fact. Capability attenuation is the one rule worth stating in code terms: a delegated context is the parent's minus a withdrawal, never a superset.

child.permissions = parent.permissions - withdrawn
assert child.permissions <= parent.permissions   # can only shrink

Run this to watch both attenuation rules at once: permissions shrink down a delegation chain, and a descendant cannot grant back what a parent withdrew.

parent = {"read_fs", "write_fs", "net", "spend", "spawn"}

def delegate(perms, withdraw):
    return perms - withdraw   # child inherits parent minus a withdrawal

child = delegate(parent, {"spend", "net"})
grandchild = delegate(child, {"write_fs"})

# A descendant tries to grant itself back "spend", already withdrawn upstream.
wanted = grandchild | {"spend"}
granted = wanted & child           # clamp to the parent's set: cannot exceed
print("parent     :", sorted(parent))
print("child      :", sorted(child))
print("grandchild :", sorted(grandchild))
print("re-grant spend ->", sorted(granted), "blocked:", "spend" not in granted)
for ctx in (child, grandchild, granted):
    assert ctx <= parent           # every step can only shrink
print("invariant holds: every delegated context is a subset of its parent")

Outside the case study, the scope here is narrow on the lower layers. Training frameworks and serving engines are named here as ecosystem categories and owned in depth by Chapter 10 and Chapter 31. The depth here is on the agent-facing layers and the standards that bind them, because that is where the caller-shift is live and where the case study has something to show.

The four primitives separate cleanly once they are put in a running stack. Model access and the sandbox are efficiency layers: they make agent compute cheap to attribute and cheap to discard. Governance and the human-in-the-loop handoff are trust layers: they make autonomy reviewable. Orchestration is where capability lives, the rhythm that lets an agent go deep without collapsing. The tooling ecosystem is the place where capability, efficiency, and trust stop being chapter themes and become software a team runs.

Further reading