Summary
Orchestration widened the unit of work from a completion to a task. The model now has to act through loops, tools, memory, retrieval, context assembly, and sometimes other agents. The harness around the model becomes part of the behavior. It governs which state is visible and which evidence is admitted, gates the tools a step is allowed to call, and decides what happens once a step has failed.
The risk grows with that surrounding machinery. An ordinary model error can now turn into a side effect. Memory serves as evidence and as liability at once, and when it outlives the task and becomes the product's memory of its user, it also becomes a switching cost and an attack surface. Retrieval can launder untrusted text into the prompt, an agent operating a screen reads whatever an attacker can render on it, and multi-agent debate, instead of exposing a mistake, tends to copy it. The part's concern is not whether an agent sounds autonomous, but which loop owns the next decision and what can interrupt it.
The takeaway is that orchestration is engineering over decision loops. State, authority, retrieval, interruption, and recovery matter as much as the model call. The open question is which agent patterns remain reliable under production load, adversarial input, tenant boundaries, and changing tool permissions. Part VII supplies that instrument layer: if a system can act through a loop, evaluation has to measure not only the answer but also the evidence, side effects, and changed state left behind.
Comments
Log in to comment