Memory Systems

The agent loop in Chapter 38 is transient unless its state survives the process that runs it. A model response may schedule work, a tool may change a workspace, and a later turn may need evidence from an earlier one. Calling all of that state memory hides the most important engineering fact: different state has different owners, recovery rules, and authority.

This chapter separates four things that are often collapsed into one store: execution history, the mutable workspace, long-term memory, and external systems. A checkpoint can reconnect them, but it cannot make an external side effect happen exactly once. A retrieval index can help find an old record, but it does not decide whether that record was safe to write, authorized to read, still valid, or complete enough to act on. Reliable memory begins by naming those boundaries.

One word, four owners

At turn $t$, the agent's operational state can be written as

$$S_t = (J_t, W_t, M_t, E_t).$$

Here, $S_t$ is the complete state relevant to turn $t$; $J_t$ is the durable execution history; $W_t$ is the workspace version or snapshot visible to the agent; $M_t$ is the long-term memory store; and $E_t$ is the agent's recorded view of external systems. The last component is only a view. The payment service, ticket tracker, Git remote, or production database remains the system of record for its own state.

An audit transcript is not automatically a replay log. Replay also needs stable step identities, recorded non-deterministic results, compatible workflow code, and durable references to any workspace or artifact the workflow expects. Likewise, a workspace snapshot is not a record of which API calls succeeded. Each store should make its guarantees explicit instead of borrowing guarantees from its neighbors.

Durable execution preserves progress, not exactly-once effects

Consider a step that creates a support ticket. The journal durably records the scheduled call, the provider creates ticket 8421, and the worker crashes before the provider's response reaches the journal. Recovery sees a scheduled step with no completed result. Two histories now look identical from inside the journal:

1. the request never reached the provider; or
2. the request committed, but its reply was lost.

The outcome is unknown. Recording intent before the call narrows the uncertainty to one named step, but it does not close the gap. AWS Lambda durable steps therefore have at-least-once execution semantics by default, and its documentation still requires idempotent business logic for interrupted attempts (Amazon Web Services 2026). Temporal states the same boundary: completed activities are not rerun during replay, but an activity that did not report completion may be retried and execute more than once (Temporal Technologies 2026).

What deterministic replay records

A minimal history for step $i$ contains the following entries, where every symbol is defined below:

$$\operatorname{Scheduled}(i,k,H(x),v), \qquad \operatorname{Completed}(i,r),$$

The next formula represents replay as a deterministic reducer:

$$(\sigma_t, c_t) = F_v(J_{\leq t}).$$

Here, $i$ is a stable step identifier; $k$ is the effect's idempotency key; $x$ is the requested input; $H(x)$ is an input fingerprint; $v$ is the workflow version; $r$ is the accepted result; $J_{\leq t}$ is the history through turn $t$; $F_v$ is the reducer implemented by workflow version $v$; $\sigma_t$ is the reconstructed workflow state; and $c_t$ is the next command or command set.

On replay, $F_v$ must emit commands compatible with the recorded history. A recorded result is substituted instead of calling the model, clock, random-number generator, or external API again. Those non-deterministic operations belong inside durable steps whose accepted outputs enter the history. If workflow code changes its branching or command order, old histories may no longer replay. Long-running systems therefore pin or explicitly migrate the workflow version rather than silently running every old execution under new code (Microsoft 2026; Microsoft 2026).

Durability and effect delivery are separate guarantees:

Unqualified “exactly once” is misleading when the journal and effect owner are independent systems. The practical choices are idempotency, reconciliation, compensation, or an explicit operator decision.

A key is a protocol, not a random string

The following sketch is safe only because the provider accepts the key, binds it to the input fingerprint, and returns the prior result on a retry:

def run_effect(journal, provider, step_id, tenant, request):
    key = journal.stable_key(step_id, tenant, request)  # stable across retries
    if completed := journal.completed(key):
        return completed.result

    fingerprint = sha256(canonical_json(request)).hexdigest()
    journal.schedule(step_id, key, fingerprint)

    known = provider.lookup(tenant=tenant, idempotency_key=key)
    if known is None:
        known = provider.apply(
            request,
            tenant=tenant,
            idempotency_key=key,
            input_fingerprint=fingerprint,
        )

    journal.complete(step_id, key, known.receipt)
    return known.result

The key must be generated and persisted before the first attempt, remain stable across retries, be scoped to the tenant and operation, and be bound to an input fingerprint so that the same key with different input is rejected. Its retention period must exceed the maximum replay and retry lifetime. The provider's lookup and mutation must share the same deduplication record; otherwise the lookup followed by the mutation creates another check-then-act race.

When the receiver has no keyed interface, recovery needs evidence from its own system of record: search for the expected ticket, compare a Git reference, reconcile an account ledger, or ask an operator. Compensation can undo some committed effects, but compensation is a new effect with its own failure modes. A transactional outbox can atomically bind a database mutation to an event destined for another service, although downstream delivery still needs deduplication. Step boundaries make these protocols observable; they do not replace them.

Step granularity also matters. A coarse step that reads a database, calls a service, and writes a file may repeat a partially completed sequence. Finer steps isolate retries and store more intermediate results, but increase history size and coordination. Choose a boundary at the smallest unit whose retry semantics can be stated and tested.

A checkpoint must bind related state

A useful checkpoint is not merely “turn 27.” It is a bundle of compatible identities:

$$C_k = (j_k, w_k, m_k, v_k, p_k).$$

Here, $C_k$ is checkpoint $k$; $j_k$ is the execution-history head; $w_k$ is an immutable workspace snapshot or version; $m_k$ is the memory version or read timestamp; $v_k$ is the workflow version; and $p_k$ is the policy and tool catalog version. Recovery restores this bundle, then reconciles any scheduled step whose outcome is unknown.

Replay, rewind, and fork are different operations:

Immutable histories make a conversation fork cheap because both branches can reference a shared ancestor. The workspace is different: concurrently writable branches need isolated namespaces if their writes must not leak. A full copy, copy-on-write clone, Git worktree, database branch, or remote sandbox can provide that isolation. Non-versioned external state is not automatically forked.

Two branches do have a merge base: their shared checkpoint. Files can use a three-way merge when the storage model preserves that ancestry. The harder problem is semantic merge: deciding which divergent plans, approvals, tool results, and external effects should survive. That is a product policy and sometimes a human decision, not an absence of shared history. LangGraph's documented time-travel surface illustrates the distinction: replay re-executes work after a checkpoint, whereas a fork creates a new checkpoint branch without modifying the original history (LangChain 2026).

Workspace durability is a stack of guarantees

The container filesystem, a persistent volume, a snapshot, and a backup answer different questions. A container filesystem normally follows the container or Pod lifetime. A Kubernetes persistent volume exists beyond an individual Pod, while an ephemeral volume follows the Pod (Kubernetes Authors 2026). Neither fact says where the volume can attach, whether two nodes may write it, what happens when its claim is deleted, or whether it can be restored after corruption.

Those properties must be checked separately:

ReadWriteOnce is an access mode, not a statement that every volume is a zonal block device. Topology depends on the storage backend and its provisioning policy. Likewise, deleting a Pod does not normally delete an ordinary persistent volume; generic ephemeral claims and explicit retention or reclaim policies behave differently. A design review should name the actual CSI driver, StorageClass, topology constraints, retention policy, and restore path rather than reasoning from the PVC label alone.

Snapshot cadence gives an RPO bound, not a backup

Suppose instantaneous snapshots occur every $\Delta$ minutes and a crash is equally likely at any phase between them. If $L$ is the amount of work since the latest snapshot, then

$$L \sim \operatorname{Uniform}(0,\Delta), \qquad \mathbb{E}[L] = \frac{\Delta}{2}, \qquad L_{\max} < \Delta, \qquad f = \frac{60}{\Delta}.$$

Here, $\Delta$ is the snapshot interval in minutes; $L$ is recoverable work loss; $\mathbb{E}[L]$ is expected loss under the uniform-crash assumption; $L_{\max}$ is the worst loss before the next snapshot; and $f$ is the number of snapshot requests per hour. The interval is therefore a simple recovery point objective (RPO) bound. It says nothing about bytes written, snapshot duration, deduplication, retention, recovery time objective (RTO), or whether a restore succeeds.

Snapshots also need a consistency contract. An application-consistent snapshot may need to pause writers, flush a journal, or record a database position. Kubernetes VolumeSnapshot provides a standard API only when the CSI driver and storage backend support it (Kubernetes Authors 2024). A restore drill must verify the workspace, the execution head, and any database or memory version together; a green “snapshot created” event is not proof of recoverability.

Cloning cost also depends on the storage backend. Kubernetes supports PVC cloning through compatible CSI drivers, but the API does not promise a particular copy algorithm or latency (Kubernetes Authors 2023). Btrfs snapshots initially share a root and isolate later changes through copy-on-write (Btrfs Maintainers 2026). File-level overlay systems, block-level copy-on-write filesystems, content-addressed manifests, and full copies have different copy-up and retention costs; “copy-on-write” alone is not a complete performance model.

MicroVM snapshots illustrate why inclusion lists matter. Firecracker serializes guest memory and emulated hardware state, while disk files remain separately managed; network continuity is not guaranteed and restore compatibility depends on the host and snapshot version (Firecracker Maintainers 2026). Cloned VM state can also duplicate random generator state and identifiers, so a restore path may need to rekey secrets, regenerate identities, and reconnect services (Brooker et al. 2021). A snapshot is only as complete as the state it explicitly includes.

Long-term memory is a governed write–manage–read loop

Execution history answers what the runtime accepted. Long-term memory answers what selected information should influence later decisions. The selection is the point: raw history may be an input to memory, but retaining every event is logging, not a memory policy.

Two influential systems illustrate different mechanisms. Generative Agents stored a natural-language stream of experiences, synthesized higher-level reflections, and retrieved records for later planning (Park et al. 2023). MemGPT treated a bounded context window and external storage as tiers managed through explicit movement between them (Packer et al. 2023). Neither establishes one mandatory store. Logs, tables, files, summaries, graphs, lexical indexes, vector indexes, and hybrids can all implement parts of the loop.

The familiar labels episodic, semantic, and procedural are useful analogies for events, facts, and learned ways of acting, but they are not mutually exclusive database classes (Squire 2004). An engineering design is clearer when it keeps orthogonal choices separate:

Retrieval-augmented generation (RAG) is a retrieval pattern, not an ownership class. A RAG corpus can be private and mutable; a memory store can be shared and read-only. The useful distinction is lifecycle: memory is normally updated as the agent interacts, while an external corpus may have a separate ingestion owner. Chapter 44 (Chapter 44) treats retrieval itself in depth.

Records need provenance and time

A durable memory record should carry more than text or an embedding:

MemoryRecord {
  id, scope, subject, kind, payload,
  source, derivation, recorded_at, valid_from, valid_until,
  version, confidence, sensitivity, purpose, retention_until,
  status, supersedes
}

scope names the authorized audience, while subject names the person or entity the record concerns; those are not the same. source links to the observation or tool result, and derivation explains how a summary or inference was produced. recorded_at is system time; valid_from and valid_until describe when the claim is true. status distinguishes active, superseded, quarantined, and tombstoned records. supersedes links a correction to the version it replaces. The remaining fields make retention, sensitivity, purpose, and confidence testable policy inputs.

The complete loop is:

1. Propose a write. Extract a candidate from an observation or outcome.
2. Authorize and classify. Establish writer, scope, subject, purpose, provenance, sensitivity, and validity.
3. Manage. Deduplicate, detect contradictions, insert a new version, supersede stale records, quarantine suspicious content, or decline the write.
4. Retrieve. Restrict the candidate set by trusted identity and purpose, then search and rerank within a context budget.
5. Use as evidence. Preserve attribution and keep retrieved content below trusted instructions; memory never grants authority by itself.
6. Revise or delete. Apply corrections, expiry, retention, and deletion to the source and every derived representation.

Authorization must constrain retrieval

For query $q_t$, the following formula defines the authorized candidate set before retrieved records reach the model:

$$\mathcal{C}_t = \{m \in M_t : \operatorname{allow}(a_t,p_t,m) \land \operatorname{live}(m,t) \land \operatorname{status}(m)=\text{active}\},$$

then retrieve within a budget:

$$R_t = \operatorname{TopBudget} \left(\operatorname{rerank}(q_t, \operatorname{retrieve}(q_t,\mathcal{C}_t)), B_t\right).$$

Here, $M_t$ is memory at time $t$; $m$ is one record; $a_t$ is the authenticated principal; $p_t$ is the authorized purpose; allow checks scope and sensitivity; live checks validity and retention; $\mathcal{C}_t$ is the allowed candidate set; $q_t$ is the query; $B_t$ is the token or byte budget; and $R_t$ is the final retrieved set. retrieve may be lexical, vector, graph, keyed, or hybrid search; rerank orders candidates; and TopBudget admits records until the budget is reached.

This invariant can be enforced by a trusted retrieval service, physical partitioning, database policies, or defense in depth across them. A user-supplied tenant label is not an authenticated principal. PostgreSQL row-level security, for example, can restrict normal reads and writes, but table owners, superusers, and roles with BYPASSRLS require deliberate treatment and tests (PostgreSQL Global Development Group 2026). The security property is zero unauthorized candidates, not merely a high relevance score.

Persistent memory also extends the lifetime of poisoned content. A malicious or mistaken statement can be extracted, summarized, and retrieved in later sessions; AgentPoison demonstrates this class of attack against agent memory and knowledge bases (Chen et al. 2024). Store writer and source, distinguish quoted evidence from derived inference, quarantine suspicious writes, and never interpret retrieved records as privileged instructions. Session logs and RAG corpora can carry the same attack; memory makes later reuse part of the default path.

Deletion must follow lineage. Removing a source row while leaving its embedding, summary, graph edge, cache entry, or export does not remove the information from the online system. A deletion manifest should enumerate derivatives, record completion, prevent a backup restore from resurrecting tombstoned records, and document when backups age out. The precise legal obligation depends on jurisdiction and purpose; the engineering contract is discoverability, attributable lineage, and verifiable deletion within the stated policy.

Evaluate the stages, not only the final answer

Memory benchmarks stress different abilities. LoCoMo uses long, multi-session conversations for question answering, event summarization, and multimodal dialogue generation (Maharana et al. 2024). LongMemEval targets information extraction, multi-session and temporal reasoning, knowledge updates, and abstention, and analyzes indexing, retrieval, and reading separately (Wu et al. 2025). MemoryAgentBench adds incremental interaction and tests accurate retrieval, test-time learning, long-range understanding, and selective forgetting (Hu et al. 2026). None of them proves tenancy isolation, poisoning resistance, or deletion completeness.

An evaluation should localize failure:

Compare systems with the same model checkpoint, prompt and policy versions, task split, write budget, retrieval budget, tool access, judge and rubric, hardware and cache state, and repeated-run policy. Include no-memory, full-history, lexical, vector, and hybrid baselines when they fit the task. Otherwise a better model, larger context budget, or looser authorization boundary can masquerade as a better memory system.

The operational contract

Before shipping persistent agent state, write down the following contract:

This contract keeps runtime recovery separate from remembered knowledge while making their coupling explicit. The next chapter narrows the memory problem to a particular owner: persistent user state used for personalization across sessions.

Further reading