Personalization Memory

Chapter 39 established the general memory layer: scoped records, explicit write and read policies, temporal validity, provenance, and deletion lineage. Personalization applies that layer to claims about a person. Its practical value is simple: help users avoid repeating durable constraints and preferences. The danger is equally simple: a system may remember the wrong thing, apply it in the wrong situation, or retain more of a person's life than the service needs.

Good personalization is therefore purpose-bound and defeasible. It retains only what helps a declared purpose, and the user's current request remains the strongest evidence of what they want now. Not every remembered detail is useful. A preference that improves a restaurant suggestion may be irrelevant to a tax question, and a fact collected for one workspace may be unauthorized in another.

Memory is evidence, not authority

A personalized response at turn $t$ can be represented as

$$y_t \sim p_\theta\!\left(y \mid P, x_t, h_t, R(q_t, M_u, a_t, p_t)\right).$$

Here, $y_t$ is the response; $p_\theta$ is the model with parameters $\theta$; $P$ is the governing policy; $x_t$ is the current request; $h_t$ is the current session context; $M_u$ is the set of records associated with user $u$; $a_t$ is the authenticated actor; $p_t$ is the authorized purpose; $q_t$ is the retrieval query; and $R$ returns the live, authorized records applicable to that request.

This ordering matters. Retrieved memory is untrusted evidence, not authority. It cannot authenticate the actor, expand permissions, override safety policy, or defeat a clear current instruction. A system should apply a remembered claim only after checking its subject, scope, purpose, validity, sensitivity, and provenance.

Where state persists is not how it reaches the model

Personalization designs are easier to reason about when two independent axes stay separate. The persistence location says where durable state lives. The application path says how that state affects inference. A server-side store can feed prompt context or a tool call; a parameter adapter can be selected by a router; and a client-side store can disclose only the records needed for one request.

Per-user adapters complicate batching and operations, but they do not make heterogeneous batching impossible. Systems such as S-LoRA demonstrate concurrent serving of many LoRA adapters (Sheng et al. 2023). Conversely, an external record is not “in the context” merely because it exists in a database. It affects a response only when an authorized application path reads and supplies it.

A user memory is a scoped claim

A useful record distinguishes what the user explicitly said from what the system observed or inferred. Treating all three as facts gives weak guesses the authority of direct instructions. A concrete schema makes that distinction reviewable:

UserMemory {
  id, subject, origin, claim, source_ref,
  scope, purpose, observed_at, valid_from, valid_until,
  confidence, sensitivity, confirmation, status, supersedes
}

subject identifies the person the claim concerns, not merely the signed-in account. origin distinguishes an explicit instruction, a user-provided fact, observed behavior, an inferred profile, and an imported or source-derived claim. source_ref preserves provenance. scope and purpose bound where and why the claim may be used. The time fields express validity rather than assuming permanence. confidence records uncertainty; sensitivity controls stricter handling; confirmation records whether the user verified the claim; and status plus supersedes preserve correction history without silently rewriting evidence.

The write policy should also reject unnecessary sensitive inferences. A model can infer private attributes from ordinary text more readily than keyword filters imply (Staab et al. 2024). Data minimization must therefore cover derived claims as well as verbatim text.

The lifecycle needs a write gate and an applicability gate

Personalization is not “extract, embed, retrieve.” It is a controlled lifecycle:

1. Create a candidate claim with the exact supporting source.
2. Classify its origin, subject, purpose, scope, sensitivity, confidence, and validity.
3. Pass a write gate that may reject, shorten, redact, confirm, or retain the candidate only for the session.
4. Append the accepted record. If it corrects an earlier claim, link it with supersedes rather than overwriting history.
5. Before retrieval ranking, filter by actor, subject, scope, purpose, status, validity, and policy.
6. Supply selected records as attributed evidence and record which ones influenced the response.
7. Accept correction, deletion, export, expiry, and feedback through the same lineage-aware state model.

Relevance ranking belongs after applicability filtering. Otherwise a highly similar record from the wrong account, project, or purpose can outrank an authorized one.

Resolve conflicts without inventing a new biography

The current request overrides a remembered preference for the current action. It does not necessarily erase the standing preference. Conflict handling should be explicit:

This distinction prevents two common errors: treating one exception as a permanent change, and converting an unobserved plan into a completed life event.

User controls are state transitions, not labels

A control is useful only when its semantics are precise. “Off,” “pause,” “reset,” and “delete” can refer to different data flows in different products. Document each transition and test it end to end.

Current product documentation illustrates why these distinctions matter. OpenAI's memory FAQ explains that removing information can require deleting both saved memory and source chats, and that some deletion logs may be retained temporarily for safety and debugging (OpenAI 2026). Anthropic distinguishes pausing memory from resetting it, offers incognito chats, and documents that deleting a source does not necessarily remove a derived memory entry (Anthropic 2026). These are examples of product contracts, not universal definitions. Implementations must expose their own exact semantics.

Privacy obligations shape the architecture

For systems subject to the EU General Data Protection Regulation, purpose limitation, data minimization, accuracy, and storage limitation are design inputs, not cleanup tasks (European Union 2016). Correction and erasure rights are conditional, as is data portability; the appropriate legal basis depends on the jurisdiction, controller role, purpose, and data category. Consent is not the only possible legal basis and should not be claimed where it is not freely given or valid.

Portability is narrower than “export everything the model inferred.” EDPB guidance explains that the statutory right applies under stated conditions to data the person provided, including some observed data, but generally not to profiles inferred by the controller (European Data Protection Board 2026). A product can still voluntarily expose inferred claims because visibility and correction improve safety. Exports should use a structured, commonly used, machine-readable format such as JSON, XML, or CSV rather than treating a PDF as sufficient portability.

Deletion is a graph operation. One source may produce a normalized claim, a profile summary, an embedding, a search-index entry, a cache value, a replica, and an audit reference. The service needs lineage from every derivative to its sources. A safe workflow first blocks further use, then performs and verifies physical deletion across governed stores according to retention and backup policy. It must also stop later rebuilds from resurrecting a deleted claim. Legal holds and mandatory records need a documented exception path, not a silent failure of the delete control.

Personalization adds distinct security and integrity risks

Memory enlarges the impact of several familiar failures:

• Sensitive inference: an apparently harmless conversation can support a claim about health, location, income, beliefs, or relationships that the user never chose to store (Staab et al. 2024).
• Prompt injection and poison: content in a chat, document, connector, or knowledge base can attempt to plant instructions that survive into later tasks. AgentPoison demonstrates targeted poisoning of agent memory or knowledge bases under its evaluated threat model (Chen et al. 2024). Imported memories must therefore remain untrusted evidence with source attribution.
• Cross-account or cross-project leakage: a missing subject, tenant, or purpose check can retrieve a relevant-looking record belonging to someone else.
• Sycophancy: remembering a user's views can make agreement easier than honest correction. OpenAI reported that memory exacerbated sycophancy in some cases in a particular model update, while explicitly not claiming that memory generally causes it (OpenAI 2025).
• Personalized falsehood: a model may align a factual answer with a user's history instead of the evidence (Sun et al. 2026). Personalize choices, examples, tone, and presentation; do not personalize objective truth.

Security tests should include poisoned sources, malicious imported profiles, account switching, shared-device sessions, stale authorization, and deletion followed by index rebuild. Encryption helps protect storage and transport, but does not decide whether a record should exist or whether a read is authorized.

Evaluate the whole lifecycle

Personalization quality is not recall accuracy alone. LaMP introduced seven personalized language-modeling tasks and retrieval-augmented profiles (Salemi et al. 2024). PrefEval shows that preference following can deteriorate as context grows and distinguishes explicit from implicit preferences (Zhao et al. 2025). Both are useful controlled evidence, but neither substitutes for product-specific safety, control, and longitudinal evaluation.

At minimum, compare a no-memory baseline with explicit-profile-only, retrieve-from-history, and synthesized-profile variants. Add full-history or adapter variants only when they match the intended architecture. Use the same model, task set, safety policy, and latency accounting so the comparison isolates memory.

“Personalization lift” should mean improvement over the matched no-memory baseline, not the absolute score of the personalized system. The wrong-personalization rate counts cases where memory makes a response less suitable, including stale, mis-scoped, overconfident, or fact-distorting uses. Report both; an average benefit can hide a smaller set of severe harms.

Synthetic users and model judges also miss failures that real people notice. A study of real-user personalization evaluation found nuanced error categories absent from its synthetic-user and LLM-judge conditions (Balepur et al. 2026). Run longitudinal tests with consenting participants, support inspection and correction, and stratify results by record origin, sensitivity, age, and scope. Vendor-reported benchmarks can motivate a hypothesis, but they are not independent validation of a product's claims.

Lower-layer constraints

The memory service cannot supply guarantees absent from the layers below it. Identity and authorization determine whose records may be read; storage and index isolation determine whether scopes actually remain separate; durable event history determines whether corrections and deletions propagate; and backup policy determines when physical erasure completes. The inference layer cannot repair a missing tenant check, and a friendly interface cannot make an untracked embedding deletable. Personalization claims should therefore be no stronger than the weakest identity, lineage, isolation, and retention guarantee on which they depend.

What remains contested

Two questions lack a general answer. First, repeated behavior does not establish whether a person has a stable preference, was adapting to a temporary constraint, or merely accepted a default. The right confirmation threshold depends on the cost of a wrong inference and cannot be derived from retrieval accuracy alone. Second, products disagree about the state transitions behind words such as pause, temporary, and delete. A common vocabulary would make controls easier to compare, but different storage and legal boundaries may require different implementations. Until stronger evidence and standards emerge, systems should expose the underlying state changes and report results by origin, sensitivity, and control mode rather than claiming a universal memory policy.

An operational contract

A production personalization service should be able to answer these questions for every applied claim:

1. Who is the subject, and which authenticated actor may use the claim?
2. What source supports it, and was it explicit, observed, inferred, or imported?
3. For which declared purpose and scope is it valid?
4. What makes it current, and what would supersede or expire it?
5. Why was it selected for this request, and how did it affect the output?
6. How can the user inspect, correct, pause, delete, and export it?
7. Which derivatives and replicas must follow a correction or deletion?
8. Which tests show benefit, wrong personalization, isolation, control fidelity, and operational cost?

If the system cannot answer those questions, it has accumulated user data, not a reliable personalization memory. Chapter 41 turns these contracts into repeatable evaluations and release gates.

Further reading