Memory for AI agents: what to store, what to forget

Language models are stateless: every request starts from nothing except what you send. Agents and assistants that seem to remember you are built on top of a memory system that decides what to save from each interaction, what to retrieve next time, and what to update or forget.

Getting memory right makes an assistant feel genuinely helpful: it remembers your preferences, avoids asking the same questions twice and builds on past work. Getting it wrong makes it feel either forgetful or invasive, and can create real privacy and security problems. This guide covers the types of agent memory, what to store, how to retrieve and update it, when to forget, and the privacy controls users need.

Why the context window is not enough

A large context window lets a model read a lot at once, but it does not persist across sessions, has no notion of importance, and gets more expensive and less reliable as it fills up. Memory is a separate system that selects what matters and re-inserts it when relevant. We cover the distinction in detail in context windows are not memory.

Four types of agent memory

Cognitive science distinctions are a useful frame, and research such as Sumers et al. (2023), "Cognitive Architectures for Language Agents" (CoALA), applies them to LLM agents.

Most practical systems combine a structured store for semantic facts, a searchable store for episodes, and procedural knowledge kept in prompts or configuration.

What to store

Store information that is useful later, stable enough to rely on, and appropriate to keep.

Good candidates:

• explicit preferences ("always send me summaries in French");
• stable facts about the user's context: role, team, time zone, tools they use;
• decisions and their reasons ("chose vendor B because of data residency");
• ongoing projects and their status;
• corrections the user made to the agent's behaviour.

Usually not worth storing:

• one-off details with no future use;
• transient emotions or offhand remarks;
• information the system can look up reliably elsewhere;
• anything sensitive unless the user explicitly wants it remembered and the product needs it.

A simple test: would a thoughtful human assistant write this down in their notes about the person? If it would feel odd or intrusive for a human to note it, do not store it.

How to write memories

• Extract, don't dump. After a conversation, have a model extract candidate memories as short, atomic statements, rather than storing raw transcripts.
• Attach metadata: source (which conversation), timestamp, confidence, and category.
• Prefer explicit over inferred. "User said they are vegetarian" is more reliable than "user ordered a salad twice." Mark inferred memories as such, or ask to confirm.
• Let users trigger memory directly: "Remember that I'm out on Fridays."

{
  "id": "mem_2391",
  "user_id": "u_118",
  "statement": "Prefers meeting summaries in French",
  "type": "preference",
  "source": "conversation 2026-08-02, user stated explicitly",
  "confidence": "explicit",
  "created": "2026-08-02",
  "last_confirmed": "2026-08-02"
}

How to retrieve memories

Retrieve selectively, based on the current task, not all at once.

• Always-on profile: a small set of core facts and preferences included in every request (a few hundred tokens at most).
• Task-relevant retrieval: search episodic and semantic memory by similarity to the current request, and include only the top matches.
• Recency and importance: weight recent and frequently confirmed memories higher.

Retrieval quality matters as much as storage. The same techniques used for document retrieval apply; see RAG chunking strategies compared.

Updating and resolving conflicts

Facts change. The user moves, changes jobs or reverses a preference. Without update logic, memory accumulates contradictions and the agent uses outdated facts.

• Check for conflicts on write: when a new memory contradicts an existing one, update or replace it rather than adding a second version.
• Keep history where it matters, marking old facts as superseded instead of deleting them silently.
• Prefer the most recent explicit statement when conflicts remain.
• Ask when unsure: "Last time you mentioned you were in Lisbon. Is that still right?"

Forgetting

Forgetting is a feature, not a failure.

• Expire time-bound memories: travel plans, temporary projects, short-term preferences.
• Decay unused memories: lower the weight of memories never retrieved or confirmed.
• Delete on request, completely, including derived summaries and embeddings.
• Set retention limits consistent with your privacy policy and regulations such as the GDPR's storage-limitation principle and right to erasure.

Privacy and trust

Memory is where AI products most often feel creepy. Design for user control from the start:

• Transparency: let users see what is remembered about them, in plain language.
• Control: let them edit or delete individual memories and turn memory off.
• Notification: briefly indicate when something new is saved ("I'll remember that you prefer French summaries").
• Scope: keep memories separate between users and between workspaces; never let one user's memory leak into another's context.
• Sensitive categories: avoid storing health, financial, or similarly sensitive information unless essential and consented.
• Security: memory stores can be targets for prompt injection, where malicious content plants false "memories." Validate what gets written, especially from tool outputs or external content.

Asking before saving sensitive or inferred memories is a natural fit for human-in-the-loop patterns.

Shared and organisational memory

Not all memory belongs to one user. Team assistants and enterprise agents often need shared memory: project decisions, team conventions, customer histories. Shared memory adds governance questions:

• Who can write? Limit writes to trusted sources or require review, since a wrong shared memory misleads everyone.
• Who can read? Respect existing permissions; an agent must not reveal a confidential project's notes to someone outside it.
• How is it corrected? Provide an owner and an easy way to fix or retire outdated team knowledge.

Treat shared memory more like a knowledge base with editorial ownership than like personal notes.

Testing memory

Memory bugs are hard to spot in single-session tests. Build multi-session evals: scenarios where a fact is stated in session one, changed in session two and needed in session three. Check that the agent uses the current fact, does not surface deleted memories, and does not leak memories between users. See evaluating agents end to end for broader agent evaluation.

Frequently asked questions

How do AI agents remember things between sessions?

The application stores selected information, such as preferences, facts and summaries of past interactions, in a database, then retrieves relevant items and adds them to the model's context in later sessions. The model itself does not retain anything between requests.

What should an AI assistant remember about users?

Explicit preferences, stable facts about their context, decisions and ongoing work that will be useful later. Avoid one-off details, offhand remarks and sensitive information unless the user wants it remembered and the product needs it.

How do I handle outdated memories?

Check for conflicts when writing new memories, replace or mark superseded facts, expire time-bound information, and confirm with the user when a stored fact may have changed.

How can users control what an AI remembers?

Provide a view of stored memories, the ability to edit or delete individual items, an option to turn memory off, and notifications when new memories are saved. Deletion should remove derived data such as embeddings and summaries too.