Long-term memory

What is long-term memory?

Long-term memory is the persistent store of information an AI agent keeps after a conversation ends, including customer facts, stated preferences, prior issues, and resolutions. It survives session boundaries, so the agent can recognize a returning customer and reuse what it already established about them.

In support, this is the difference between an agent that asks for an order number every single time and one that opens with the shipment already in view. Most production systems hold this in a database or vector index outside the model, written during or after a conversation.

How long-term memory works

Long-term memory runs as a four-stage loop: extract, write, retrieve, inject. Extraction happens during or just after a conversation, when a summarization pass pulls durable facts out of a transcript: the customer's plan, their timezone, the fact that a refund was already issued in March. Most turns produce nothing worth keeping, and the extraction step's job is deciding that.

Writing stores each fact with an identity key, a timestamp, and a source pointer back to the conversation it came from. Retrieval, at the start of the next session, queries that store using the customer identifier plus the incoming message, usually through embedding similarity so a question about "my subscription" surfaces plan details without matching on the exact word. This retrieval layer is the same machinery that powers semantic memory for general product knowledge.

Injection places the retrieved records into the model's prompt, where they join short-term memory for the current turn. The whole loop is one component of broader AI agent memory, and its usefulness depends far more on retrieval precision than on how much has been stored.

Types of long-term memory

• Semantic: Durable facts and concepts held independently of when they were learned, such as a customer's account tier, entitlements, or preferred contact channel.

• Episodic: Records of specific past interactions with their sequence and timestamps intact, so the agent can say what happened in the March ticket and when.

• Procedural: Learned or configured routines for how a task gets done, such as the escalation path this account requires before a refund is approved.

• Profile or entity memory: A structured summary keyed to a person, account, or organization, usually rewritten in place as new information arrives, which keeps it compact but loses history.

• Shared or organizational: Facts written by one agent and read by another, useful across a team, and the type most likely to leak permissions if scoping is loose.

Long-term memory vs short-term memory vs the context window vs RAG

These four are routinely treated as one thing, and the conflation is what produces agents that appear to forget mid-sentence. Short-term memory holds the current conversation and disappears when the session closes. The context window is the fixed token budget the model can read on any single turn, a hard capacity limit that no memory design removes. Retrieval-augmented generation fetches documents from a knowledge corpus that describes the product, not the customer. Long-term memory is the only one of the four that persists customer-specific state across sessions and deliberately writes new records back.

If your agent already knows the policy but keeps re-asking who it is talking to, you need long-term memory. If it gives correct but generic answers, the gap is retrieval over content. Those are separate builds with separate owners, and teams routinely fund the wrong one.

Why long-term memory matters for customer experience

Without persistence, every conversation restarts from zero. The customer re-explains the broken device, re-supplies the order number, and repeats context they gave twenty minutes ago on another channel. That repetition is one of the most reliable drivers of low satisfaction scores in support, because it signals that nobody on the other side is keeping track.

Continuity also changes what the agent can do. Knowing that a refund was already issued last month lets the agent handle the follow-up as a second contact about one problem, which is a materially different conversation.

The tradeoff is real: a system that remembers can remember incorrectly, and a wrong stored fact is more damaging than an empty profile because the agent asserts it with confidence and the customer has no way to correct the record.

How is long-term memory measured?

Start from what your own logs can show you, because there is no published figure for long-term memory that a support team is graded against. Academic benchmarks for long-context and long-horizon dialogue are built on their own task taxonomies and dialogue sets, so their scores do not transfer to your customer records.

Measure it internally along three axes. Recall coverage: given a set of questions whose answers were established in earlier sessions, how often does the right record come back. Write precision: sampled from stored records, what share are still accurate and correctly attributed. Staleness: the age distribution of records that were retrieved and used. Position matters as much as presence, since Liu et al. showed that models attend unevenly to material depending on where it sits in a long input, so a fact retrieved but buried can still be ignored.

How AI agents change long-term memory

Classic support systems already stored customer history in CRM fields and ticket threads, but reading it required a human to open the record and skim. AI agents change the mechanism: retrieval is automatic, semantic, and happens before every reply, so stored history now influences the answer whether or not anyone reviewed it.

That shifts memory from a reporting artifact to an execution dependency. When an agent has tool access, a remembered fact can trigger an action: an agentic AI system that recalls a prior failed delivery may reship without asking. This is why memory writes deserve the same scrutiny as any other write path, and why retention and redaction choices belong in the same review, a question worked through in this guide to anonymizing customer data for training and compliance.

Multi-agent setups raise the stakes again. When agents hand off work under something like the A2A protocol, one agent's stored assumption becomes another agent's premise, and errors compound quietly across the chain.

What to look for in long-term memory

Start with the write path: can you inspect what was extracted, edit it, and delete a single record without dropping the whole profile. Systems that only expose an on/off switch make correction impossible.

Check the identity model. Memory keyed to an email address behaves differently from memory keyed to an account, and B2B support usually needs both, with clear rules for what a teammate may see.

Governance is the axis buyers underweight. You need retention windows, per-field deletion to satisfy GDPR erasure requests, and audit logs showing which record influenced which reply. In regulated settings, expect SOC 2 Type II and ISO 27001 as table stakes, ISO 42001 where AI governance is under review, and HIPAA with a BAA wherever health data enters a transcript.

The operational constraint is review capacity. Automatic extraction generates records faster than any team can audit them, so sampling and confidence thresholds are part of the design.

Long-term memory and conversation history

Long-term memory is derived from conversation history but is not the same asset. Raw transcripts are complete and nearly unusable at retrieval time, while episodic memory compresses each interaction into a recallable event with its context preserved.

The relationship matters for human-facing tooling too. An agent assist sidebar that surfaces three stored facts about a caller is more useful than one that surfaces four prior transcripts, because the human has seconds, and compression is what makes the history readable in that window.

What does long-term memory mean in plain terms?

Think of long-term memory as the notebook a good account manager keeps: a few lines per customer, updated after each call, reread before the next one. The notebook is small on purpose, because its value comes from being scannable in ten seconds.

Without it, the customer who explained their setup in detail on Monday explains it again on Thursday to the same system, and reasonably concludes nobody was listening the first time.

The tradeoff is that notebooks go stale. A note saying "prefers email" written two years ago will steer a reply today unless something forces a review. Whether that matters depends on how consequential the remembered fact is, which is why teams usually give preferences a long life and account state a short one, as discussed in this comparison of AI agents and traditional chatbots.

Common long-term memory mistakes

Storing transcripts and calling it memory is the most common failure. Full conversation logs retrieve poorly because similarity search returns whole documents dense with irrelevant turns, and the useful sentence gets buried in the middle of a long input where the model attends to it least.

The second is writing without a correction path. If extraction misreads a customer's statement as a durable fact, that error persists silently and gets restated with confidence in every future session until someone notices the pattern from the outside.

The third is ignoring the identity boundary. Memory scoped too broadly leaks one user's details into another user's conversation, which in a shared workspace is a privacy incident rather than a quality bug.

The fourth is retrieving everything available. Injecting a large profile into every prompt consumes the context budget and dilutes the current question, so answers get worse as the stored profile grows. Retrieval limits are a design decision that has to be made explicitly.