What is it?
A language model does not remember your previous request. Each call only knows what is in the text sent with it (What is a Context Window?). So when an assistant seems to remember you, your application is storing information and putting it back into the prompt.
It is helpful to separate kinds of memory:
- Working / short-term memory: the current conversation and intermediate results inside the context window.
- Long-term memory: information persisted outside the model across sessions — user preferences, past decisions, facts learned.
- Procedural memory: instructions and learned ways of doing tasks, often stored as prompts or notes.
- Knowledge memory: documents and reference material, usually accessed through What is RAG (Retrieval-Augmented Generation)?.
These categories borrow words from psychology; they are design labels, not a claim that the model works like a brain.
Why does it matter?
Without memory, every session starts from scratch and long tasks overflow the window. With poor memory, an agent repeats mistakes, forgets constraints, or confidently acts on stale or wrong "remembered" facts. Memory also creates privacy duties: what you store about users must be disclosed, protected, correctable and deletable (AI Privacy and Security: What to Know Before You Paste Data).
How does it work?
Common techniques, usually combined:
- Sliding window: keep the last N messages. Simple; older detail is lost.
- Summarisation: periodically compress older turns into a short summary that stays in context.
- Extracted facts / notes: have the system (or the model via a tool) write durable entries — "prefers metric units", "project deadline is 14 June" — into a store such as a database or file.
- Retrieval: embed memories (What are Embeddings in AI?) and fetch the few most relevant for the current request, rather than loading everything.
- Scratchpads / task state: structured working notes for multi-step work — plan, completed steps, open questions.
Every approach has trade-offs: summaries lose detail and can embed errors; retrieved memories may be irrelevant; stored facts go stale. Good systems let memories carry timestamps and sources, allow correction, and give users visibility and control.
Example
A travel-planning agent keeps three things: (1) the current conversation; (2) a small JSON profile — { "home_airport": "LIS", "dietary": "vegetarian" } — saved after the user confirms it; (3) a task note listing the itinerary decisions so far. On each turn the prompt includes the profile, the task note and only the recent messages. When the user says "use my usual airport", the agent can resolve it. When the user changes their mind, the profile is updated rather than appended to forever.
When should I use it?
Add the lightest memory that meets the need. A one-off Q&A needs none. A multi-turn assistant needs conversation handling. A personal assistant or long-running agent needs persistent facts and a way to retrieve them. Let users view and delete what is stored.
Common mistakes
- Storing everything. Noise crowds out signal and raises cost and risk.
- Treating summaries as ground truth.
- No expiry or correction path for outdated memories.
- Cross-user leakage from badly scoped stores.
- Saving sensitive data without a clear need or consent.
- Confusing memory with RAG. RAG retrieves from a document corpus; agent memory is state about the interaction and user — though the retrieval machinery can be shared.