Agent memory: one word, several mechanisms

Published 28 July 2026 · updated 16 August 2026

A stateless API call starts with nothing: it doesn’t know who you are, what your company does, what you decided last month. Some chat products already add history, summaries or memory, so you have to name the product and how it’s set up. The missing continuity isn’t a flaw in the model. It’s a choice someone made in the architecture. The subject turned noisy in 2026, and the word memory now covers four mechanisms that don’t stand in for one another. Telling the four apart is the first thing to do, before you choose any tool.

Four memories that get conflated

The compiled wiki

Domain knowledge, readable by a human, versioned in a Git repository. Its first quality isn’t technical: a colleague with the right access can open the wiki to correct it and to review it before an audit. It’s often the easiest layer to explain, as long as the Git permissions and approvals and logs are themselves governed.

The memory tool

Anthropic provides a memory tool through which the model can ask to read and to write. The storage stays on your side: /memories is a convention your application maps to its own storage. Your code runs the operations and your code authorises them, which is an advantage and a burden at the same time.

Context editing and compaction

These two get mixed up all the time. Context editing removes or trims chosen tool results under rules you configure. Compaction summarises the state of the conversation when the platform is set up to do so. Both manage the working window. But neither one gives you durable memory across sessions on its own.

Managed memory stores

Some platforms offer stores shared across sessions and agents, with namespaces, versions and policy controls. These features aren’t universal. Check who gets to write, how history gets kept, whether versions can be rewritten, and what evidence ties a change to an identity and a source.

Karpathy’s actual words

On 3 April 2026, Andrej Karpathy published a gist of roughly 1,500 words describing a pattern he calls the LLM Wiki. Three layers: immutable raw sources, a wiki of Markdown files the model owns entirely, a schema that says how the wiki is organised. Three operations: ingest, query, lint the health of the whole.

In this pattern the model creates the wiki from the source material and the model maintains it, while people pick the sources, the structure and the questions. For company use I’d add one more step: the model proposes the changes that matter and an owner approves them. Letting the compiler run shouldn’t erase editorial authority.

Identity is not memory

The costliest confusion isn’t about which kind of memory but about what goes into it. The identity file says who the agent is, what it does, what it will never do. That file changes rarely, and only when a human decides to change it. Memory is designed to change. As long as the two stay apart, a passing fact can’t rewrite the agent’s mandate. Once they merge, a note saved badly on a Tuesday redefines the scope for six months, and nothing reports the change.

The content never to write into a memory

A persistent memory may be listed, then retrieved just in time during later sessions; you shouldn’t need to inject it wholesale. Every recalled item can influence the agent. That leads to three simple exclusions:

no passwords, tokens or access keys; use a secrets vault
no personal data the task doesn’t need
nothing the code repository or the business system already documents

The second line carries the legal weight: a memory that keeps a resident’s file or a colleague’s salary is processing personal data, and the purpose, the retention period and the right of access come with it. For a municipality the applicable regime is cantonal, a subject developed here.

Agent memory creates one more risk: a document or message may try to get itself stored as an instruction. So every new item needs a source, an author, a date, a confidence level and an approval state. And content from an untrusted source must never touch the agent’s identity, permissions or security rules.

Making a memory live

I recommend three operating rules, all about rhythm rather than tooling:

one intake: everything passes through a single intake folder; the model may propose a location and an owner approves it according to risk
scheduled processing: ingest and lint run at fixed hours; a memory whose upkeep waits for a daily human decision doesn’t get kept up
every new page cites the old: a graph whose files grow while its links stay flat describes a warehouse, not a memory

The hard half of the work is forgetting. A memory with no review date fills with notes that no longer hold, and the rising file count makes it look healthy. When a fact changes, you don’t overwrite the old fact. You write the new fact and mark the old one as replaced, so a false belief stays traceable to its source. Personal data also has to stay correctable, and removable from the indexes, the caches and the backups that hold a copy. And when two sources conflict, somebody has to resolve the conflict explicitly.

The cost

The most common sales argument is the token you save: a document compiled once costs less to serve than the same document re-read at every question. True, with one caveat that governs the whole calculation: the prefix cache works by exact match, so a memory that gets rewritten every turn costs more than a stable one, at equal content. The other levers on an AI bill are set out in a separate article.

Five checks before putting a memory into service

Who writes into this memory?
What happens when two pages contradict each other?
Which personal data goes in, and for how long?
Who can read what?
How do you leave?

The fifth question explains the appeal of Karpathy’s pattern, which ties itself to no technology: versioned plain text can be read with no special software, edited directly and kept after a supplier changes.

Add five controls: provenance for every fact, approval of high-impact updates, protection against persistent prompt injection, an expiry or review date, and a tested path to correct and to delete through every derivative store.

The lifecycle

Healthy memory moves through visible states: proposed → validated → available to authorised roles → reviewed or expired → corrected or deleted. Provenance, classification and access policy travel with the item the whole way. A store that only knows how to write and to read isn’t ready for company use.

Where this taxonomy comes from

The four mechanisms are a practical taxonomy, not terminology every provider shares. Check the storage, the versions, the access and the deletion in the platform you actually use.

The four mechanisms were reviewed on 14 August 2026.