Agent Memory, Measured: 3M Tokens on Disk, 550 in the Prompt

My agent has about three million tokens of knowledge available to it. A typical question puts 550 of them into the prompt, and takes 63 milliseconds to do it. There is no vector database anywhere in that path — it is a folder of markdown files, a SQLite index, and a rule about what to read first.

I went looking for numbers to compare that against and could not find any. Search for agent memory and you get comparison posts: eight frameworks ranked, Mem0 versus Zep versus Letta versus Cognee, dimension tables, verdicts. The token figures inside them come from the vendors themselves, and they are mostly aimed at each other — one publishes a critique claiming a competitor burns 600,000 tokens per conversation against its own 1,800.

Those might all be true. The problem is that none of them is a measurement you can reproduce, and none of them tells you the thing you actually need to know, which is not which system is best but how many tokens one retrieval puts in your prompt. That number is measurable on any setup, including one with no vendor in it at all. Here is mine.

Why the retrieval number is the only one that matters

I measured 23,968 real agent turns recently and the headline was that prompt tokens outnumbered output tokens 309 to 1. The agents wrote 27 million tokens and read 8.39 billion.

If that ratio is even roughly right for your workload — and it is a property of how agent loops work, not of any particular tool — then almost your entire bill is decided by what goes into the prompt. Output is a rounding error. Model choice matters less than people think. What matters is retrieval discipline: when the agent needs to know something, how much does it drag in?

That reframes memory from a storage problem into a context budget problem. Storage is cheap and getting cheaper. Context is the expensive, scarce, quadratically-billed resource. A memory system's job is not to hold a lot; it is to hand over a little.

What I actually built

No framework. The whole thing is:

• A folder of markdown files — notes per customer, per project, per incident
• A CLI that mutates them atomically and keeps an audit log
• A SQLite database with an FTS5 full-text index over everything
• A generated one-page digest per subject
• A written rule about the order you're allowed to read things in

That last one is the part people skip, and it is the part that does the work.

The numbers

Measured on my own vault, today. Byte counts are exact; token figures are the standard characters-divided-by-four estimate, and my content is a mix of English and Turkish technical prose, which tokenizes worse than plain English — so the real corpus numbers are somewhat higher than shown, which only sharpens the point.

The index is over eleven times larger than the thing it indexes. On a storage-shaped mental model that looks like a problem. It is not, and understanding why is most of the point of this article: the index never enters the prompt. It lives on disk, gets consulted in 63 milliseconds, and returns half a kilobyte of text. Disk is cheap. Context is not. Spending 138 MB of disk to avoid spending 20,000 tokens of context is a trade you should take every single time.

What summarising actually buys

The subject-level digests are the sharpest measurement I have. Across 21 subjects with 337 underlying files:

That 652× case is a subject with 63 files of accumulated history. When the agent needs to know the current state of it, reading the folder would cost roughly 600,000 tokens. Reading the digest costs about 940. Both answer the question. Only one of them is affordable, and only one of them leaves room in the context window to actually do the work afterwards.

The rule that does the work

Having the index is not the mechanism. The mechanism is a written retrieval order that both I and the agent follow, with a hard instruction to stop as soon as the question is answered:

1. Unsure which subject? Run a finder query. It returns pointers and snippets, not documents.
2. Exact token — a ticket ID, an IP, a domain, a hostname? Full-text search. It goes straight to the line.
3. Need current state of a subject? Read the digest. About 550 tokens.
4. Need one specific part? Read that section by name, not the file.
5. Still missing something? Then open the single document the digest pointed at.

And an explicit list of things that are forbidden because they are token sinks:

• Reading raw chronological logs
• Reading an entire subject folder
• Handing a full document to the model when a section would do
• Reading anything else "to be sure" after the answer was already found
• Any unfiltered bulk scan

That last one is the most common failure in practice, and the most human. The agent finds the answer, then reads three more files to feel confident. Each of those is a permanent addition to the context for the rest of the session — and per the 309:1 measurement, everything you add gets re-sent on every subsequent turn. Confidence-reading is the single most expensive habit an agent can have.

Where this approach is genuinely worse

I am not claiming a folder of markdown beats a purpose-built memory platform. Some honest limits:

There are no embeddings. Search is lexical — FTS5, not semantic. Ask for "the thing where the disk filled up" and it will miss a note that says "volume exhausted". A vector store handles that; this does not. I deferred embeddings deliberately, because lexical search turned out to answer the overwhelming majority of real questions — most of what I look up is an identifier, and identifiers are exact strings.

Digests are generated, so they go stale. A digest is only as good as its last regeneration. A system that updates memory continuously as the agent works does not have this problem.

It is single-user and local. No multi-tenant story, no shared team memory, no hosted anything. That is a feature for me and a blocker for a product.

Nothing here is novel. Files, a full-text index, and a summary. The interesting part is not the architecture, it is that the numbers are good enough that the architecture stops mattering.

What to measure in your own setup

Whatever you are using — a framework, a vector store, a folder like mine — these four numbers tell you almost everything, and none of them requires a vendor's cooperation:

Tokens per retrieval. Take a normal question, capture what your memory layer hands the model, count it. If it is in the thousands, that cost is paid again on every subsequent turn of that session.

Corpus-to-retrieval ratio. Mine is roughly 3,000,000 to 550. The absolute size of the corpus should be nearly irrelevant to the cost of one question. If growing your knowledge base makes each question more expensive, retrieval is not doing its job.

Prompt-to-output ratio over a whole session. Mine measured 309:1. This is the number that tells you whether you have a retrieval problem at all — a high ratio means the agent is carrying history rather than doing work.

Retrieval latency. Mine is 63 ms, which is far below the threshold where anyone would notice. If yours is in the hundreds of milliseconds, you are paying real time on every step of every loop.

FAQ

How many tokens should an agent memory retrieval use?

Few hundred rather than few thousand. In this measured setup the median subject digest is about 550 tokens and a typical full-text search result is 490–630. The reason the ceiling matters is compounding: an agent turn resends the whole conversation, so anything retrieval adds is billed again on every following turn of the session.

Do I need a vector database for agent memory?

Not necessarily, and it is worth measuring before assuming. Lexical full-text search answered the large majority of real questions in this setup, because most lookups are exact identifiers — ticket numbers, IP addresses, hostnames, domains. Embeddings earn their place when you need to match meaning rather than strings; the honest failure mode of a lexical index is that "the disk filled up" will not find a note that says "volume exhausted".

Why is the search index larger than the data it indexes?

Because a full-text index stores tokens and their positions in addition to the source text, so a multiple of the corpus size is normal. It does not matter, because the index lives on disk and is never sent to the model. In this setup the index is 11.4× the corpus at 138 MB, and it returns roughly 550 tokens per query in 63 ms. Disk is cheap; context is not.

What is the difference between RAG and agent memory?

In practice they are the same mechanism aimed at different content: RAG usually retrieves from a fixed document corpus to answer a question, while agent memory retrieves from knowledge that the agent itself accumulated and keeps updating. The cost question is identical for both — how many tokens does one retrieval put into the prompt — which is why it is worth measuring the same way regardless of which word you use for it.

How do I stop an agent from reading too much context?

Write an explicit retrieval order, tell it to stop as soon as the question is answered, and name the forbidden operations — reading whole folders, reading raw logs, passing a full document where a section would do, and reading extra files "to be sure" after the answer was already found. That last one is the most common and most expensive, because everything added to context is re-sent on every later turn.

Does compressing knowledge into summaries lose information?

Yes, and that is the trade being made deliberately. A digest that compresses 2.45 MB to 3.7 KB is discarding almost everything, and it works because the discarded part is history the agent does not need to answer the current question. The safeguard is that the digest carries pointers, so when detail is genuinely required the agent opens exactly one source document instead of the whole folder.

The thing I got wrong first

I built the index before I wrote the rule, and for a while it changed nothing. The agent had full-text search available and kept reading whole files anyway, because nothing told it not to and reading more feels safer than reading less.

The retrieval order was the actual fix, and it is fifteen lines of plain text. The index made a cheap path possible; the written rule made it the default. If you take one thing from this, take that ordering — a retrieval mechanism nobody is required to use is a mechanism that will not be used.

That is the same shape as something I ran into auditing my agents' guardrails, where five of seven rules turned out to exist only as documentation while the thing meant to enforce them had never been wired up. Writing the capability and making it the default path are two separate pieces of work, and finishing the first one feels almost exactly like finishing both.