Keeping Agent Token Costs Down, With an Idea From Robotics

Ask an AI agent a question about an investigation, and it searches the evidence, reads documents and builds an answer. Ask a related question later, and some of that work can happen again.

Without persistent memory, the agent has little to carry forward beyond the context supplied to its next run. Documents get retrieved again. Relationships get reconstructed. The team pays, in time and in tokens, for work it has already done.

At Donna, we borrowed an idea from robotics: build a map as you work, then use it to decide where to look next. We built it for investigations involving mailboxes, chat exports, contracts and bank statements, with new evidence arriving throughout the matter. The same principle applies wherever agents repeatedly work over a growing body of information.

The Cost of Rediscovering Evidence

Most agent platforms, ours included, start each run in a fresh, isolated worker. That is the right call for isolation and security. The agent fans out helper agents, which search, read, re-read and summarise, and then it answers. Ten minutes later the same investigator asks a follow-up on the same matter, and a new worker starts from zero.

When we profiled our own runs, a large share of the time went to helper agents rediscovering context, and then to the lead agent re-reading files a helper had already read. Prompt caching makes each repeated token cheaper. It does not stop the agent paying to re-learn what it already knew.

And an investigation rarely consists of unrelated questions. Who approved these payments? When did they know? Who else was on the thread? Those questions keep landing on the same small set of documents.

More Than a Saved Answer

The obvious fix is a cache. An investigation breaks one in three ways.

• The evidence changes. A new mailbox can alter a timeline or contradict an earlier finding. An answer can become incomplete even when none of the documents it cited has changed.
• Access differs. A firm, its client and an outside expert may work on the same matter while seeing different documents. Knowledge drawn from a restricted source needs the same care as the source itself.
• Questions overlap. Two questions can need much of the same evidence without using the same words. A cache keyed on the prompt misses the paraphrase.

Retrieval finds chunks, but it does not remember what the last run learned. We wanted something that knows what it covers, which version of the evidence it was built from and who may see it. Robotics solved a version of this problem decades ago.

What a Robot Carries Forward

A robot dropped into an unfamiliar building has no map and does not know where it is. It needs a map to work out where it is, and it needs to know where it is to build a map. Simultaneous localisation and mapping, or SLAM, solves both together, in a loop.

The useful idea for us was broader than any particular SLAM algorithm. A map does not remove the need to look around. The world may have changed, and parts of the map may be incomplete. But the robot has something to build on, and it points its sensors where the map is thin.

Replace the robot with an agent, the building with an investigation, and the lidar with search and read tools. This is an architectural inspiration rather than a direct implementation of SLAM. Documents are not physical landmarks, and access control raises problems a robot never faces. But the principle transfers well: use what you know to decide what you need to examine next.

A Map for Each Matter

Donna’s map is a matter brief: a tree of short pages, stored per matter. There is one page per document, written when the document is processed, with a summary, the parties, key dates and obligations. Index pages nest by folder and split as they grow, so the root always fits in a prompt. Above them sit pages for people and organisations, a timeline, topic pages, and answer pages for questions people actually asked. Every page is kept short, so reading the brief costs a fraction of re-reading the documents.

Localisation is the part most agent stacks skip. Every run starts with a small root index of the brief in its prompt, built for the person asking, and a rule: read the page before you search, and go to the documents only for what the page does not hold.

Consider an investigator asking who approved a payment. An earlier run may already have located the email chain and identified the people involved, and left a route card on the page that answered it. A follow-up about when the finance team was told can begin there instead of repeating the first search. The agent may still need another message or an exact passage. The saving comes from narrowing the work.

One Run Through the Loop

A question arrives, and the agent localises: it reads the relevant page of the brief. If the page holds the answer, it answers from there. If it does not, because the page does not cover it or the question needs exact wording, helper agents go to the documents, but only for the gap.

While that happens, a hook on the agent runtime watches every tool call across the whole tree, the lead agent and its helpers alike. It records what was searched and which documents matched, which pages of which documents were read, and which pages of the brief were opened. We do not ask the model to take notes. Models are poor bookkeepers, and note-taking costs tokens.

After the answer is saved, code files that observation onto the brief, with no model call. Each document the run touched gets a route card, an updated coverage level and the passages the run retrieved. The question and the run’s own answer, with its citations, become an answer page. When someone later asks the same thing in different words, it merges into the same answer page, and answer pages nobody reads fade out. New uploads trigger the same compile, batched so a burst of files triggers one compile, not hundreds.

The model’s only new job is to read the brief first. The number we watch is the share of runs answered from the brief without searching or reading a document. On a new matter it is low, as it should be. As the matter matures, it should climb.

The Map Points the Way. The Documents Prove It.

A matter brief is a working summary of the evidence. The documents remain the record. Pages in the brief carry short passages quoted from text a run retrieved, with page numbers, and point back to the documents they came from. When Donna answers, each citation has to match text taken from the evidence, and the investigator can open the cited passage in its document.

A citation shows where a statement came from. It does not, by itself, show that the statement represents the source fairly. That is why the investigator reviews the findings. Reusing earlier work should save the search, never the check.

Freshness Includes What Has Arrived Since

There are two ways a matter brief can fall behind.

First, a document can change. Every page about a document records the revision it was built from, its watermark. When a newer revision lands, the compile sees the mismatch and rebuilds the page. That is a comparison in code, not a model judging whether something still looks right. The compile runs on every upload, and again at the start of every run under a strict time limit. If it runs out of time, it fails open and the run reads the brief as it stands.

Second, new evidence can arrive. This is harder. Every existing source may stay the same while a new email changes how all of them read. When documents are added, the pages on people, organisations and events are recomputed, and a background pass reconciles names and flags contradictions. An earlier answer, though, records what was found in the evidence reviewed at the time. It is a lead, not a conclusion about the whole matter.

That matters most for questions about absence or completeness. “Nothing in the documents reviewed supports this” is a narrower statement than “there is no evidence of this anywhere in the matter.” Earlier work can guide the search for missing evidence, a complete timeline or everyone involved in an event. It cannot establish what has not yet been reviewed. For an investigator, an explicit gap is safer than a confident answer built on incomplete coverage.

Shared Knowledge, Controlled Access

This is where the robotics analogy ends, and where we spent the most engineering. A shared brief is only useful if it respects the permissions of the evidence behind it.

Every page carries an audience on a ladder: shared with everyone on the matter, the firm, a named group of people, or one person. When code writes a page, it takes the narrowest rung of all its sources. If a page would mix two people’s private files, no audience is safe for both, so the page is dropped rather than clamped.

At read time we do not trust the stored audience. It is recomputed from the sources’ permissions, and anything that cannot be resolved fails closed. The stored root index also carries a fingerprint of the documents, folders and audiences it was built from. If permissions have moved since, it is not served.

When someone loses access to a file, the change propagates. Answer pages built from it are refiled with a narrower audience or deleted. Earlier answers in that person’s conversations that cited the file are withdrawn and replaced with a note saying so. That stops Donna showing the answer again. It cannot recall what was already read or exported.

The aim is straightforward. Work done by one investigator can help another who has access to the same evidence, without widening access to the evidence itself.

When Later Questions Cost Less

The opportunity is greatest when questions overlap. The first question may need substantial discovery. A related question can then reuse part of that work, with less repeated searching, reading and context processing. Filing costs no model calls, and the brief is plain text, so it stays small.

That does not make every later question cheaper. A new issue, a large upload or a request for an exhaustive review can need substantial fresh work, and maintaining the map has a cost of its own. The fair comparison is the total cost of answering a representative set of questions, including the upkeep, alongside speed, answer quality and correct permissions. A high reuse rate only counts if the agent still searches when it should.

What We Took Away

• Write the map with code, not the model. A hook on tool calls gives you the observation for free, and filing is deterministic.
• Make localisation a rule. A map the agent is merely offered gets politely ignored.
• Scope on write, recompute on read, fail closed. Drop anything that cannot be safely scoped.
• Pick one metric. The share of runs answered from the map tells you whether it is earning its keep.

For investigation teams, the result is simple to state. Related questions start from evidence already located, findings stay connected to their sources, and earlier answers stay within the permissions of the documents behind them. The robotics lesson is not that a map replaces observation. It is that each observation can make the next action better informed, and every run should leave the matter better understood than it found it.