Agency17 June 20264 min read

A five-layer memory for an AI agent that works a client for nine months

A context window is not memory. For an agent that handles one client's accounting across nine months, we built memory as five distinct layers (facts, history, decisions) with a promotion path that turns a one-off ruling into a standing rule.

tsukumo

A five-layer memory for an AI agent that works a client for nine months

Short version: a context window is not memory. For an agent that handles one client's accounting across nine months, the hard part was not the model, it was remembering: the rules of this specific mandate, the decisions made along the way, the history of what was said and done. We built that as five distinct layers, each answering a different question, with a promotion path that turns a repeated decision into a standing rule. Collapse all of that into "stuff more into the context window" and the agent forgets exactly the things a long client relationship depends on.

Why a bigger context window doesn't fix memory#

Because a context window is working space, not memory, and the two have different jobs.

A context window is re-filled every session, capped in size, and flat: it has no idea whether a line is an immutable fact, a one-time exception, or a rule that should apply forever. Over nine months with a client, the agent has to recall a ruling made in month two without re-reading month two, and apply it as policy rather than as trivia. That is a storage-and-retrieval problem, and bolting a bigger window onto it just makes the forgetting more expensive.

The test of memory is not "can the model see it right now." It is "can the agent recall the right thing, eight months later, without being handed everything." A window fails that test by construction.

The five layers#

Each layer answers a different question, so each is a different store.

| Layer | What it holds | Question it answers |
|-------|---------------|---------------------|
| Rules / mandate graph | The durable truth of this client: entities, mandates, standing rules | "What is always true here?" |
| Working store | Recent state, fast access | "What is happening right now?" |
| Decision journal | What was decided, when, and why | "Why did we do it this way?" |
| Communications archive | Semantic index of past exchanges | "What was said about this before?" |
| Activity log | What the agent actually did | "What happened, exactly?" |

The graph at the top is the slowest-changing and most authoritative. The activity log at the bottom is append-only history. The three in between are where a living client relationship is actually held: current state, the reasons behind decisions, and the searchable record of what was communicated.

The layer most teams skip: the decision journal#

The difference between an agent that feels reliable and one that feels amnesiac is whether it remembers why.

Most memory designs store facts and history but not decisions and their reasons. That gap is exactly where a client loses trust. When the agent has to explain why an invoice was handled a certain way, "because that is what we decided in March, for this reason" is a different relationship than re-deriving it from scratch and maybe landing somewhere new. The decision journal is what lets the agent be consistent with its own past, which is most of what "remembering a client" actually means.

The promotion path: from decision to rule#

A repeated, stable decision should stop being a decision and become a rule. That transition has to be deliberate.

A one-off ruling lives in the decision journal. When the same ruling recurs and proves stable, it is promoted up into the rules graph, where it becomes a standing constraint the agent applies by default instead of re-deciding each time. The key word is deliberate. Promotion is a controlled step, not an automatic one, because the dangerous failure is a one-time exception quietly hardening into policy. The promotion path is how an agent learns a client's preferences over months without mistaking an accident for a rule.

Memory that only accumulates becomes noise. The promotion path is what gives the system a spine: it decides what graduates from "we did this once" to "this is how we do it," and keeps the graph small and authoritative.

Five layers vs one big store#

| One undifferentiated store | Five purpose-built layers |
|----------------------------|---------------------------|
| Facts, decisions, and chatter mixed together | Each kind of memory in its own store |
| Recall is "search everything and hope" | Recall targets the layer that holds the answer |
| No notion of a standing rule | Rules graph holds what is always true |
| Exceptions and policy look identical | Promotion path separates them, deliberately |
| Grows into noise | Stays authoritative because each layer is bounded |

What this means for your team#

If you are building an agent meant to hold a relationship over time, design memory as an architecture, not a retrieval call. Decide which store answers "what is always true," which holds "what we decided and why," and how a decision graduates into a rule. The model is the easy part. The memory is what makes the agent feel like it has been working with you for months rather than meeting you fresh every session.

We built this for an accounting agent where forgetting a client's standing decision is not a glitch, it is a broken trust. If your team needs an agent that actually remembers, that's the work we do.

Common questions

Why isn't a large context window enough for agent memory?

A context window is short-term working space, not durable memory. It is re-filled every session, capped in size, and has no notion of what is a fact versus a one-off versus a standing rule. Over a long client relationship the agent needs to recall a decision made eight months ago without re-reading everything, which is a storage and retrieval problem the window does not solve.

What are the layers of agent memory?

In the system we built: a rules/mandate graph (the durable truth about the client), a fast working store for recent state, a decision journal (what was decided and why), a semantic archive of past communications, and an activity log (what the agent actually did). Each answers a different question, so each is a different store.

How does an agent promote a decision into a rule?

Through an explicit promotion path. A one-off decision lives in the decision journal. When the same decision recurs and proves stable, it is promoted up into the rules graph, where it becomes a standing constraint the agent applies by default. Promotion is deliberate, not automatic, so a one-time exception never silently becomes policy.

Is this just RAG?

RAG (retrieval over documents) is one layer, not the whole system. Semantic retrieval helps recall past communications, but it cannot tell a fact from a decision from an exception, and it has no promotion path. Real agent memory is an architecture of several stores with different jobs, of which retrieval is one.

Want this running on your team?

Talk to us about your team