Memory for AI agents is data managed by a dynamic system, so storage and retrieval cannot be treated like a standard database problem.
2
Agents need to decide what to retain, and those decisions are stochastic because language is fuzzy and the agent operates under constraints.
3
Better memory storage and retrieval can lower marginal costs, allowing agents to handle longer sessions and more complex tasks.
Summary
Ben Labashian argues that memory is one of the hardest parts of building reliable AI agents. Calling memory simply "data" makes storage and retrieval easier to reason about, but agents decide dynamically what to keep and how to use it. The same query can produce different results, and language makes it difficult to define a perfect retention algorithm. He discusses short-term, working, sensory, and long-term memory, including episodic, semantic, and procedural forms. Practical approaches include named entity recognition, semantic caching, compression, cascading memory, and vector-store offloading. Memory also has a direct economic effect: efficient retrieval lowers marginal cost and makes longer sessions and more complex work more viable. In the discussion, Labashian recommends planning agent tasks explicitly and dividing large contexts among specialist agents coordinated by a central planner.
Agent memory is data handled by a nontraditional system
Labashian starts with a practical definition: memory is data for agents. That framing connects memory to storage, retrieval, databases, and embeddings. The difficulty is that the system using the data is dynamic. Agents are tool-based systems operating under constraints, deciding what information to access and what work to perform. Storage can use familiar infrastructure such as embedded vectors and PGVector, but the agent's behavior is less predictable than a conventional program. The same task or query can return different results, so the data layer cannot fully determine how memory behaves.
Language is fuzzy, Labashian says. The meaning one person intends is not always the meaning another person receives, and this makes retention difficult for agents. An LLM must decide what information is important enough to save, often within limits on context, storage, and computation. Labashian does not think there will be a perfect algorithm for memory, because such an algorithm would require language to be constrained more precisely than it is. Retrieval and storage methods still matter, but they operate within this uncertainty.
Human memory categories are useful even when they are inaccurate
Labashian treats the usual memory categories as imperfect models that can still help with agent design. Short-term or working memory covers the current interaction and context. Long-term memory can be divided into episodic memory for past conversations, semantic memory for preferences, and procedural memory for methods. Sensory memory is sometimes used to describe a perception layer that decides what enters memory. These categories do not claim to reproduce how the human brain works. They give engineers practical ways to discuss different retention and retrieval needs.
Agents must filter memory because retaining everything is expensive
Saving every conversation creates both economic and computational problems. Labashian points to the cost of storing and processing long histories, along with the quadratic growth of search. The agent must decide what to retain, but that classification is itself nondeterministic. He uses Claude Code compaction as an example: when a conversation is compacted, details can disappear and the agent may approach the task differently afterward. The problem is therefore deciding what information matters before the context is compressed or discarded.
Memory systems can combine familiar data techniques with agent-specific decisions
Labashian names several approaches to memory management, including importance scoring, cascading memory, intelligent compression, and vector-store offloading. Some resemble established techniques such as FIFO and LIFO queues, summarization, recommendations, and user-preference storage. Named entity recognition can extract important information while reducing the search space. Semantic caching can keep frequently accessed information available without repeatedly querying a vector database. These methods reduce storage and retrieval work, but the agent still has to decide when and where memory belongs.
Efficient memory changes what agents can afford to do
Memory has an economic effect beyond infrastructure efficiency. When storage and retrieval become cheaper, the marginal cost of using an agent falls. That can make longer sessions, less frequent compression, and more complex tasks viable. Labashian describes this using marginal cost and marginal benefit curves: improvements in memory move the cost curve so agents can do more before the cost outweighs the value. He expects lower costs for complex tasks to support the use of agents across more kinds of work.
Planning is better than repeatedly asking an agent to fix a failing task
In the question period, Labashian identifies a common bad memory and context strategy: continuing to tell an agent to figure out a failing problem instead of making a plan. He recommends asking the agent for a step-by-step plan, criticizing it, and checking that the agent follows it. Simply pressing enter and disengaging leaves the agent without enough structure. He says structured methods are more effective for retaining information, although the right structure depends on the problem.
Specialist agents can keep large contexts manageable
For a system handling many long conversations, Labashian suggests splitting context among agents that each focus on a particular topic or function. A central planner can route a topic to the appropriate specialist, which returns information to the planner before the result goes to the user. The related memories can be partitioned into separate areas of a larger vector store. This approach limits the context each specialist must handle and makes stored information more relevant to the task it supports.
"If you can split up that context and say, this topic is for this agent, this topic is for this agent, and then suddenly it's just good at that one aspect, feeds up to the central planner."Ben Labashian27:40
Who should watch
You are designing an agent that needs to retain information across conversations and need practical ways to decide what gets stored.
Your system is hitting context, retrieval, or cost limits as conversations become longer.
You are considering multi-agent designs and want a concrete reason to partition memory and route work through a central planner.
