# How Agent Memory Will Change Your Organization

Devin Stein, Dosu | MLOps Community | 24:33
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=ik2eYzfEbEE
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/how-agent-memory-will-change-your-organization
Published: 2025-08-27
Tags: caching, long-context, memory, rag

## TL;DR
- Dosu is building agents that save useful facts from previous investigations so later questions can be answered with retrieval instead of repeated research.
- Those facts need the same maintenance as documentation because code, conversations, and organizational practices change over time.
- Dosu limits unproductive agent runs with loop limits, reflection steps, early exits, and planned failure-mode classification.

## Summary
Devin Stein describes Dosu's new agent architecture, which combines retrieval with deeper agent research. Dosu originally used asynchronous agents for GitHub issues, where a two-to-ten-minute response could replace a typical response time of several days. Users later wanted faster answers in Slack and other synchronous channels, creating a conflict between speed and the research needed for difficult questions. Dosu's answer is fact-based reasoning. After an agent investigates a problem, it can save useful internal learnings as facts. A later request can retrieve those facts immediately and research only the missing information. Stein argues that this creates a growing knowledge base, but he is clear that the system is hard to maintain. Facts need to be removed, reconciled, consolidated, and updated when organizational knowledge changes. Dosu treats these facts as another form of documentation and monitors changes such as pull requests for conflicts. Stein distinguishes internal facts from memories created through user feedback or external interactions.

## Key ideas
### Dosu began with slow, research-heavy agents because quality mattered more than speed
[00:00](https://www.youtube.com/watch?v=ik2eYzfEbEE&t=0s)
Dosu is an intelligent knowledge base for software teams and agents. Its GitHub app investigates issues by looking through code, commits, conversations, and tickets. Devin Stein says these investigations usually take two to ten minutes, which is fast compared with the roughly three-day response time common in GitHub support. The asynchronous design gave the agent time to research difficult questions and produce better answers. Users later wanted to use Dosu in Slack, Discord, and its app, where they expected immediate responses.

### Retrieval and agent workflows make different speed and complexity trade-offs
[03:48](https://www.youtube.com/watch?v=ik2eYzfEbEE&t=228s)
Stein contrasts typical retrieval-augmented generation pipelines with agent workflows. RAG is relatively fast and cheap, and its quality depends heavily on whether retrieval finds the right context. It becomes harder when data grows beyond the context window, when several data sources are involved, or when a question requires multiple steps. Agents are slower and use more tokens, but their performance depends on the quality of their tools and they can investigate complex problems. Dosu wanted both behaviors without maintaining two separate agents.

### An agent should use what it knows before starting new research
[06:13](https://www.youtube.com/watch?v=ik2eYzfEbEE&t=373s)
Stein compares the design to an engineer onboarding. A new engineer starts slowly because they have no internal knowledge. With experience, they recognize where code lives, which recent pull requests matter, and which conversations may contain the answer. They respond immediately when the answer is already familiar. When they are unsure, they research only the gaps. Dosu aims to give agents the same progression, so an agent can use existing knowledge first and investigate only what it still needs to answer the question.

### Fact-based reasoning turns successful investigations into reusable knowledge
[10:26](https://www.youtube.com/watch?v=ik2eYzfEbEE&t=626s)
Dosu calls its approach fact-based reasoning. The system keeps a set of facts about a topic and checks whether those facts already answer a new request. If they do, it can use retrieval and return quickly. If they do not, the agent identifies the gap, researches it, and can save newly learned facts after it produces a correct answer or successful pull request. Stein says this avoids repeating work, such as listing the same directory during every investigation, and reduces both latency and cost.

### Facts need continuous maintenance because organizations keep changing
[12:59](https://www.youtube.com/watch?v=ik2eYzfEbEE&t=779s)
Stein compares saved facts with a cache, then points to cache invalidation as the hard part. Organizations change, so learned facts can become false or conflict with newer information. Dosu needs to prune outdated facts, reconcile contradictions, and consolidate information from different sources. The product can inspect pull requests and conversations to ask whether a change affects existing knowledge. When it finds a contradiction, it can surface the issue for users to decide what is true.

### Agents can generate organizational documentation while answering questions
[15:27](https://www.youtube.com/watch?v=ik2eYzfEbEE&t=927s)
Treating agent facts as documentation gives Dosu a living knowledge base. An agent's investigation produces more than a response because it can preserve the learning behind that response. Stein compares this with an engineer documenting the result of a difficult investigation. In this design, documentation is produced as part of the agent's work and can then be maintained when the underlying code or organizational knowledge changes.

### Facts and memories differ by where the learning comes from
[16:17](https://www.youtube.com/watch?v=ik2eYzfEbEE&t=977s)
Stein sees facts and memories as related but distinct. Facts are internal learnings created during an agent's research process. They help the agent form a conclusion but are usually hidden from the user, who sees the response or document. Memories come from external interactions or explicit feedback, such as learning a user's preferred response format or being told to inspect a particular part of the codebase. Facts are internal to response generation, while memories come from interactions around the agent.

### Dosu stops agents from spending forever on questions they cannot solve
[18:02](https://www.youtube.com/watch?v=ik2eYzfEbEE&t=1082s)
When an agent does not know an answer, it can keep working and consume resources without making progress. Dosu uses hard-coded loop limits for some query types. After each loop, it runs a reflection step that checks whether the agent is progressing. If it is stuck for a set number of steps, the system exits early. Stein says the team is working toward more specific failure-mode classification, such as detecting repeated tool queries that produce no new information.

## Notable quotes
- Devin Stein: "When you talk to teammates, you don't ask them if they should answer fast or slow. You expect them to figure it out." (06:35)
- Devin Stein: "The goal being that day n agent is actually fast where the same way a human onboards can this agent on board by learning these facts over time." (10:50)
- Devin Stein: "Facts are very much internal as part of the response generation process." (17:17)
- Devin Stein: "We basically have a hard-coded limit in terms of the number of loops we're willing to tolerate for certain types of queries." (18:48)

## Tools & references mentioned
- Dosu
- MLOps Community
- Demetrios Brinkmann
- GitHub
- Jira
- Slack
- Discord
- RAG
- Grok
- ChatGPT
- Perplexity
- OpenAI
- Airflow
- Apache Superset

## Who should watch
- You are building an agent that needs to answer quickly in some contexts and research difficult questions in others.
- Your system stores agent-generated knowledge and you need a plan for updating or removing facts when the underlying organization changes.
- You are deciding how to stop agents from repeating tool calls or continuing indefinitely when an answer is unavailable.

## Related talks

- [Knowledge is Eventually Consistent](https://mlopstalks.com/talks/knowledge-is-eventually-consistent) (Devin Stein, Dosu, 55:15)
- [Overcoming Agentic Memory Management Challenges](https://mlopstalks.com/talks/overcoming-agentic-memory-management-challenges) (Biswaroop Bhattacharjee, Prem AI, 50:35)
- [9 Commandments Building AI Agents](https://mlopstalks.com/talks/9-commandments-building-ai-agents) (Paul van der Boor & Dmitri Jarnikov, Prosus Group, 1:20:34)
- [AI Needs Memory: Here's How It Works](https://mlopstalks.com/talks/ai-needs-memory-heres-how-it-works) (Ben Labashian, Workhelix, 29:02)
- [Multi-Agent Personalization with Shared Memory: From Email to Website to Proposal](https://mlopstalks.com/talks/multi-agent-personalization-with-shared-memory-from-email-to-website-to-proposal) (Hamed Taheri, Personize.ai, 16:57)