# 9 Commandments Building AI Agents

Paul van der Boor & Dmitri Jarnikov, Prosus Group | MLOps Community | 1:20:34
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=QsP-sbUJBEo
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/9-commandments-building-ai-agents
Published: 2025-08-01
Tags: agents, memory, planning, tool-use

## TL;DR
- Useful agents reduce cognitive load by choosing structured interactions, asking for clarification when requests are ambiguous, and avoiding unnecessary messages.
- An agent should reason about its path, use tools, observe its situation, reflect on results, and know when to stop.
- Memory should capture user preferences and successful or failed execution paths, then feed those learnings back through context, retrieval, caching, or training.

## Summary
The discussion explains why agents often create more work than they remove. Paul van der Boor describes interfaces that guide users with buttons and structured choices instead of producing long answers. He also explains how Prosus tests tools repeatedly, decides what to build or buy, and collects successful and unsuccessful user journeys as training data. Dmitri Jarnikov defines an agent as a system that chooses its own path to a goal and decides when it is finished. His design principles include the reason-act loop, observation, planning, reflection, tool abstraction, sandboxed code execution, memory, and execution budgets. The speakers distinguish static plans from dynamic plans, explain why agents need human help when they are uncertain, and describe memory as both personalization and a record of how tasks should be completed. They are frank about unresolved problems, especially stopping criteria and the tradeoff between speed, quality, and cost.

## Key ideas
### Agents should reduce cognitive load instead of producing more information
[01:09](https://www.youtube.com/watch?v=QsP-sbUJBEo&t=69s)
Demetrios Brinkmann and Paul van der Boor describe an agent connected to GitHub that was disabled after about six hours because it kept sending verbose notifications. Paul says AI systems can be disrespectful of people's finite time when they report everything instead of filtering for importance. In e-commerce, a response to a search such as "iPhone 15" can create an overwhelming wall of text. Paul's proposed interface gives users visual and structured choices such as more options, cheaper options, or options nearby. Buttons let people indicate their next goal without writing another detailed request.

### Teams need continuous experiments to know what is ready for production
[05:08](https://www.youtube.com/watch?v=QsP-sbUJBEo&t=308s)
Paul says the Prosus AI team runs hackathons, codathons, and similar experiments because tools change quickly. They tried products such as Devin and other models at different points and found that some improved substantially while others did not. The team uses these experiments to separate real production opportunities from demos. Paul expects models to become cheaper, context windows to grow, and modalities such as voice, video, and imagery to improve. That expectation affects what the team builds today. They still build specific capabilities, such as enterprise authentication and permissions, when they believe general providers will not solve the problem for them.

### Build or buy depends on the problem and the speed required
[10:46](https://www.youtube.com/watch?v=QsP-sbUJBEo&t=646s)
Paul rejects a permanent rule that a team should always build or always buy. Building has become faster and cheaper, especially with tools such as Devin and Cursor, so a small team may create a custom solution rather than evaluate several imperfect vendors, negotiate contracts, and wait for integration. At the same time, Paul encourages the team to use outside solutions when they solve a general problem well enough. He gives Arcade as an example they are adopting. He also describes how MCP replaced an earlier internal tool-controller solution with a shared standard, reducing the amount of custom work required.

### General agent capabilities should let subject-matter experts create focused systems
[18:14](https://www.youtube.com/watch?v=QsP-sbUJBEo&t=1094s)
Prosus is building Token as a general agentic capability with tools that employees can connect to systems such as Salesforce, monday.com, Databricks, SAP, and internal data environments. Employees can create focused workspaces without programming. An HR employee could define an HR support bot, specify the data it may access, describe how it should behave, and publish it for others. Paul frames this as giving employees their own AI workforce, including agents that act like junior analysts or interns. Engineers build the underlying integrations and controls, while subject-matter experts provide the domain instructions.

### Agent memory includes both personal context and lessons about execution
[21:49](https://www.youtube.com/watch?v=QsP-sbUJBEo&t=1309s)
Paul separates memory about a user from memory about how to complete a task. User memory includes preferences and past interactions. Execution memory records whether an agent reached its objective and which path it took. In e-commerce, a successful apartment search or food-ordering interaction can become a positive example, while a failed web-browsing attempt can show what to avoid. Those experiences can be stored as cache or retrieval data, or used in later training. Dmitri adds that short-term scratchpad memory records events within one task, while long-term memory persists across sessions and can contain user profiles or general lessons about execution.

### The agent loop needs observation and reflection around reasoning and action
[36:08](https://www.youtube.com/watch?v=QsP-sbUJBEo&t=2168s)
Dmitri defines an agent as a system that selects its own path toward a goal and decides where to stop. He describes the familiar reason-act cycle, then expands it into observe, think, act, and reflect. Observation checks the user's circumstances and whether enough information is available. Thinking clarifies the task and chooses how to proceed. Action executes the plan. Reflection examines the result and can trigger another pass. A coding agent might run tests, inspect failures, and revise its code. The cycle is not rigid. A transcription-and-translation task may need almost no observation or reflection, while a data-analysis request may need to ask questions and inspect available documentation before acting.

### Every capability should be available through a simple tool interface
[46:37](https://www.youtube.com/watch?v=QsP-sbUJBEo&t=2797s)
Dmitri argues that anything an agent can call or use should be abstracted as a tool with an input and output interface. This can include APIs, other agents, and people. A human can answer a question when the agent lacks information or approve a payment. Treating that interaction as a tool call gives the agent a consistent way to request help. Dmitri also recommends a sandboxed code executor for tasks that cannot be covered by individual tools, such as data analysis, graph creation, or data ingestion. Code execution covers unusual cases without forcing the agent to choose from a list of hundreds or thousands of tools.

### Static plans converge more reliably, while dynamic plans can run forever
[52:41](https://www.youtube.com/watch?v=QsP-sbUJBEo&t=3161s)
A static plan lays out the sequence of tool calls in advance. Dmitri considers it suitable for predictable work such as transcribing and summarizing a meeting. It increases the chance of reaching a result, but an error in an intermediate step can leave the agent unable to recover. Dynamic planning chooses the next action after observing the previous result. It can recover from errors, but it may continue indefinitely as the agent keeps trying to improve the answer. Dmitri says agents struggle with knowing when better has become the enemy of good. In an anecdote, an unrestricted coding agent might search the web, email package developers, post on forums, or hire someone if it has enough permissions.

### Execution budgets must balance cost, speed, quality, and urgency
[1:13:14](https://www.youtube.com/watch?v=QsP-sbUJBEo&t=4394s)
Dmitri says simple cost caps are fairly easy to estimate from model calls and infrastructure use. A food-ordering agent can compare the value delivered with the cost of its execution. The harder problem is choosing among tools with different prices, speeds, and quality. An agent may need to decide whether an answer should arrive immediately, take longer for better quality, or use a cheaper option. Dmitri describes this as an urgency and tradeoff problem rather than only a technical limit. The speakers leave open how agents should express and follow changing priorities during a task.

## Notable quotes
- Demetrios Brinkmann: "I call it like the agents or the LLMs and AI in general is disrespectful of my time." (02:14)
- Dmitri Jarnikov: "For me AI agent is a solution that is able to achieve a task by selecting by itself the path towards the goal and also by defining itself what the end is, so where to stop." (34:07)
- Dmitri Jarnikov: "Everything that it can call, everything it can use, whether they're mechanical or human, it's a different story, could be and should be abstracted away as a tool with some simple interface for input output." (47:29)
- Dmitri Jarnikov: "It's very difficult to structure a balance between this overconfidence and constant doubt and I think it's still an open question." (58:19)
- Dmitri Jarnikov: "How do you explain to agents this very human concept that better is the worst enemy of good?" (54:39)

## Tools & references mentioned
- Prosus Group
- Token
- Devin
- Manus
- DeepSeek
- Arcade
- Composio
- MCP
- Salesforce
- monday.com
- Databricks
- SAP
- React
- Claude
- Gemini
- ChatGPT
- OSWorld
- Operator
- Fiverr
- Siri
- Frugal ML

## Who should watch
- You are building agents that send too many updates, make users repeat themselves, or fail on ordinary web interactions.
- Your team is deciding which agent infrastructure to build internally and which capabilities to obtain from vendors or standards such as MCP.
- You need practical design guidance for memory, planning, human handoffs, sandboxed code, stopping rules, or cost controls.

## Related talks

- [AI Agents: The Evolution of Our Digital Friends](https://mlopstalks.com/talks/ai-agents-the-evolution-of-our-digital-friends) (, 10:16)
- [Sub-Agent Architectures: What You Can Leverage](https://mlopstalks.com/talks/sub-agent-architectures-what-you-can-leverage) (Sidd Seethepalli, Vellum, 14:48)
- [AI Agents: The Future of Productivity, or Just a Fad?](https://mlopstalks.com/talks/ai-agents-the-future-of-productivity-or-just-a-fad) (Sam Partee, Arcade AI, 35:18)
- [Why Agents Are Stupid & What We Can Do About It](https://mlopstalks.com/talks/why-agents-are-stupid-what-we-can-do-about-it) (Dan Jeffries, Kentauros AI, 31:58)
- [AI Agent Development Tradeoffs You NEED to Know](https://mlopstalks.com/talks/ai-agent-development-tradeoffs-you-need-to-know) (Sherwood Callaway, 11X, 57:07)