# Building Multi-Player AI Systems (and why it's SO hard) Tula Masterman, MeshAgent | MLOps Community | 13:53 Source: https://www.youtube.com/watch?v=N-8akaeXtfA Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks. Page: https://mlopstalks.com/talks/building-multi-player-ai-systems-and-why-its-so-hard Published: 2025-09-15 Tags: agents, memory, multi-agent, observability, tool-use ## TL;DR - MeshAgent puts humans, agents, tools, and data sources into secure Rooms where they share live context and access controls. - Rooms start when a participant joins, stop when the last participant leaves, and keep an audit trail of messages, tool calls, and timing. - Teams can build agents with Python, TypeScript, JavaScript, or Dart, then test chat and voice agents in MeshAgent Studio. ## Summary Tula Masterman argues that most agent systems stop at one person working with one agent or with a team of agents. Real business work involves several people, agents, tools, and data sources contributing to one shared deliverable. MeshAgent addresses this with secure Rooms. A Room is a workspace and security boundary where human and agent participants share context, communicate through the same infrastructure, and work with shared documents. Masterman uses a health insurance company responding to an RFP as an example. Underwriters, actuaries, sales, legal staff, and specialist agents can work in the same client-specific Room. MeshAgent also provides SDKs, a browser-based Studio, tools, logging, observability, and infrastructure management. In the discussion, Masterman says teams can run anything from several Rooms to thousands, configure agent memory themselves, and connect telemetry through OpenTelemetry. The talk gives a practical picture of collaborative agents, while leaving memory design and operational integrations open to the builder. ## Key ideas ### Agent systems need to support whole teams, not just one user [00:58](https://www.youtube.com/watch?v=N-8akaeXtfA&t=58s) Masterman says many current systems focus on one person talking to one agent, or one person coordinating a group of agents. The harder case starts when an entire team needs to work with those agents and build something together. MeshAgent focuses on that multi-person setting. Its platform lets teams build, deploy, share, and operate agents collaboratively, while MeshAgent handles infrastructure and scaling behind the scenes. ### A Room creates one security boundary for people, agents, tools, and data [02:17](https://www.youtube.com/watch?v=N-8akaeXtfA&t=137s) MeshAgent groups humans, agents, tools, and data sources inside a secure Room. Context inside the Room stays there, and the Room becomes the place where a project team works. Masterman describes business processes as spanning several roles and systems, so participants need a shared connection point for exchanging information. The same communication infrastructure is used for human and agent teammates. ### Rooms are temporary workspaces that can preserve their outputs [04:01](https://www.youtube.com/watch?v=N-8akaeXtfA&t=241s) A Room starts when any human or agent participant joins and spins down when the last participant leaves. This avoids keeping infrastructure active when nobody is using it. Documents and other artifacts can be saved to external storage when they need to leave the Room. Masterman presents the Room as a scoped workspace rather than a permanently running environment. ### A shared RFP gives human specialists and agents one deliverable [04:57](https://www.youtube.com/watch?v=N-8akaeXtfA&t=297s) Masterman uses a health insurance company responding to an RFP to show how a Room could work. Underwriting, actuarial, sales, and legal staff can join the same client-specific workspace. Agents can help with risk or liability analysis through a custom AI model, then return information that document-writing agents can use in the RFP. Humans can check progress, provide missing information, and respond when an agent asks for help. ### MeshAgent records the work behind an agent's final output [07:58](https://www.youtube.com/watch?v=N-8akaeXtfA&t=478s) MeshAgent Studio records which tools agents called, which messages they sent, and how long operations took. This gives the team more than the final document or answer. It also provides an activity trail showing how the result was produced and where people contributed. Masterman connects this logging and observability to the team's ability to understand and review the work. ### Developers can build agents in several languages and test them in Studio [08:37](https://www.youtube.com/watch?v=N-8akaeXtfA&t=517s) MeshAgent provides Python, TypeScript, JavaScript, and Dart SDKs. Masterman says developers may not know the final shape of an application while they are building it, so they need a place to test and evaluate agents. In MeshAgent Studio, a human participant can share a Room with chat and voice agents. Those agents can use tools to write documents or save information to Room storage. ### Room scale is automatic, while agent memory remains a design choice [11:11](https://www.youtube.com/watch?v=N-8akaeXtfA&t=671s) In the question period, Masterman says Rooms can scale from around ten active workspaces to thousands, as long as participants are present. Teams can also configure a Room to stay open. Agent memory is configurable: builders can flush state, persist it when returning to a Room, or implement another approach. Masterman also points out that context window limits still apply. ### Operational integrations can use OpenTelemetry and custom logging [12:58](https://www.youtube.com/watch?v=N-8akaeXtfA&t=778s) Masterman says MeshAgent's telemetry currently runs through OpenTelemetry. The platform is intended to accept additional integrations and custom observability features when teams need them. This keeps the operational setup open to the monitoring systems an operations team already uses, rather than requiring one fixed view. ## Notable quotes - "I think the problem that a lot of people aren't addressing yet is what happens when my entire team shows up and I have multiple people who need to be interacting with these agents and actually all building something together collaboratively." (01:37) - "And we do all of this by uniting your teams, be that both humans and agents as well as tools as well as different data sources within this concept of a secure room." (02:17) - "So that way you have a really good understanding of not only the final product and output that the agent is producing, but what they performed every single step of the way." (07:58) - "You could have like 10 rooms running. You could have thousands of rooms running." (11:27) ## Tools & references mentioned - MeshAgent - MeshAgent Studio - Python - TypeScript - JavaScript - Dart - OpenTelemetry - LinkedIn ## Who should watch - You are designing an agent product where several employees need to work with the same agents and shared artifacts. - Your team needs client- or project-specific workspaces with access boundaries and a record of agent actions. - You are deciding how much of agent memory, telemetry, scaling, and infrastructure your application should own. ## Related talks - [The Future of AI: Building Multi-Agent Systems](https://mlopstalks.com/talks/the-future-of-ai-building-multi-agent-systems) (, 17:30) - [Autonomous Multi Agent AI Systems](https://mlopstalks.com/talks/autonomous-multi-agent-ai-systems) (, 14:41) - [A New Way of Building with AI](https://mlopstalks.com/talks/a-new-way-of-building-with-ai) (Jiquan Ngiam, Lutra AI, 1:04:46) - [9 Commandments Building AI Agents](https://mlopstalks.com/talks/9-commandments-building-ai-agents) (Paul van der Boor & Dmitri Jarnikov, Prosus Group, 1:20:34) - [Agentic Relationship Management](https://mlopstalks.com/talks/agentic-relationship-management) (Ashe Magalhaes, Hearth AI, 10:01)