# Responsible Autonomy: Building Governance Frameworks for AI That Act in the Real World via MCP

Saurabh Mishra, Optum | MLOps Community | 27:55

Source: https://www.youtube.com/watch?v=OCer65weyTk
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/responsible-autonomy-building-governance-frameworks-for-ai-that-act-in-the-real
Published: 2026-08-19
Tags: governance, mcp, observability, security

## TL;DR
- AI agents need governance around identity, permissions, policies, human approval, and accountability before they can safely act in enterprise systems.
- MCP gives agents a standard way to discover tools, access knowledge, and invoke APIs, while agent gateways and registries control how those tools are used.
- Safe agent execution requires isolation, short-lived credentials, least-privilege access, network controls, testing in simulated environments, and observability.

## Summary
Saurabh Mishra explains how AI has moved from predictive models and chatbots toward agents that can reason, use tools, and perform actions in business systems. MCP provides a standard way for agents to communicate with tools and services, but this creates risks when agents can change infrastructure, access CRM or payroll systems, or trigger other workflows. Mishra presents governance as a set of controls around the agent lifecycle. These include agent gateways, individual agent identities, scoped permissions, approved-agent registries, policy enforcement, model input and output filtering, sandboxing, simulations, evaluations, and observability. He uses the Gemini Enterprise agent platform as an example of a system organised around building, scaling, governing, and optimising agents. In the question period, he describes isolated micro VMs with separate kernels, short-lived OAuth or OIDC tokens, least-privilege access, and network security as layers for containing sandbox failures.

## Key ideas
### Agents have moved from generating answers to taking actions
[03:11](https://www.youtube.com/watch?v=OCer65weyTk&t=191s)
Mishra describes a progression from chatbots that answer questions, through copilots that assist developers and business users, to autonomous agents that make decisions and interact with enterprise applications. MCP lets an agent discover tools, access enterprise knowledge, invoke APIs, and work across systems. That capability changes the risk. An agent connected to cloud infrastructure could provision resources, rotate secrets, approve pull requests, or update production configuration. A poorly governed decision could cause a security breach, create compliance problems, or damage an organisation's reputation.

### Responsible autonomy begins with policy, identity, and human approval
[04:34](https://www.youtube.com/watch?v=OCer65weyTk&t=274s)
Mishra defines governance as the group of controls needed when agents act through MCP. Organisations need policies that describe what an agent may do, identity and access controls that bind actions to a specific agent, and explicit decisions about when a person must approve an action. He presents accountability as part of the same control loop. The agent's ability to perform an action is therefore only one part of deployment. The organisation also needs boundaries around the tools and services the agent can reach.

### An agent observes, decides, and acts
[11:12](https://www.youtube.com/watch?v=OCer65weyTk&t=672s)
Mishra breaks an agent into three stages. First, it observes by gathering information from user input, APIs, sensors, or other sources. It then thinks and decides whether the gathered information is relevant and what response or action follows. Finally, it acts by generating content, calling a tool, or executing a task. He connects this model to the progression from retrieval-augmented generation to API execution, workflows, and multi-agent systems. MCP enters when agents need to expose or use real-world tools through a common interface.

### The Gemini Enterprise agent platform groups agent work into four pillars
[13:30](https://www.youtube.com/watch?v=OCer65weyTk&t=810s)
Using Google's Gemini Enterprise agent platform, formerly Vertex AI, Mishra describes four pillars: build, scale, govern, and optimise. The build area includes an agent development kit, third-party frameworks such as LangGraph, Crew AI, and LangChain, a visual Agent Studio, and reusable templates in Agent Garden. The platform can work with Gemini, OpenAI, Mistral, Anthropic, and DeepSeek models. Mishra says the platform also connects with MCP-enabled cloud services such as Cloud Run and Google Kubernetes Engine.

### Production scale needs runtime controls and persistent state
[17:58](https://www.youtube.com/watch?v=OCer65weyTk&t=1078s)
Mishra says building an agent is easier than running large numbers of agents in production. His scaling examples include an agent runtime for secure execution, sessions for maintaining conversation and execution context, sandboxes for safely running tools or code, and an agent memory bank for persistent or long-term state. He gives the example of a sequential agent whose output becomes the input for a second agent. The platform must preserve enough state to pass that information between sessions.

### Governance controls define who an agent is and what it can reach
[19:20](https://www.youtube.com/watch?v=OCer65weyTk&t=1160s)
The governance pillar includes an agent gateway, agent identity, an agent registry, agent policies, and model protection. The gateway handles authentication, authorisation, rate limiting, routing, and policy enforcement. Each agent can receive its own short-lived credentials, least-privilege access, or delegated permissions instead of using a person's credentials. The registry records approved agents, their builders, models, tools, and ownership. Policies can limit access, such as allowing an HR agent to reach payroll services while preventing a technical agent from doing so. Model Armor filters harmful prompts and unsafe outputs before they reach enterprise systems.

### Testing and observability measure whether an agent is safe and useful
[22:10](https://www.youtube.com/watch?v=OCer65weyTk&t=1330s)
Mishra places evaluation, simulation, optimisation, and observability in the optimisation pillar. Evaluations measure accuracy, hallucination rates, relevance, tool success, and safety metrics. Simulations test an agent in an environment similar to production before release. An optimiser can recommend changes to planning, tool selection, performance, security, compliance, and cost. Observability covers token use, latency, cost, error rates, and business KPIs, in the same way infrastructure observability tracks service spans and failures.

### Sandbox isolation needs several independent controls
[25:35](https://www.youtube.com/watch?v=OCer65weyTk&t=1535s)
In response to a question about agents escaping sandboxes, Mishra describes isolated containers or micro VMs. Kata Containers give each micro VM its own kernel, so an affected environment does not directly affect another container through a shared kernel. He adds short-lived OAuth or OIDC tokens, least-privilege access, network security, agent policies, agent identity, and an agent gateway. His answer uses several layers because no single authentication control prevents every kind of sandbox failure.

## Notable quotes
- "Just because an AI agent can perform exercises, that is the only way that thing is looking for." (03:55)
- "So, all these accountabilities come under the one single loop known as the governance." (05:13)
- "Building the agent is very easy, but running thousands of enterprises in the brain is very difficult hard part for anyone." (17:58)
- "So, every agent is going through via agent gateway." (19:38)
- "Each container having own micro VM and those micro VM having the own kernel." (26:15)

## Tools & references mentioned
- MCP
- Model Context Protocol
- Agentic AI Foundation
- CNCG
- AIFF
- Gemini Enterprise agent platform
- Vertex AI
- Gemini
- OpenAI
- Mistral
- Anthropic
- DeepSeek
- LangGraph
- Crew AI
- LangChain
- Cloud Run
- Google Kubernetes Engine
- Kubernetes
- Kata Containers
- OAuth
- OIDC
- Model Armor
- RAG

## Who should watch
- You are building MCP-connected agents that can change infrastructure, update business systems, or execute code, and need a concrete set of controls around those actions.
- Your team is moving an agent from a prototype into production and needs guidance on identities, sandboxes, evaluation, observability, and policy boundaries.
- You want to understand how agent gateways, registries, short-lived credentials, and human approval fit into one governance model.

## Related talks

- [MCP is not going to change everything (yet)](https://mlopstalks.com/talks/mcp-is-not-going-to-change-everything-yet) (Sam Partee, Arcade AI & Rahul Parundekar, AI Hero, 1:04:43)
- [AI Agents Are About to Get Real, Here's Why It Matters](https://mlopstalks.com/talks/ai-agents-are-about-to-get-real-heres-why-it-matters) (Sam Partee, Arcade AI, 13:20)
- [Enterprise-ready MCP](https://mlopstalks.com/talks/enterprise-ready-mcp) (Jiquan Ngiam, MintMCP, 28:36)
- [MCP Security: What Happens When Your Agents Talk to Everything?](https://mlopstalks.com/talks/mcp-security-what-happens-when-your-agents-talk-to-everything) (, 24:26)
- [Governance for AI Agent Deployment](https://mlopstalks.com/talks/governance-for-ai-agent-deployment) (Spencer Reagan, Airia, 54:18)