# Simulation Techniques for AI Agents from Self-Driving

Brooke Hopkins, Coval | MLOps Community | 33:58
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=zEvzPoodA08
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/simulation-techniques-for-ai-agents-from-self-driving
Published: 2024-12-17
Tags: agents, evals, reliability, testing

## TL;DR
- AI agent evaluation should move from fixed prompt-and-output tests toward probabilistic simulation across many dynamic scenarios.
- Teams can isolate layers of an agent, such as voice, language, and external systems, to balance cost, latency, and evaluation signal.
- Reliable agents need fallback mechanisms, redundancy, and monitoring because errors can compound across multiple steps.

## Summary
Brooke Hopkins compares autonomous AI agents with self-driving cars. Both perceive an environment, make decisions, and act across a sequence of changing situations. She argues that manual testing and static expected outputs are too slow for agents, especially when a task can be completed through many valid paths. Instead, teams should simulate large numbers of dynamic scenarios and measure aggregate outcomes alongside individual failures. Hopkins also describes testing different layers independently, such as evaluating an agent's logic without simulating voice or mocking websites while testing conversation behavior. This lets teams manage the tradeoff between cost, latency, and signal. She recommends using human behavior as a reference, with metrics such as task duration and abstract action steps. Since agent errors can compound, reliable systems need redundancy and graceful fallback. Her strongest recommendation is to design toward full autonomy from the start, because relying on a human in the loop can make reliability harder to measure and improve.

## Key ideas
### Agents can be tested using lessons from self-driving systems
[00:45](https://www.youtube.com/watch?v=zEvzPoodA08&t=45s)
Hopkins defines an agent as an autonomous system that perceives its environment and responds to it. A self-driving car navigates from point A to point B while reacting to the world around it. An AI agent does something similar when it interacts with a user, voice interface, website, or API. She draws on her previous work leading evaluation infrastructure at Waymo, where her team launched simulations, maintained datasets, and ran them on distributed compute.

### Static pass-fail tests miss how generative systems behave
[04:37](https://www.youtube.com/watch?v=zEvzPoodA08&t=277s)
Traditional software testing uses unit tests, integration tests, regression tests, and release tests with clear pass-or-fail results. Traditional machine learning also relies on aggregate measures such as F1 score, precision, and recall. Foundation models require both aggregate statistics and attention to individual examples. A self-driving system may need good overall collision statistics while still being checked on whether it stopped at a particular stop sign. Agent evaluation has the same tension.

### Large-scale probabilistic evaluation is more practical than maintaining every case
[05:59](https://www.youtube.com/watch?v=zEvzPoodA08&t=359s)
Self-driving evaluation moved away from manually authored scenarios with one exact expected result. Teams can generate cases from logs or synthetic data, simulate thousands of them, and measure events such as reaching a destination, hard braking, or collisions. Hopkins applies this idea to agents. Rather than checking one appointment-booking prompt, teams could run many variations and measure how often the goal is achieved with an LLM judge or heuristics. The result is a test suite that can change as the agent changes.

### Simulation should isolate the layers that matter for a question
[08:08](https://www.youtube.com/watch?v=zEvzPoodA08&t=488s)
Self-driving stacks contain sensors, perception, localization, prediction, planning, and control. A useful simulation does not always need to reproduce every layer. For a voice agent, text may be enough to test logical consistency. For an AI sales development representative, a team might evaluate the conversation while mocking websites and web agents. Isolating components helps engineers determine whether a problem comes from transcription, reasoning, or another part of the system.

### Evaluation balances cost, latency, and signal
[11:04](https://www.youtube.com/watch?v=zEvzPoodA08&t=664s)
Hopkins describes a three-way tradeoff in simulation. Running more tests can provide more signal, but it increases cost. Using cheap compute can reduce cost, but it may make tests too slow. Running no tests produces low cost and low signal. The right setup depends on the evaluation need. Teams can start with the layer and scenario that provide useful information instead of simulating the entire system for every experiment.

### Human behavior provides a reference even when many paths are correct
[11:51](https://www.youtube.com/watch?v=zEvzPoodA08&t=711s)
There may be many valid ways to book an appointment or drive between two points. Hopkins suggests comparing an agent with human behavior at an abstract level. An LLM judge can identify the steps the agent took and how long it took to reach the goal. Human labels can describe a typical sequence, such as collecting an email address, collecting a phone number, and promising a callback. The comparison can reveal that an agent is repeating itself or going around in circles without requiring one exact answer.

### Reliability requires fallback paths because agent errors compound
[14:55](https://www.youtube.com/watch?v=zEvzPoodA08&t=895s)
An early mistake in a conversation can cause later mistakes, which makes an agent look unreliable when viewed only as a single chain. Hopkins compares this with software infrastructure, where servers, networks, packet delivery, and web applications can all fail. Systems handle that risk with redundancy and fallback mechanisms. Agents can use multiple model calls, cached examples, or non-generative systems to recover when the first response is wrong. The goal is for the system to fail gracefully rather than depend on one perfect prompt.

### Dynamic simulations reveal coverage without breaking when behavior changes
[20:19](https://www.youtube.com/watch?v=zEvzPoodA08&t=1219s)
Hopkins argues for a testing environment that gives an agent dynamic input as it makes decisions. A simulated conversation can respond to the agent rather than replaying a fixed transcript. This keeps scenarios useful when agent behavior changes. The system can also show which paths the agent took and which paths were exercised most often. Production examples, including unexpected speech, irrelevant details, or background noise, can be replayed as trajectories that respond dynamically to the current agent.

## Notable quotes
- Brooke Hopkins: "I think we should be doing the same thing with agent, I think right now people are too focused on doing individual test cases." (07:28)
- Brooke Hopkins: "You can make every you can get the most amount of signal by running an infinite amount of tests but then you'll have very high costs." (11:04)
- Brooke Hopkins: "I think actually you should shoot for full autonomy from the beginning because it forces you to think about what do we do for fallback mechanisms if the agent fails." (18:48)
- Brooke Hopkins: "We need to create a new testing paradigm that's much closer to robotics." (20:19)

## Tools & references mentioned
- Coval
- Waymo
- Stanford
- University of California, Berkeley
- OpenAI
- ElevenLabs
- Tesla
- WebRTC

## Who should watch
- You are testing a conversational or web agent manually and need a way to run many variations of the same task.
- Your team needs to separate failures in speech, reasoning, external tools, and other agent layers.
- You are deciding how much autonomy, fallback behavior, telemetry, and simulation to build into a production AI system.

## Related talks

- [Simulate to Scale: How realistic simulations power reliable agents in production](https://mlopstalks.com/talks/simulate-to-scale-how-realistic-simulations-power-reliable-agents-in-production) (Sachi Shah, Sierra, 21:10)
- [The Science of Improving AI Agents](https://mlopstalks.com/talks/the-science-of-improving-ai-agents) (, 26:43)
- [The Truth About AI Agents](https://mlopstalks.com/talks/the-truth-about-ai-agents) (Silen Naihin, AutoGPT, 31:40)
- [Evaluating AI Agents: Why It Matters and How We Do It](https://mlopstalks.com/talks/evaluating-ai-agents-why-it-matters-and-how-we-do-it) (Annie Condon & Jeff Groom, Acre Security, 13:27)
- [AI Agents for Consumers](https://mlopstalks.com/talks/ai-agents-for-consumers) (Shaun Wei, RealChar, 57:27)