# The Cost of AI: FinOps Strategies for Intelligent Agents

Advait Patel, Broadcom | Agents in Production 2025 | 31:05
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=hIwqH3yAS5E
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/the-cost-of-ai-finops-strategies-for-intelligent-agents
Published: 2025-09-19
Tags: agents, cost, guardrails, observability

## TL;DR
- AI agents can create major cloud costs when retries, API calls, permissions, logging, or autoscaling have no limits.
- Teams can control agent spending with resource limits, cost alerts, sandboxes, kill switches, simulations, tagging, and scheduled cost reviews.
- Advait Patel's Track, Throttle, Train, and Terminate framework combines observability, restricted permissions, cost-aware behavior, and cleanup of idle or misbehaving resources.

## Summary
Advait Patel explains how autonomous AI agents can quietly increase cloud spending. An agent may retry forever, reload full context on every step, retain logs indefinitely, misuse autoscaling, or use permissions to start expensive services. He describes a support bot that created about 10,000 tickets in one day because of a prompt loop and no rate limit. The fix was throttling and guardrails. Patel recommends resource limits, cost alerts, sandbox testing, kill switches, circuit breakers, cost simulations, tagging, and regular cost reviews. His Track, Throttle, Train, and Terminate framework gives teams a simple operating model: observe agent behavior, restrict access, teach agents to use resources efficiently, and remove idle or misbehaving resources. He also discusses policy as code with Open Policy Agent, cost estimation, cloud-native monitoring, regional pricing, and prompts that account for token cost. The central point is practical: agent spending is often a visibility and governance problem that must be addressed before production.

## Key ideas
### Autonomy can increase cloud spending when agents have no stopping conditions
[01:26](https://www.youtube.com/watch?v=hIwqH3yAS5E&t=86s)
Patel says agents are appearing in data pipelines, security monitoring, and automation bots, where they are expected to make decisions and save time. Autonomy does not mean optimization. He has seen agents fire thousands of API calls, start unnecessary GPU instances, and loop indefinitely after a missing condition check. A stated survey figure says around 30% of generative AI workloads blow through their budget, usually because someone failed to add brakes. The cost problem often begins with an ordinary design omission rather than an unusually powerful model.

### Retries, context, logs, autoscaling, and permissions create recurring cost leaks
[03:31](https://www.youtube.com/watch?v=hIwqH3yAS5E&t=211s)
Patel lists several common causes of agent overspending. Agents may use infinite retry loops or reload the full context for every request, increasing bandwidth and model costs. They may retain debug logs forever, misuse autoscaling policies, or hold permissions that let them start expensive services. Teams also lack visibility into what agents are doing. His description connects operational settings directly to billing: retention, scaling, context handling, and access control all affect the cloud bill when an agent runs continuously.

### A support bot showed how a small control can prevent a large bill
[05:45](https://www.youtube.com/watch?v=hIwqH3yAS5E&t=345s)
Patel describes a support GPT bot intended to generate help desk tickets for support engineers. A prompt loop and the absence of a rate limit caused it to create approximately 10,000 tickets in one day. The operation increased cost by four times. Patel says the incident was avoidable, and that guardrails and throttling were the fix. The example gives his broader argument a concrete shape: an agent can perform a seemingly harmless task at a damaging scale when its execution is not bounded.

### Cost control starts with limits, visibility, testing, and ownership
[06:57](https://www.youtube.com/watch?v=hIwqH3yAS5E&t=417s)
Patel's cloud-native playbook begins with CPU, memory, and concurrency limits. He recommends cost alerts, sandbox environments, kill switches, circuit breakers, and policies before agents reach production. Cost simulations and cloud cost explorers cannot predict the exact bill, but they can provide a range. Tags and labels show which team owns a workload and which inference or agent is using resources. Patel also recommends recurring reviews. His team reviews costs every two weeks and, for longer workloads, nightly. He frames this as operational discipline rather than a requirement for expensive third-party tooling.

### Track, Throttle, Train, and Terminate is Patel's operating framework
[10:54](https://www.youtube.com/watch?v=hIwqH3yAS5E&t=654s)
Patel summarizes his approach as four actions. Track agent behavior with logs, alerts, and observability. Throttle execution and begin with least-privilege access instead of giving agents unlimited power. Train agents to use resources more efficiently by learning how long executions take and what they consume. Terminate idle, orphaned, misbehaving, or misconfigured resources. He says this loop can prevent about 85% to 90% of cost leaks when it is performed consistently. The framework is meant to be easy to remember and apply to agents and workflows.

### Policy as code can block expensive actions before they happen
[16:03](https://www.youtube.com/watch?v=hIwqH3yAS5E&t=963s)
Patel recommends Open Policy Agent and GCP policy tools for enforcing agent rules. One example blocks agents from starting GPUs unless a specific permission has been granted. Other policies can limit which roles agents assume and which actions they take. The aim is to encode boundaries rather than trust an agent to make the right decision. Patel describes a Terraform example that implements such a policy. This approach gives teams a direct way to prevent an agent from exceeding its allowed permissions or starting resources that its task does not require.

### Budget-aware agents should account for token cost and regional pricing
[17:59](https://www.youtube.com/watch?v=hIwqH3yAS5E&t=1079s)
Patel recommends putting cost awareness into prompts or the agent planner and weighing token cost against accuracy. He also suggests distributing work to cheaper cloud regions when customer location does not require the agent to run in a particular region. APIs can check regional pricing before tasks are assigned. In his comparison, this is like managing a team: the right agent should do the right job in a cost-effective location. The decision belongs in the agent's planning process rather than being left entirely to the cloud bill.

### Five checks belong in place before an agent reaches production
[19:53](https://www.youtube.com/watch?v=hIwqH3yAS5E&t=1193s)
Patel's deployment checklist asks whether CPU and memory limits exist, rate limits and retries are configured, logging retention and rotation are set, budget alerts are enabled, and a kill switch is available. A sandbox or development environment is not automatically cost-free, so the checks apply before pushing an agent into any environment. If a team cannot answer yes to all five items, Patel says it is taking a large risk. Additional checks can be added for a particular use case, but these are his baseline controls.

### Scraped content needs filtering before it reaches the model context
[29:09](https://www.youtube.com/watch?v=hIwqH3yAS5E&t=1749s)
In the discussion after the talk, Patel addresses web scrapers that send irrelevant page content into the model. He says language models do not know which retrieved material is garbage unless the system defines that distinction. A guardrail or policy can identify patterns such as specified X, Y, and Z conditions, mark the content as garbage, and stop it from moving to the next step. Without that policy, the agent can take everything it scraped and inject it into the model context, increasing input tokens and cost.

## Notable quotes
- Advait Patel: "Just because something is autonomous, that doesn't mean it is optimized." (02:26)
- Advait Patel: "Don't trust the agent to do the right thing, but enforce it with code." (16:49)
- Advait Patel: "AI agents are powerful, that is no doubt in it, but without boundaries they are very dangerous." (22:24)
- Advait Patel: "Most budget issues aren't tech-related issues, but they are visibility and governance issues." (22:46)
- Advait Patel: "Smarter agents need smarter boundaries as well." (23:08)

## Tools & references mentioned
- Broadcom
- DockSec
- AWS
- GCP
- Terraform
- FinOps.ai
- CAST AI
- Kubecost
- Prometheus
- Grafana
- Wavefront
- Open Policy Agent
- OPA
- GCP's policy tools
- GCP's recommender tools
- AWS Compute Optimizer
- LLM

## Who should watch
- You are deploying agents that can call APIs, start cloud resources, retry tasks, or write large amounts of data, and you need concrete spending controls.
- Your team has cost alerts but cannot explain which agent, workload, or inference generated a bill.
- You are building an agent and want a pre-production checklist covering limits, retries, logging, permissions, and shutdown controls.

## Related talks

- [Cutting Costs with Artificial Intelligence](https://mlopstalks.com/talks/cutting-costs-with-artificial-intelligence) (, 12:36)
- [The Future of Compute: How AI Agents Are Reshaping Infrastructure](https://mlopstalks.com/talks/the-future-of-compute-how-ai-agents-are-reshaping-infrastructure) (Diego Oppenheimer, Guardrails AI, 38:34)
- [How to Actually Use Cost Effective AI in Your Business](https://mlopstalks.com/talks/how-to-actually-use-cost-effective-ai-in-your-business) (Eddie Mattia, Outerbounds & Scott Perry, AWS, 49:05)
- [Autonomous Agents at Work: From OpenClaw Hype to Enterprise Reality](https://mlopstalks.com/talks/autonomous-agents-at-work-from-openclaw-hype-to-enterprise-reality) (Pramod Krishnan, PwC, 42:20)
- [Omnigent: Composition, Control, and Collaboration for AI Agents](https://mlopstalks.com/talks/omnigent-composition-control-and-collaboration-for-ai-agents) (Denny Lee, Databricks, 58:17)