Watch on YouTubeShorter prompts and outputs can reduce AI costs, but the useful limit depends on the model and task, so teams need a benchmark such as a proposed "CaveBench."
AI value becomes easier to discuss when teams attach token costs to a business unit metric, while keeping exploratory work separate from production workloads.
Agents should follow the same software delivery and governance controls as people, including proactive blocks, reactive checks, policy as code, and controlled deployment pipelines.
James Barney describes AI enablement at MetLife as connecting new AI capabilities to existing business problems across different laws, regulations, and customer rights. In the FinOps Foundation AI working group, he focuses on the trade-offs between model cost, speed, and accuracy. He supports measuring how far shorter prompts can go before quality drops, though the answer will vary by model and task. For AI economics, he recommends finding the unit metric that matters to each business stakeholder and tying it to the token cost of an outcome. Exploration can be treated like local development, while production workloads need separate budgets and controls. On governance, agents should face the same protections as humans, with policy expressed in code. James also argues that APIs, command-line tools, and agent interfaces may replace many dashboards, although dashboards still help people discover problems they did not know to ask about.
James says his team gives business partners the tools they need while accounting for laws, regulations, and customer rights across regions. The work connects new technology to existing business problems at a global insurer. Regional experts work with government regulators so the company can apply AI in a safe and reasonable way. James still writes code and reviews pull requests, including work on a GitHub action for agent skills.
The FinOps Foundation AI working group began with a practical question: which model should be used for which workload? James says the answer usually involves balancing the model's cost, speed, and accuracy for a specific task. The group publishes white papers and code repositories, including examples for token counting and chargeback. These materials are intended to start a conversation rather than act as a final authority.
Demetrios describes the caveman skill as using very few words to save input and output tokens. He wants to know how far this can go before the response quality falls off a cliff. James says the answer depends on the model, so a useful benchmark would compare models across different task complexities and amounts of back and forth. They jokingly call the proposed benchmark "CaveBench," alongside the model benchmarks people already discuss.
James says a million tokens can be assigned a price, but that does not explain what those tokens accomplished. New model releases also create uncertainty because developers may want access immediately while the provider is still learning about demand, regressions, and operational issues. He describes the launch of Anthropic's Fable model as resembling a test drive, with an initial price that may change as the model proves its value.
MetLife looks at whether AI avoids spending elsewhere and whether it helps the business reach outcomes faster. James says the second measure is difficult because the company does not produce one simple type of widget. He asks each stakeholder what their equivalent of a widget is. One person may accept a proposed north-star metric, while another will reject it and reveal a different metric tied to revenue or a business KPI. Collecting these unit metrics gives the team something to connect to AI spending.
James gives the example of a process that consumes $10 worth of tokens to produce one outcome. The business can decide whether that price is acceptable or ask for a lower cost. From there, the team can compare cheaper models, prompt-efficiency techniques, and other changes. A cheaper model may be slower or may need to run twice, so the apparent saving can disappear. The unit metric provides a goal for testing whether a change moved the workload closer to the desired result.
James recommends starting exploration with the smartest model available, then moving to a smaller model and adding harness or agent capability if the quality remains acceptable. He compares local experimentation to running tests on a laptop, while production has a separate pipeline and workload. A developer can have a monthly exploration budget, with task-oriented production usage tracked separately. He admits the boundary is becoming harder to draw because experiments can quickly become the product.
James divides governance into proactive prevention and reactive response. If a person cannot create a public cloud bucket, an agent should not be able to do it either. If a Kubernetes configuration is out of policy, a reactive control can flag or correct it. He recommends policy as code so an agent can read the rule directly, and he argues that agent actions should run through software delivery pipelines rather than manual clicks. If a capability can be described in a document or configuration file, it should be controlled there.
James says a person may no longer need a dashboard showing the top token-consuming applications if an agent can query the underlying API or command-line tool. An MCP app could collect information from security, FinOps, and cloud configuration systems and bring it into one conversation. The limitation is the latent shopper problem: a dashboard can expose a spike or pattern that nobody thought to ask about. An LLM can inspect known data, but people may still need direct contact with the data to discover new questions.
"If you remove the word agent from all this AI agent, get rid of it, the same rulebook sort of applies for cloud provisioning as it did for letting humans provision stuff."James Barney34:00