AI agents need evaluation beyond ordinary software testing because their non-deterministic decisions are difficult to inspect from user-visible results alone.
2
Acre Security versions prompts, instructions, code, tools, and evaluation logs for each agent.
3
Small teams can use existing evaluation and observability tools, then tune metrics to their own business needs instead of building everything themselves.
Summary
Annie Condon and Jeff Groom explain how Acre Security evaluates AI agents used in physical access control. They describe agents as software systems with non-deterministic loops involving one or more language models, which makes it hard to know why an agent produced a result. Evaluation gives the team an inside view of prompts, tool calls, ordering, outputs, and other decisions. Acre versions the prompts, instructions, code, tools, and evaluation logs associated with each agent. Condon demonstrates Logfire for tracing individual agent runs and Confident AI, backed by DeepEval, for running evaluations, reviewing metrics, and troubleshooting individual cases. The team uses built-in metrics where they fit and creates custom metrics for its own use cases. Groom argues that evaluation can determine whether an AI product succeeds, so teams should use existing tooling when possible and tie scoring closely to the business workflow.
Agent evaluation is needed because agent behavior is non-deterministic
Acre Security builds AI agents for physical access control, where installers, operators, and enterprise users may interact with systems through natural language. Annie Condon says agents are still software systems and need at least the level of testing used for traditional software. Jeff Groom describes an agent as a loop in which the system may call one or more language models several times, with decisions that are not always visible. A user-visible answer does not explain which decisions or tool calls produced it. Evaluation provides the equivalent of an X-ray, allowing the team to inspect what happened inside the run.
Metrics and evaluation frameworks must account for the agent's particular behavior
Agent evaluation shares some methods with software and machine learning testing, but the non-deterministic parts change what needs to be measured. Condon says teams must choose metrics deliberately, and that these metrics can be more creative than conventional tests because the problem is different. She also separates agent tools from evaluation tooling. The latter makes the process systematic. The evaluation framework needs to support the metrics that matter for the application rather than treating every agent as the same kind of system.
Acre versions every component that can change an agent's behavior
Acre versions the prompts, instructions, code, and tools used by an agent. The team also stores evaluation logs. This gives it a record of what was tested and makes changes easier to inspect. Versioning covers both the code and the surrounding instructions that shape an agent run. The logs add evidence about how the agent behaved, rather than leaving the team to judge quality from a few successful interactions.
Logfire lets the team inspect individual agent runs
Acre uses Logfire, built by Pydantic, to drill into agent runs. The system records the duration of a run, the system prompt, the user's prompt, and the agent's output. It also shows which tools the agent used and the order in which it used them. Condon illustrates this with a math agent handling the prompt "2 + 2 * 4," where the trace breaks down the agent's actions and tool calls. Acre uses these traces before deployment and in production, where they can help the team trace the results of individual tools.
Confident AI and DeepEval make evaluation practical for a small team
Acre uses Confident AI and its open-source DeepEval library to trace evaluation runs. The system gives an overview of how many cases passed and how the agent scored against selected metrics. The team can then open individual evaluation cases to troubleshoot them. DeepEval includes built-in metrics, especially for conversational agents, and also allows custom metrics. Condon says this matters because Acre's evaluation team consists of Annie, Jeff, and one other developer. Existing tools let them run quality evaluations quickly without building the whole platform themselves.
Evaluation tooling has to connect scores to the business workflow
Jeff Groom says evaluations can determine whether an AI project or product succeeds. He recommends using commercial or existing tools when a small team cannot build its own platform, while retaining control over the metrics. The metrics and scoring need to be tuned to the company's business case and tied closely to the business loop. A general evaluation suite is not enough if its measures do not reflect what the agent must actually do for users.
"You want to try to find something that you don't have to reinvent the wheel on the Eval platform itself, but that you can fine-tune the metrics so that they work for your business case."Jeff Groom09:35
Who should watch
You are building an AI agent and need to inspect tool calls, prompts, outputs, and decisions before putting it into production.
Your team is small and you want a practical evaluation process without building an evaluation platform from scratch.
Your existing metrics do not reflect the actual business workflow your agent supports.
