Reliable agents need explicit business context, expert rules, evaluation, and operational safeguards around the model.
2
Subject matter expert corrections should become versioned context updates and regression tests in a living ground truth loop.
3
Separating model reasoning from business logic lets teams update policies, test changes, and replace models without retraining or changing prompts.
Summary
Sanjana Sharma argues that model benchmarks do not predict whether an agent will work in production. Real workflows contain ambiguous requirements, conflicting records, missing fields, undocumented rules, and changing processes. Teams should define the work precisely, capture expert context, and build evaluation that changes as experts correct the system. In a healthcare example, encoding clinicians' undocumented rules raised reliability from 73% to 91% without changing the model. Sharma compares agent operations with software engineering practices such as unit tests, version control, code review, CI/CD, and audit logs. Her architecture separates the language model from a versioned context layer containing rules, workflows, regulations, and exceptions. Production failures become test cases, expert corrections become context updates, and changes ship only after the full regression suite passes. This creates a living ground truth that improves system reliability while the underlying model stays fixed.
Production reliability depends on context around the model
Sharma describes a gap between strong benchmark results and brittle enterprise deployments. Production introduces edge cases, ambiguous requirements, conflicting data sources, and users who do not follow the happy path. Assistive systems can remain human-supervised, while fully autonomous agents work best in clean, rule-heavy settings. The difficult work sits in the middle, where judgment and context matter. Sharma says teams need to move from model-first thinking to system-first thinking, with explicit rules, versioned routines, and disciplined evaluation.
Organizations must prepare the work before automating it
Before trusting an agent, a company needs clear processes, defined rules, consistent data, and subject matter experts who agree on what good looks like. Sharma frames this as an operating manual for an AI workforce. The work definition must be precise, since vague tasks create failures. Context preparation must capture unwritten expert rules. Evaluation must change when experts correct outputs, creating what she calls a living regression test rather than relying only on an initial annotated dataset.
Undocumented expertise is the missing layer of business context
Sharma divides business context into structured data, unstructured material such as documents and emails, and undocumented expertise. Agents can generally access the first two, although quality varies. The third layer contains the tacit rules that experienced people apply every day. She says companies often provide the first two layers and then wonder why reliability stops around 70 to 80 percent. In her view, the missing expert knowledge determines whether an agent behaves predictably.
Clinical rules raised reliability without changing the model
In a healthcare claims example, an agent was stuck at 73% reliability because of timeline inconsistencies, conflicting EHR fields, and edge cases. The team worked with clinicians to uncover rules that had never been written down. One rule treated an admission less than eight hours before surgery as a surgical admission, even when a field was missing. Another handled a vital-sign record that continued past midnight. The team encoded these rules as context, test cases, and regression checks. Reliability rose from 73% to 91% with the same model.
Agent operations should borrow software engineering controls
Sharma compares reliable agent development with software engineering. Software teams define the work, prepare context through repositories and design documents, and review outputs with unit tests, version history, code review, CI/CD, logging, and debugging. Agents need equivalents: evaluation cases for expected outputs, versioning for context and expert rules, subject matter expert review, automated regression tests from production failures, and decision audit trails that record reasoning and tool calls. These controls make failures inspectable and changes safer.
A production agent is a whole system, not a model wrapper
Sharma recommends designing from a specific workflow and its edge cases rather than starting with a model's capabilities. Her production stack begins with a clear use case, measurable success criteria, and business agreement. It then covers user interaction and feedback, the full solution architecture, engineering and operations, and change management. Users need training, gradual rollout, and clear escalation paths. Teams often build impressive middle-layer architecture while leaving the business problem and user trust unresolved.
Gold datasets miss the failures that matter in production
Static gold datasets provide clean metrics and easy comparisons, but they assume production data will be clean and unambiguous. Sharma lists date-format differences, expert disagreements, missing fields, upstream API changes, and undocumented legacy rules as sources of failure. An 80% score can hide regulatory violations, customer-facing errors, and downstream failures. In healthcare, she says 80% accuracy is catastrophic. Evaluation must cover the full distribution of production complexity rather than only the happy path.
Sharma's architecture keeps the language model as a reasoning engine while placing procedural knowledge, regulations, workflows, heuristics, exceptions, and expert rules in a separate versioned context layer. Meta AI agents can turn subject matter feedback into structured updates, which experts review like code. Teams can replace the model without changing the business logic, update policies without retraining, compare or roll back context versions, and test every update. She attributes a move from weekly firefighting to stable monthly refinement to this architecture.
Living ground truth turns failures into durable improvements
The living ground truth loop starts with a model, context rules, and a baseline evaluation suite. Production traces reveal failures, inconsistencies, and expert disagreements. Corrections become structured context updates, while failing cases become regression tests. The team applies the update and reruns the full suite; if a historical test fails, the change does not ship. Once all tests pass, the new context version is promoted. Reliability improves because the context becomes richer and more aligned with expert judgment, even when the model and prompts stay unchanged.
"You can't always trust an agent if your business context is not ready for automation."04:18
Who should watch
You are deploying agents in healthcare, finance, logistics, or another setting where a small number of wrong decisions creates serious harm.
Your team has benchmark scores but still spends time investigating ambiguous production failures and undocumented business rules.
You need a practical design for expert feedback, context versioning, regression tests, and audit trails without retraining the model for every policy change.
