A useful evaluation framework starts with a small set of high-quality examples rather than weeks of work before the first test.
2
Evaluation needs to evolve along with the agent, including its metrics, data, rubrics, and framework.
3
LLM-as-a-judge is most useful early when its questions are simple enough to match a clear expected behavior.
Summary
Mariana Prazeres explains why the last step from an AI demo to a reliable product is difficult. Small prompt changes can make a demo look nearly finished, but agents still fail inconsistently, and fixing one behavior can break another. She proposes an evaluation loop that defines success, runs the agent, diagnoses failures, and changes the agent. The loop itself also needs revision. Product goals change, examples become stale, and metrics may measure academic performance instead of user satisfaction. Mariana recommends starting with a small, manually curated dataset and simple checks, such as verifying a number or checking whether the agent uses the correct name. Teams can later add production examples, more judges, CI runs, tracked scores, categorization, and human review. Her advice is to build the smallest useful framework today and improve it as the product becomes clearer.
Reliable agents need an evaluation loop because demos hide inconsistent behavior
Mariana says a simple prompt can make an AI product feel 90% complete, while moving from that demo to a reliable system is the hardest part. The agent may work in some cases and fail in others, and fixing one bug can break another behavior. An evaluation loop defines success, runs the agent, examines the results, and guides improvements. The definition of success can also change as the product and the agent's behavior develop.
Evaluation projects fail when teams delay testing or lose contact with the product
Teams can spend weeks building an evaluation framework before running it, rely on informal judgment, or create a framework that nobody checks again. They can also abandon it when the scores look good but the product experience still feels poor. Structural problems include evaluation data that does not match real product use, metrics that measure the wrong thing, and engineers evaluating without product context. A framework can also become stale as the AI feature changes.
Iteration applies to metrics and data as well as prompts and models
Mariana separates iteration on the agent from iteration on the evaluation itself. If the product goal changes, the score and rubric should change too. Metrics do not need to be perfect at the beginning, since a simple measure is more useful than having no measure. Evaluation examples must also change as goals change. A dataset can start with five or ten examples, may omit expected outputs when nondeterministic behavior makes them difficult, and should generally avoid synthetic data early on.
A small evaluation can give an early product a useful amount of feedback
In the early stage, Mariana recommends a small set of high-quality examples curated by people who understand what the agent should do. The checks can be simple Python scripts, such as testing whether an output equals a required number, or basic LLM-as-a-judge questions such as whether the agent addressed the user by the correct name. The purpose is to avoid being completely blind while the product is still taking shape.
Evaluation systems can grow from manual examples into tracked production feedback
As the product develops, teams can grow from a few examples to hundreds of real-world examples taken from production. The evaluation may use several LLM judges, run in CI or on a schedule, and track scores over time. Mariana mentions LangSmith as an example of tooling that can support this loop. At this stage, evaluation has a visible purpose, although parts of the process may still be manual.
Advanced evaluation features should follow the product instead of arriving first
A more advanced system may categorize thousands of examples, add humans in the loop, and fine-tune after correcting examples that failed in production. Deployment could then depend on passing evaluation thresholds. Mariana cautions that this is only one possible path. Some products need only a few examples, while others need many examples and metrics or one strong metric over a large dataset.
The right framework cannot be known before the agent has evolved
Mariana's central recommendation is to avoid skipping the early steps. The best evaluation framework depends on how the AI feature develops, so teams cannot fully design it in advance. They should build the smallest framework that gives useful information and make it a little better over time. This approach allows the evaluation to change when the product, data, and definition of success change.
Simple LLM judges are easier to calibrate against human judgment
In the discussion, Mariana says early LLM judges should be extremely simple and directly tied to the behavior the team wants. A question such as whether the agent used the correct name is easier to calibrate than a request to score every style guideline at once. She also says user feedback can enter the evaluation as a score, including thumbs-up or thumbs-down data, and can later help collect examples or support fine-tuning after correction.
