Most production AI applications are still chatbots, although they now call tools, retrieve information, take actions, and often support richer interfaces.
2
Multi-turn evaluation is needed to find repetition, broken context, poor logical flow, and other problems that cannot be seen in a single input-output pair.
3
MLflow is extending its existing model-development workflows for agent development, with tracing, LLM judges, feedback collection, prompt optimization, review tools, and governance in one platform.
Summary
Corey Zumar, Jules Damji, and Danny Chiao describe how MLflow is adapting to production GenAI applications. They say most current use cases remain chatbots, but these systems increasingly call tools, retrieve information, write data, use memory, and operate across multiple turns. That makes evaluation harder. Teams need judges and human feedback that can assess entire conversations, detect repetition, check context retention, and identify whether the user achieved their goal. The guests also discuss how feedback should move from user ratings into filtered traces, domain-expert review, root-cause analysis, and automated tests. MLflow is being built for both traditional model developers and agent developers because the underlying iteration cycle is similar, even though the tools differ. The conversation ends with governance concerns around model access, tool permissions, budgets, traces, and sensitive information. The guests argue that storing raw trace data in governed tables makes analysis and redaction more practical than locking all data away by default.
Most production AI applications are still tool-calling chatbots
Corey Zumar says chatbots remain the dominant production use case, with more than half of use cases still using a chat interface. Some began as text-only systems and now support multiple modalities, voice, or images. Underneath, the architecture has changed. Chatbots increasingly retrieve from knowledge bases, call real-time APIs, and take actions that write information. Danny Chiao adds that these systems now return richer HTML, include human handoffs, and may contain multiple agents and guardrails for higher-stakes work.
Multi-turn evaluation exposes failures that single-turn tests miss
Danny Chiao says observability and evaluation for chatbots have lagged behind the complexity of the systems. A single input and output cannot show whether an agent repeats itself across a conversation. Corey Zumar gives the example of a 20-turn exchange where the agent keeps rephrasing the same answer. Multi-turn judges can also examine whether users escalate to human support, whether the conversation is logically consistent, whether context is retained, and whether the user reached the intended goal. Jules Damji describes built-in and custom judges for these checks.
Agent memory needs separate handling for active context and past interactions
Jules Damji frames long conversations as a memory and state-management problem. A 15-minute history can help when a new request follows naturally from the earlier discussion, but unrelated requests can be harmed by carrying all that context forward. He distinguishes short-term memory from long-term memory. Older conversation material can be moved into long-term storage and retrieved when needed, while the active session can be cleared or kept smaller. The goal is to preserve useful references without making every new request depend on the entire previous conversation.
User feedback should become filtered debugging data and automated tests
Corey Zumar says teams often deploy chatbots without collecting structured feedback, leaving users to express frustration in the chat. Thumbs up, thumbs down, ratings, and written explanations give developers better signals. In MLflow, that feedback can filter large collections of traces so developers can focus on interactions with high frustration or incomplete task results. Domain experts can then investigate the filtered examples. Once a cause is understood and fixed, the original feedback can help form an automated evaluation or unit test, reducing the need to ask users whether the next version is better.
Prompt optimization should avoid turning prompts into brittle rule systems
Corey Zumar says prompts still strongly affect agent quality, even after interest in fine-tuning and other methods. Prompt optimization can use negative feedback, examples of desired behavior, and existing prompts to generate improved instructions. Demetrios Brinkmann warns that manually adding every edge case can produce an enormous prompt and damage usability. Corey agrees that this can become a rule-based system written in English. He says optimizer frameworks integrated with MLflow can evaluate many samples together, which helps avoid overfitting to individual cases.
MLflow is serving model and agent developers through related workflows
Corey Zumar says MLflow will continue serving data scientists who develop traditional models while also addressing agent development. Both workflows involve building an initial system, measuring quality, iterating, deploying, and collecting production signals. The details differ. Model developers track training, validation, and test loss, while agent developers need traces, stack traces, domain-expert review, and LLM judges. The guests argue that many people move between these roles, so a separate product would force them to learn and operate another platform. MLflow instead presents different tools for different objectives within one platform.
Coding assistants make evaluation a task that should happen earlier
Corey Zumar says coding assistants have accelerated the creation of agents. Developers can add tools, connect data sources, or change prompts in hours rather than waiting weeks for hand-coded work. That also moves quality checks earlier in the development process. MLflow makes its judges and feedback functions available to coding assistants through MCP, so an assistant can help generate evaluation inputs, assess quality, and iterate on an agent. Jules Damji says coding assistants can produce software, but developers still need enough background to choose meaningful metrics and understand how to assess the result.
Governance must control model access, tools, costs, and sensitive traces
Danny Chiao and Corey Zumar describe governance as a set of access and cost controls. Organizations need to decide who can use particular agents, which data those agents can access, which models and tools are available, and how much each project can spend. Corey gives the example of separate teams with very different workloads and budgets. They also discuss sensitive information in traces and memory, including personal information and private workplace situations that may not look like conventional identifiers. The guests prefer governed access to raw trace data, so teams can analyze and redact it while restricting access through Unity Catalog.
"The agent developer needs traces. The model developer might want them way in the future after they deploy something, but it's not even something they're thinking about as they start to build a new use case."Corey Zumar37:50
Who should watch
You are building a chatbot that calls tools or keeps conversation history, and single-turn tests are missing failures in production.
Your team collects traces and user ratings but has no practical path from negative feedback to domain-expert review and regression tests.
You are deciding whether model and agent development need separate platforms, or how to govern tool access, sensitive traces, and model spending.
