# Agents in production

237 sessions · follows the tags agents, tool-use, mcp, context-engineering
Page: https://mlopstalks.com/threads/agents-in-production

What can an agent do on its own, and which tools should it get? Once it starts acting, someone needs a way to find out when it goes wrong.

## 2021

1 session.

- [Machine Learning in Cybersecurity](https://mlopstalks.com/talks/machine-learning-in-cybersecurity) (Monika Venčkauskaitė, Vinted). Pushed back: Monika presented government use of open source intelligence as requiring special permissions and serving security purposes, while Demetrios Brinkmann raised concerns that governments could abuse the tools for surveillance. [41:28](https://www.youtube.com/watch?v=S-BZwydzd-Q&t=2488s)

## 2023

18 sessions.

- [LLMs as Intelligent Assistants](https://mlopstalks.com/talks/llms-as-intelligent-assistants) (Sarah Aerni, Salesforce). Pushed back: Sarah Aerni rejects replacing human review with fully autonomous generated output and argues that human review remains critical. [23:40](https://www.youtube.com/watch?v=E0929WqB72k&t=1420s)
- [Building Production Copilots](https://mlopstalks.com/talks/building-production-copilots) (Tristan Zajonc, Continual). Pushed back: Tristan Zajonc says autonomous agents do not work reliably yet, while the host expresses doubt based on what has been implemented so far. [17:18](https://www.youtube.com/watch?v=rl9JvK1hz40&t=1038s)
- [Building Reliable AI Agents](https://mlopstalks.com/talks/building-reliable-ai-agents) (Travis Fischer). Pushed back: Fully autonomous agents are not yet the right default for production; more deterministic, code-driven agents should be used instead. [9:12](https://www.youtube.com/watch?v=A6wUIOMz7bE&t=552s)
- [Using LLMs to Power Consumer Search at Scale](https://mlopstalks.com/talks/using-llms-to-power-consumer-search-at-scale) (Aravind Srinivas, Perplexity AI). Pushed back: Aravind Srinivas rejects describing Perplexity as merely an LLM wrapper, arguing that the product requires substantial orchestration and software engineering around the models. [18:18](https://www.youtube.com/watch?v=HzGiVzYbf2I&t=1098s)

14 more from 2023 on this thread: https://mlopstalks.com/threads/agents-in-production/2023

## 2024

39 sessions.

- [MLOps at the Crossroads](https://mlopstalks.com/talks/mlops-at-the-crossroads) (Patrick Barker, Kentauros AI & Farhood Etaati, AIMedic). Pushed back: LLMOps should be treated as a distinct specialization with specialized tools rather than only as an extension of existing MLOps. [19:55](https://www.youtube.com/watch?v=A0tWDRBh5RI&t=1195s)
- [Becoming an AI Evangelist](https://mlopstalks.com/talks/becoming-an-ai-evangelist) (Alex Volkov, Weights & Biases). Pushed back: Demetrios Brinkmann says agents are not yet ready, while Alex Volkov argues that they are improving and should not be expected to remain at their current level forever. [1:21](https://www.youtube.com/watch?v=87qxhsi5wYU&t=81s)
- [Managing Small Knowledge Graphs for Multi-agent Systems](https://mlopstalks.com/talks/managing-small-knowledge-graphs-for-multi-agent-systems) (Tom Smoker, WhyHow.ai). Pushed back: Tom Smoker disputes the idea that one natural-language instruction should be trusted to make a multi-agent system complete a task correctly every time. [36:41](https://www.youtube.com/watch?v=LhWtpV-ZEeI&t=2201s)
- [Navigating the AI Frontier: The Power of Synthetic Data and Agent Evaluations in LLM Development](https://mlopstalks.com/talks/navigating-the-ai-frontier-the-power-of-synthetic-data-and-agent-evaluations-in) (Boris Selitser, Okareo). Pushed back: Boris Selitser argued that online intervention is often undesirable for complex agents because they may recover or find useful workarounds that developers did not anticipate. [13:42](https://www.youtube.com/watch?v=YOClk49Cy2o&t=822s)

35 more from 2024 on this thread: https://mlopstalks.com/threads/agents-in-production/2024

## 2025

94 sessions.

- [Machine Learning, AI Agents, and Autonomy](https://mlopstalks.com/talks/machine-learning-ai-agents-and-autonomy) (Egor Kraev, Wise Plc). Pushed back: Egor Kraev disputes the idea that LLMs should be judged as complete solutions or agents, arguing that they are additional components in a larger system. [9:33](https://www.youtube.com/watch?v=zte3QDbQSek&t=573s)
- [AI Agents: The Future of ML Engineering?](https://mlopstalks.com/talks/ai-agents-the-future-of-ml-engineering) (Matt Squire, Fuzzy Labs). Pushed back: Success on Kaggle competitions was disputed as evidence that an agent can perform general machine learning engineering or automate scientific discovery. [7:38](https://www.youtube.com/watch?v=ButA1OyQAW8&t=458s)
- [The Agent Landscape - Lessons Learned Putting Agents Into Production](https://mlopstalks.com/talks/the-agent-landscape-lessons-learned-putting-agents-into-production) (Paul van der Boor & Floris Fok, Prosus Group). Pushed back: The speakers reject the idea that declining cost per token automatically means that agent systems are becoming cheaper overall, because cost per answer can rise as agents use more tokens and calls. [21:33](https://www.youtube.com/watch?v=lRGldru7ohU&t=1293s)
- [Web Agents: The Cutting Edge of AI is Here?](https://mlopstalks.com/talks/web-agents-the-cutting-edge-of-ai-is-here) (Paul van der Boor & Chiara Caratelli, Prosus Group). Pushed back: The team found that performance on the WebArena benchmark did not translate reliably to the web-agent tasks they cared about. [17:49](https://www.youtube.com/watch?v=QbxN_PN7kZc&t=1069s)

90 more from 2025 on this thread: https://mlopstalks.com/threads/agents-in-production/2025

## 2026

85 sessions.

- [AI REWIND 2025 - MLOps Reading Group Year-end Special](https://mlopstalks.com/talks/ai-rewind-2025-mlops-reading-group-year-end-special) (Sophia Skowronski, Breckinridge Capital Advisors & Rohan Prasad, EvolutionIQ & Nehil Jain, Stealth AI Startup & Sonam Gupta, AICamp & Lucas Pavanelli, Stone). Pushed back: Rohan Prasad challenged the assumption that adding all available information to ever-larger context windows reliably improves LLM results. [15:10](https://www.youtube.com/watch?v=Wgfi3qWnPgE&t=910s)
- [Building Agentic Tools for Production](https://mlopstalks.com/talks/building-agentic-tools-for-production) (Sam Partee, Arcade AI). Pushed back: Sam Partee rejected the idea that all tools should be put in one MCP gateway and said agents should not have too many tools. [19:26](https://www.youtube.com/watch?v=8TRQo3mvl7g&t=1166s)
- [Enterprise AI Operations: The Missing Piece](https://mlopstalks.com/talks/enterprise-ai-operations-the-missing-piece) (Rani Radhakrishnan, PwC US). Pushed back: Starting with one agent and scaling sequentially is not the only valid approach; parallel development can work when processes are standardized and repeated across departments. [9:42](https://www.youtube.com/watch?v=jTmV_jlob5I&t=582s)
- [Tool definitions are the new Prompt Engineering](https://mlopstalks.com/talks/tool-definitions-are-the-new-prompt-engineering) (Chiara Caratelli, Prosus Group & Alex Salazar, Arcade.dev). Pushed back: Alex Salazar argued that most people should currently use carefully selected, specific tools, while the long-term goal is to let agents choose from many tools. [11:49](https://www.youtube.com/watch?v=rMhvSkXbv4A&t=709s)

81 more from 2026 on this thread: https://mlopstalks.com/threads/agents-in-production/2026