# Too much lock-in for too little gain: agent frameworks are a dead-end Valliappa Lakshmanan | MLOps Community | 35:37 Source: https://www.youtube.com/watch?v=HUHjSFWr9pg Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks. Page: https://mlopstalks.com/talks/too-much-lock-in-for-too-little-gain-agent-frameworks-are-a-dead-end Published: 2025-08-27 Tags: agents, continual-learning, human-in-the-loop, workflows ## TL;DR - Agentic systems should keep the orchestration under explicit developer control while using simple composable patterns for individual agents. - Human edits, evaluations, and user actions should be captured so the system can improve from a prototype toward production. - Teams should use standard deployment, monitoring, persistence, and guardrail tools without tying the application to one LLM, cloud, or agent framework. ## Summary Valliappa Lakshmanan argues that agent frameworks often add lock-in without giving teams enough value in return. He proposes building agents as small, independently implemented components with structured inputs and outputs, dynamic context, and prompts kept separate from application code. The workflow itself should remain explicit and controllable because business processes often do not fit a framework's fixed orchestration patterns. His example is an education application that creates and reviews workbooks. Users choose or correct roles, edit generated text, and guide a multi-agent review panel. Those actions become training data. Lakshmanan recommends standard infrastructure for deployment, monitoring, persistence, and security, with specialized services added only when simple approaches stop working. He also discusses asynchronous guardrails, append-only memory, conflict-resolution agents, and a post-training pipeline based on scores and human feedback. He is candid that fully autonomous workflows are unreliable today, especially when agents are allowed to re-plan too deeply or argue directly with one another. ## Key ideas ### Agent systems should stay independent of the model and hosting location [00:00](https://www.youtube.com/watch?v=HUHjSFWr9pg&t=0s) Lakshmanan wants agentic systems to remain independent of the LLM and the place where the system runs. Legal requirements may force a team to use a particular model, while latency may require a smaller or faster one. The system should also allow components to be removed or replaced as the field changes. He warns against rigid workflow structures because business processes may not fit sequential, parallel, or orchestrator-worker patterns. His preferred fallback is explicit orchestration code, which gives the team control over how the business workflow runs. ### Traceability has to cover every action an agent takes [01:44](https://www.youtube.com/watch?v=HUHjSFWr9pg&t=104s) Lakshmanan connects traceability to several practical needs. People may ask why a system produced a response, auditors may need a record of actions, and teams need metrics about adoption, correctness, and business KPIs. Evaluation also depends on seeing what happened at each step. He says an agentic system should record every action, along with the inputs and outputs that led to it. This record also supports a learning path in which human corrections gradually allow the system to handle easier cases with less supervision. ### Simple patterns and standard tools provide a middle ground [03:58](https://www.youtube.com/watch?v=HUHjSFWr9pg&t=238s) Lakshmanan contrasts Anthropic's advice to use simple composable patterns with Sierra's description of the large investment needed to orchestrate, secure, and maintain agents. He accepts both points. His approach is to compose simple patterns, then buy or reuse monitoring, evaluation, and guardrail capabilities rather than building every supporting feature. The agents should follow business workflows and business KPIs, while the surrounding application runs on proven microservice infrastructure. This preserves production practices without making the core application depend on one agent framework. ### Human edits turn a workflow into a source of training data [07:07](https://www.youtube.com/watch?v=HUHjSFWr9pg&t=427s) His workbook example lets a user correct the system at each stage. The system may choose a math writer for a topic, while the user changes that role to a historian. The user can also ask for bullet points, remove keywords, and edit the draft. A panel of agents reviews the material from different perspectives, and a secretary agent turns those reviews into instructions for the writer. Lakshmanan records the original AI output, the user's correction, and the resulting version so the feedback can later train task assigners, prompts, or models. ### Agents need structured state and prompts outside the code [11:57](https://www.youtube.com/watch?v=HUHjSFWr9pg&t=717s) Each agent should have a specific job, receive stateful context, use the tools and knowledge available to the workflow, and produce a structured output. Structured results are easier to evaluate and handle more reliably. Lakshmanan recommends implementing each agent independently, with inputs and outputs containing the full state needed by the next step. Prompts should be separate from application code and stored as templates. That separation keeps configuration, prompt optimization, and later reinforcement learning manageable when models or workflows change. ### A co-pilot workflow captures rules that procedures leave out [16:22](https://www.youtube.com/watch?v=HUHjSFWr9pg&t=982s) Lakshmanan says a fully autonomous workflow often performs poorly because a written standard operating procedure does not contain all the rules people apply in practice. A co-pilot workflow lets users choose the next step, correct generated content, and reveal unwritten decisions. Those interactions give the system information about how to move through the process. He therefore treats the user interface as part of the learning system, especially when the workflow involves complex choices about which agent should act next. ### Use simple persistence first and add services when the data requires them [21:41](https://www.youtube.com/watch?v=HUHjSFWr9pg&t=1301s) For short-term memory, the application can often accumulate state and pass it between agents. A persistent memory service becomes useful when the state grows large or must survive across sessions. Lakshmanan makes a similar argument for retrieval. A direct search over large document chunks may be enough before a team needs a more elaborate retrieval system or database. He applies the same rule to caching and other persistent primitives: start with the simple implementation, then adopt a standalone service when the application's size or lifetime makes it necessary. ### Continuous learning needs deliberate feedback collection [19:24](https://www.youtube.com/watch?v=HUHjSFWr9pg&t=1164s) Lakshmanan says a prototype rarely works unchanged in production. Teams need to collect scores, evaluation criteria, and human corrections, then use them for preference tuning, instruction tuning, adapter tuning, or other post-training work. He also warns about automation fatigue. As a system improves, users provide fewer corrections because they assume it is correct. A data program is therefore needed to obtain useful feedback even when ordinary usage stops producing enough corrections. ### Parallel agents should report to a resolver instead of arguing with each other [29:01](https://www.youtube.com/watch?v=HUHjSFWr9pg&t=1741s) Parallel work is acceptable when several agents review the same item, but Lakshmanan recommends a separate resolution step afterward. In his panel example, agents with conflicting perspectives submit their work to a secretary that summarizes the disagreement and produces instructions. He treats memory as an append-only log and leaves conflict resolution to the consumer of that memory. He also recommends keeping agent depth to one layer followed by resolution. When agents repeatedly re-plan or directly debate with agents that have opposing objectives, he says their behavior can escalate quickly and become difficult to control. ## Notable quotes - "You want to be able to do it in a general purpose way and the most general purpose way is to be able to explicitly code an orchestration mechanism." (01:23) - "Agent frameworks are too much lock in for too little gain." (04:40) - "You cannot give that off to a framework because that is often where the domain knowledge comes in." (13:58) - "So I recommend if people design systems to go one deep and no deeper." (27:32) ## Tools & references mentioned - MLOps Community - Demetrios Brinkmann - Anthropic - Sierra - O'Reilly - GitHub - Lambda - Cloud Run - Pantic AI - LlamaIndex - Jinja - direct preference optimization - instruction tuning - preference tuning - adapter tuning - React - Figma - Streamlit ## Who should watch - You are building an agentic application and need to change models, hosting locations, or workflow logic without rewriting the system. - Your prototype depends on human review, but you do not yet have a clear way to record corrections and turn them into evaluation or training data. - You are deciding whether to adopt an agent framework or assemble agents with ordinary application code and standard infrastructure. ## Related talks - [AI Agents: The Future of Productivity, or Just a Fad?](https://mlopstalks.com/talks/ai-agents-the-future-of-productivity-or-just-a-fad) (Sam Partee, Arcade AI, 35:18) - [AI Agent Development Tradeoffs You NEED to Know](https://mlopstalks.com/talks/ai-agent-development-tradeoffs-you-need-to-know) (Sherwood Callaway, 11X, 57:07) - [How Agentic Workflows Will Change Everything](https://mlopstalks.com/talks/how-agentic-workflows-will-change-everything) (Raj Rikhy, Microsoft, 49:13) - [How to Build Execution Layers That Don't Burn Out](https://mlopstalks.com/talks/how-to-build-execution-layers-that-dont-burn-out) (Tanmay Tiwari, 10:07) - [Why Agents are Driving Software Development to the Cloud](https://mlopstalks.com/talks/why-agents-are-driving-software-development-to-the-cloud) (Zach Lloyd, Warp, 51:08)