# Coding with AI Chip Huyen, Tep Studio | Agents in Production 2025 | 44:29 Source: https://www.youtube.com/watch?v=xY1FcjIbErQ Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks. Page: https://mlopstalks.com/talks/coding-with-ai Published: 2026-01-19 Tags: agents, coding-agents, developer-experience, mcp, planning ## TL;DR - AI coding productivity is better measured by mental effort and interruption rate than by elapsed engineering time or lines of code. - Coding agents work best when engineers give them clear specifications, use modular codebases, and verify what they produce. - As coding automation improves, engineers will spend more time planning, reviewing, and guiding work instead of writing every line by hand. ## Summary Chip Huyen describes how AI coding tools are changing software work. She compares IDE assistants, terminal agents, GitHub-based workflows, and web interfaces, then proposes measuring progress through interruption rate. A task may take a day while requiring only an hour of human attention, which lets an engineer run several tasks in parallel. Huyen argues that agents currently perform better on new code than on tangled legacy systems, so code structure and language choice affect their performance. She recommends a workflow built around planning, execution, and verification. Clear specifications remain important, as do rules that constrain dependencies, scale, and API use. Huyen is direct about the limits: agents still need supervision, tool overload can make them worse, and verification is difficult for some tasks. She expects engineers to write less code and spend more time specifying systems, reviewing changes, and understanding failures. ## Key ideas ### AI coding tools range from editor completion to agents that work from a terminal [00:39](https://www.youtube.com/watch?v=xY1FcjIbErQ&t=39s) Huyen groups AI coding interfaces into several forms. IDE tools such as Cursor provide completion and code interaction inside an editor. Terminal tools let an engineer assign work from the command line. A GitHub-based workflow could ask an agent to address an issue, submit a pull request, or review a repository. Web tools can take an interface mockup and generate an application. She says preferences vary because some companies only provide IDE tools, while individual engineers often move toward more automated interfaces as they gain experience. ### Elapsed time does not show how much attention an AI-assisted task requires [05:32](https://www.youtube.com/watch?v=xY1FcjIbErQ&t=332s) Traditional measures such as engineering time and lines of code fit poorly when an agent works in the background. Huyen gives the example of assigning a task that takes a day, checking the result later, and spending only half an hour or an hour of mental effort on it. She is willing to let tasks take longer if they require less attention, because she can start many tasks and return to the results later. AI also changes the cost of abandoning code. When an existing codebase becomes too difficult to repair, engineers may generate a replacement instead of protecting the code they already wrote. ### Coding automation can be described through levels and interruption rate [08:00](https://www.youtube.com/watch?v=xY1FcjIbErQ&t=480s) Huyen borrows an automation framework from Graham Mubeck's work on self-driving cars. The levels run from autocomplete, through limited task automation and broader feature or application creation, toward highly autonomous work. She says current coding agents are mostly between levels one and three, and she has not seen reliable level-four automation for difficult legacy systems or low-level CUDA optimization. Her practical measure is interruption rate. If an engineer must repeatedly take control, the agent still demands constant attention. If the engineer can assign a task and inspect it only when it finishes, confidence is higher. ### Lower interruption rates let engineers run more work and make subagents safer to use [12:08](https://www.youtube.com/watch?v=xY1FcjIbErQ&t=728s) Huyen says interruptions limit how many agents a person can supervise because each interruption adds context and demands mental effort. She can track only a few coding agents at once, while a friend can track more. Subagents create another risk because a user cannot interrupt a running task, so a bad subagent can waste tokens and money before returning a result. A main agent should therefore delegate only when it has high confidence that the subtask can finish independently. Lower interruption rates also reduce the amount of context that must be reintroduced after an agent goes off course. ### Experience and task definition strongly affect how often people interrupt agents [15:32](https://www.youtube.com/watch?v=xY1FcjIbErQ&t=932s) Huyen observes that people from nontechnical backgrounds may interrupt less because they have less ability to judge whether generated code is correct. Her limited data suggests senior engineers may also interrupt less than junior engineers. Senior engineers are more accustomed to writing design documents and communicating requirements, so they can give an agent a clearer target. She also describes an exploratory workflow: an initial agent run helps someone discover what they actually want, then a detailed specification can be given to a fresh run. The second run usually needs fewer interruptions because the requirements are clearer. ### Agents handle new modular code better than tangled existing systems [19:01](https://www.youtube.com/watch?v=xY1FcjIbErQ&t=1141s) Huyen says AI coding agents generally do better when building new features in a new codebase than when changing an existing one. Poorly structured systems with intertwined components make it hard for an agent to find the right place to work. Teams may need to refactor such systems into smaller, modular parts. Tool and language choice also matter. She gives JavaScript and Python as an example, saying models can perform worse on JavaScript because the internet contains more poor-quality JavaScript code. In her own work, changing an application from JavaScript to Python immediately reduced the agent's task complexity. ### Code structure changes how many steps an agent needs to complete a task [22:23](https://www.youtube.com/watch?v=xY1FcjIbErQ&t=1343s) Huyen tracks the number of background steps an agent takes for each instruction. As a codebase grows, the agent spends more steps searching for files, functions, and relevant locations. She wants this complexity graph to stay flat rather than rise with every instruction. To reduce the search burden, she makes files shorter, keeps the code modular, and avoids duplicate code. She also says tool designers should provide enough tools for useful work without overwhelming the agent. Developers she spoke with generally avoid giving an agent more than 20 tools, and MCP servers can create problems when they add many overlapping tools. ### Spec-driven development puts planning and verification around AI-generated execution [26:44](https://www.youtube.com/watch?v=xY1FcjIbErQ&t=1604s) Huyen describes a three-part workflow: provide a specification and generate a plan, execute the plan by generating code, then verify that the result meets the original goal. She does not expect specification work to disappear because communicating intent to an AI system still requires engineering judgment. In her own rules files, she states which technologies to use, tells the agent to read documentation instead of inventing API calls, and defines the expected scale. She also studies the errors that cause interruptions and changes her process or codebase to reduce them. She connects this work with system thinking and expects more engineers to review and guide code than write it directly. ## Notable quotes - "I'm okay with things taking more time if it requires less mental energy for me." (06:27) - "The highest interruption rate is actually limiting my ability to do a lot of things at the same time." (13:06) - "I do think we are reaching the point when a lot of us are actually reviewing code more than writing code." (28:20) - "I do think users need to write spec instructions. I don't think it is going to go away." (27:02) ## Tools & references mentioned - Cursor - Codex - Gemini CLI - GitHub - GitLab - Graham Mubeck - Tesla - CUDA - Python - JavaScript - Rust - MCP - Google Calendar - BigQuery - GCP - AWS ## Who should watch - You are deciding whether an IDE assistant, terminal agent, or another coding interface fits your workflow. - Your team is trying to measure AI coding productivity without relying on lines of code or elapsed engineering time. - You maintain a large codebase and need practical guidance on specifications, modularity, agent tools, and verification. ## Related talks - [Building Artificial Engineering Intelligence for Engineering Teams](https://mlopstalks.com/talks/building-artificial-engineering-intelligence-for-engineering-teams) (Aishwarya Shankar, Intelligence AI, 10:10) - [AI Coding Agents Change Software Development Forever](https://mlopstalks.com/talks/ai-coding-agents-change-software-development-forever) (Michele Catasta, Replit & Aparna Dhinakaran, Arize AI & Erik Schluntz, Anthropic, 20:32) - [Co-Engineering: The New Era of Human-AI Collaboration](https://mlopstalks.com/talks/co-engineering-the-new-era-of-human-ai-collaboration) (Kiriti Badam, OpenAI, 29:25) - [The Future of Coding: AI Agents & the Next Tech Revolution](https://mlopstalks.com/talks/the-future-of-coding-ai-agents-the-next-tech-revolution) (Ricky Doar, Cursor, 26:45) - [How Agents Changed Vibe Coding Forever](https://mlopstalks.com/talks/how-agents-changed-vibe-coding-forever) (Beyang Liu, Sourcegraph, 49:49)