# Loop Engineering

David DeStefano & Sam Christensen, EvolutionIQ & Valdimar Eggertsson, Snjallgögn (Smart Data inc.) & Sparsh Jain, CentralAgent AI | MLOps Reading Group | 56:10

Source: https://www.youtube.com/watch?v=-DrnzIBASbg
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/loop-engineering
Published: 2026-08-17
Tags: agents, coding-agents, evals, observability, workflows

## TL;DR
- Loop engineering moves software work from manually prompting an agent after every turn to systems that discover tasks, delegate work, verify changes, and decide what to do next.
- A useful loop needs explicit quality definitions, failure modes, success measures, enforcement mechanisms, and persisted learning from earlier runs.
- More agents and longer runs do not automatically improve results, so independent verification, bounded parallelism, and token-cost measurement are needed.

## Summary
This reading group discusses loop engineering rather than walking through a named paper or book. Sparsh Jain frames it as the next stage after prompt, context, and harness engineering. A loop discovers work, plans it, delegates tasks, verifies changes, stores state, and repeats until a goal or stopping condition is reached. Arthur Coleman describes building a production platform with Claude and adding regression tests, validation tests, architectural plans, code review, and adversarial review by Gemini and Fable. Valdimar Eggertsson presents experiments around OpenCode and the SWE-bench-style software engineering benchmark, with preparation, implementation, verification, repair, and scheduled issue discovery. The discussion focuses on cost, failure modes, parallel agents, templates, and built-in commands such as goal. The presenters are candid about the limits. Automation can create more QA work, burn tokens quickly, and produce convincing reports that fail in real use. Human approval remains part of the systems described.

## Key ideas
### Loop engineering follows several layers of control around an agent
[04:12](https://www.youtube.com/watch?v=-DrnzIBASbg&t=252s)
Sparsh Jain describes a progression from prompt engineering to context engineering, harness engineering, and finally loop engineering. Prompts give a model an input and receive an output. Context engineering adds tools, instructions, and information. Harness engineering adds the surrounding environment and scaffolding needed for longer tasks. Loop engineering adds feedback that can verify and improve the work across repeated runs. The shift is from checking every turn to building a system that contains its own checks and can produce a feature or product through a longer process.

### A production harness needs tests before the agent starts changing code
[11:27](https://www.youtube.com/watch?v=-DrnzIBASbg&t=687s)
Arthur Coleman began with a simple file upload site and found that Claude could break existing functionality while implementing the next change. His first lesson was to add regression testing early. He then added architectural plans and validation tests after seeing problems such as hard-coded variables that should have been configurable. He moved toward formal ticket and board management, although epics often grew when new defects and unplanned work appeared. His experience treats the harness as the part that protects stability, scalability, and security when the person directing the work is not a production-grade coder.

### Quality controls can make the delivery process fail if they are added without limits
[13:24](https://www.youtube.com/watch?v=-DrnzIBASbg&t=804s)
Coleman researched open-source QA practices and spent two weeks adding quality controls to his process. When he placed the checks on push, the push process failed, so he removed them and began running a modified version manually. He also added a dialectic review in which Gemini examines Claude's code adversarially. Coleman found Gemini more useful for detailed architecture and implementation review, while Fable helped with the larger picture. The arrangement improved his results, but it also made the process expensive and difficult to manage.

### A loop needs explicit failure records, success measures, and enforcement
[21:01](https://www.youtube.com/watch?v=-DrnzIBASbg&t=1261s)
Coleman describes a loop that collects design debates, implementation plans, code reviews, telemetry, and feedback. It recognizes and diagnoses issues, proposes changes, gets approval, integrates them, measures the result, and repeats. He says teams should define what quality means, record each failure with its problem, suspected cause, and response, and define how each loop will be enforced. Tests alone are not enough if nothing makes them run. He also warns that loop construction creates its own QA backlog, sometimes producing more tickets about the loops than about the application code.

### Preparation and independent verification can turn an agent into a controlled workflow
[24:35](https://www.youtube.com/watch?v=-DrnzIBASbg&t=1475s)
Valdimar Eggertsson describes experiments using OpenCode around a coding agent. A preparation phase uses a permission-limited research agent to inspect the repository and write a machine-readable plan before code changes. An implementation loop then works through individual plan steps, runs programmatic tests, asks an independent reviewer to assess the current step, and repairs problems before continuing. The system limits retries so it does not run forever. Eggertsson says his experiments were exploratory rather than scientific, but the added preparation and control flow improved results in his tests.

### Outer loops can discover new work after the inner coding loop finishes
[32:20](https://www.youtube.com/watch?v=-DrnzIBASbg&t=1940s)
Eggertsson adds an outer loop around the workflow. A GitHub issue webhook can trigger the coding loop, while an audit agent can inspect the repository on a schedule and find issues without a person opening them first. The audit agent records learnings so later runs have more context. In this design, the inner loop handles planning, implementation, verification, review, and repair. The outer loop decides when to start that workflow and supplies new work. The distinction makes loop engineering broader than a single agent completing one task.

### Observability and evaluation determine whether repeated runs are actually improving
[36:26](https://www.youtube.com/watch?v=-DrnzIBASbg&t=2186s)
Sparsh Jain says his team found that existing harnesses and technical infrastructure were insufficient for loop-based systems. They added an observability layer to inspect the system at both broad and detailed levels, followed by an evaluation layer. Only then did they build looping infrastructure. He separates inner loops, which can use language models, from outer loops, which can be more deterministic or design-based. His warning is that adding agents does not itself improve output. The system needs to know what to measure, how to evaluate it, and how to communicate results in a form agents can use.

### Token cost and parallelism need hard boundaries
[41:15](https://www.youtube.com/watch?v=-DrnzIBASbg&t=2475s)
The discussion turns to the cost of repeated agent runs. Valdimar Eggertsson says his experiment used a cheaper model and still consumed millions of tokens in the imagined full run. Another participant argues for separating work by layers and limiting the number of agents, because parallel generative changes can create confusion. Sparsh Jain describes a rebuttal step that checks assumptions before execution. It can cost two or three times as many tokens at the beginning, but it reduces later failures in his experience. Coleman recommends measuring token use at every stage and using those logs to find expensive parts of the workflow.

## Notable quotes
- Sparsh Jain: "Now the model is not the only thing that's smart. It's a feedback loop that is smart enough that it can verify and optimize itself." (07:45)
- Arthur Coleman: "What I need is a self-evolving platform to help me optimize, speed my process, and reduce my stress." (16:43)
- Valdimar Eggertsson: "The goal is to take away the tasks that the human in the loop usually handles and giving it to the system." (25:58)
- Sparsh Jain: "The more doesn't make better." (37:49)
- Arthur Coleman: "It is not one loop, it's multiple loops. Do one at a time." (23:32)

## Tools & references mentioned
- Claude Code
- Codex
- Claude
- OpenCode
- Gemini
- Fable
- DeepSeek
- Cursor
- GitHub
- Anthropic
- SWE-bench
- Figma

## Who should watch
- You are building coding-agent workflows and need a concrete way to separate planning, implementation, verification, repair, and scheduling.
- Your agent can write code, but regressions, missing tests, token costs, or unbounded runs are becoming the larger problem.
- You want examples of loop design from an individual builder and from exploratory experiments before committing to full automation.

## Related talks

- [Feedback Loops for Agentic Workflows](https://mlopstalks.com/talks/feedback-loops-for-agentic-workflows) (, 21:16)
- [AI Coding Agents Change Software Development Forever](https://mlopstalks.com/talks/ai-coding-agents-change-software-development-forever) (Michele Catasta, Replit & Aparna Dhinakaran, Arize AI & Erik Schluntz, Anthropic, 20:32)
- [Co-Engineering: The New Era of Human-AI Collaboration](https://mlopstalks.com/talks/co-engineering-the-new-era-of-human-ai-collaboration) (Kiriti Badam, OpenAI, 29:25)
- [Building Reliable AI Agents](https://mlopstalks.com/talks/building-reliable-ai-agents) (Travis Fischer, 17:43)
- [Coding with AI](https://mlopstalks.com/talks/coding-with-ai) (Chip Huyen, Tep Studio, 44:29)