Everything We Got Wrong About Research-Plan-Implement

Dexter Horthy, HumanLayer26:46 · Mar 2026 · 138K views
Thumbnail for Everything We Got Wrong About Research-Plan-Implement Watch on YouTube
TL;DR
  1. 1

    Research should be generated from questions without giving the research agent the ticket, because telling it the desired feature encourages opinions instead of facts.

  2. 2

    Large monolithic prompts skip important workflow steps, so RPI should be split into smaller stages with fewer instructions and explicit control flow.

  3. 3

    Engineers should read and own production code, while using shorter design and structure documents to catch mistakes before implementation.

Summary

Dexter Horthy revisits the Research-Plan-Implement workflow after seeing experts get good results while their teams struggled. Research often became opinionated because agents were told what feature they were building. Planning also failed when a large prompt skipped its questions and design discussion, often because the workflow exceeded the model's instruction budget. Horthy now separates the process into questions, research, design, structure, plan, work tree, implement, and pull request, which he calls crispy. The design discussion aligns the engineer and agent on the desired solution. The structure outline gives a shorter view of the implementation order and testing checkpoints. Horthy has also reversed his advice about reading code. He says people responsible for production systems should read and own the code, even if that limits speed to roughly 2 to 3x. He is candid that formal verification may change this later, but he does not endorse treating code as unreadable assembly today.

Key ideas
03:54

RPI worked for experts because it depended on expert interaction

HumanLayer gave its coding-agent tools to thousands of engineers, from small startups to Fortune 500 companies. Experts often spent substantial time talking with Claude and shipped quickly, while their teams did not get the same results. Horthy traced the gap to workflow knowledge rather than a single prompt. Skilled engineers knew how to turn a ticket into questions, review the agent's assumptions, and steer the process. Other users ran commands such as research code base or create plan and accepted whatever came back. The method therefore looked repeatable, but important human decisions were hidden in the expert's use of it.

08:09

Research should establish facts before the agent sees the requested feature

Horthy says good research describes how the existing code works, without implementation opinions. A skilled engineer first turns a ticket into questions that force the agent to inspect the relevant parts of the codebase. The research context then receives those questions without the ticket or the intended feature. If the model knows what the team wants to build, it tends to interpret evidence through that goal and produce opinions. His revised workflow uses one context window to generate questions and a fresh context window to perform the research. He compares this to query planning for an LLM reading a codebase.

12:21

A large prompt cannot reliably enforce a long workflow

Horthy describes an instruction budget for frontier models. He cites research suggesting that models follow roughly 150 to 200 instructions with good consistency, with performance becoming less reliable beyond that. The old planning prompt had 85 or more instructions before adding the system prompt, Claude MD, tools, and MCP instructions. Important steps could therefore be skipped. The fix is to split the workflow into smaller prompts and use ordinary program control flow where possible. Horthy's revised sequence separates questions, research, design, structure, plan, work tree, implementation, and pull request. Each prompt is under 40 instructions, and he is still reducing them.

15:05

Short design documents give people an earlier place to correct the agent

The design discussion captures the current state, desired end state, patterns to follow, resolved decisions, and open questions. The agent writes down what it found and what it intends to do, then asks the engineer about gaps. Horthy calls this a chance to do brain surgery on the agent before it writes thousands of lines. The document is around 200 lines in his example, compared with a plan of about 1,000 lines. It also helps teams review decisions before code review. Horthy sends his design discussions to HumanLayer's code owner so bad decisions can be corrected while they are still cheap to change.

16:50

Structure outlines make plans easier to inspect and correct

The structure outline answers how the team will reach the design. It describes the major phases, their order, and how the work will be tested, without specifying every line of code. Horthy compares it with a C header file: it exposes signatures and changed types without showing the implementation. This creates a shorter review artifact, about two pages in his example, instead of an eight-page plan. The outline lets the engineer check the agent's direction before detailed implementation. It is especially useful for sensitive or complex work where a team wants to verify each substantial block before the agent continues.

17:54

Vertical plans create checkpoints where horizontal plans hide failures

Horthy says models tend to write horizontal plans that complete all database work, then all service work, then the API, and finally the frontend. That can produce a large amount of code with no working checkpoint, leaving the team to locate the failure afterward. A vertical plan cuts through the system in working slices. Horthy's example starts with a mocked API endpoint, connects it to the frontend, then fills in the service layer and database work. The total code can be similar, but each slice gives the engineer a point to test and fix the direction before more code is written.

08:40

Reading production code is part of owning the result

Horthy withdraws his earlier advice that engineers could skip code reading and rely on plans. He says the team tried that approach for about six months and had to replace large parts of the system. Open-source projects may accept a different level of oversight because nobody is paged at 3:00 a.m. or fined millions of dollars when they fail. For software used by customers, especially in regulated settings, Horthy asks engineers to read the code. He is skeptical of agent swarms that increase output while leaving quality unverified. His current target is 2 to 3x speed with near-human quality rather than 10x output that may be discarded later.

21:27

Crispy adds structure while keeping humans responsible

Horthy's revised workflow is questions, research, design, structure outline, plan, work tree, implement, and pull request. He calls the approach crispy after selecting the stages he wants to emphasize. The method uses static artifacts so the important decisions survive across context windows and do not depend on automatic compaction. The human reviews shorter documents for direction and alignment, then gives the code a deeper review. Horthy says this also improves team coordination because architecture decisions can be reviewed before implementation. The process has more stages than RPI, and he admits that adoption and measurement remain unresolved problems.

"The lesson is don't use prompts for control flow if you can use control flow for control flow."14:20
Who should watch
  • You are introducing coding agents to a team and the results vary sharply between experienced users and everyone else.
  • Your workflow relies on one large prompt and agents regularly skip questions, design review, or testing checkpoints.
  • You are deciding how much code engineers should inspect in production systems and want Horthy's current position after reversing his earlier advice.