Lessons from 25 Trillion Tokens: Scaling AI-Assisted Development at Kilo

Scott Breitenother, Kilo25:55 · Mar 2026 · 4,024 views
Thumbnail for Lessons from 25 Trillion Tokens: Scaling AI-Assisted Development at Kilo Watch on YouTube
TL;DR
  1. 1

    Kilo increased its release pace from one feature every two to three weeks to one or two features a week without quadrupling its team or extending working hours.

  2. 2

    Kilo gives each engineer end-to-end ownership of a feature and asks them to orchestrate several AI agents instead of making collaboration the default.

  3. 3

    AI adoption depends on trust, which is affected by latency, context, model choice, and whether agents produce useful results without constant interruptions.

Summary

Scott Breitenother describes how Kilo changed its engineering model after processing 25 trillion tokens and serving more than 1.5 million developers. The company has about 15 engineers, each owning features from design through deployment and user feedback. Developers increasingly act as orchestrators, while agents handle coding, documentation, debugging, and other execution work. Scott says the transition happens gradually, from autocomplete to chat, single-agent delegation, and finally multi-agent orchestration. Trust can fall quickly when suggestions are slow, agents edit the wrong files, or autonomous work requires constant review. Kilo therefore focuses on context, choosing suitable models, instrumentation, and fast removal of rough edges. Scott also says the work has shifted toward architecture, product judgment, and review. At Kilo, every engineer is effectively their own product manager, with one PM focused on the shared platform. The approach creates speed, but it also makes extended thinking and feature ownership more demanding.

Key ideas
01:24

Kilo increased shipping speed by changing its engineering process

Scott says Kilo moved from shipping one feature every two to three weeks to shipping one or two features a week. The company did not quadruple its team, change its deadlines, or rely on extreme working hours. It changed how engineers work, moving away from a 2023 development model toward one where AI handles much of the execution. Kilo had about 15 engineers at the time of the talk. Scott points readers to the company's Friday release notes as evidence of its pace. He says the main constraint is no longer typing code. Process, handoffs, and unnecessary coordination are the larger sources of delay.

03:55

Each engineer owns a feature from conception through user feedback

Kilo assigns one engineer to each feature rather than assigning ownership to a team. Scott gives Suresh's code-review work as an example. Suresh owns the feature from conception and coding through deployment, then works directly with users to collect feedback and make changes. Engineers still talk when another person's input adds value, but collaboration is not the default. Scott argues that many meetings and handoffs act as delays or comfort rituals. When coding becomes easier through AI, he says, reducing those process delays creates more speed than adding more coding capacity.

06:19

Developers move through an AI adoption ladder as they gain trust

Scott describes a progression from autocomplete to chat, then to delegating work to a single agent, and finally to orchestration across several agents. Autocomplete keeps the developer in direct control. A single agent is like a driving lesson where the developer can still intervene. Orchestration is more hands-off, with several agents running tasks. At Kilo, developers commonly use two, three, or four parallel agents. Scott says people usually need time to see the results before they trust the system. A few slow suggestions, an edit in the wrong file, or repeated permission requests can send someone back down the ladder.

09:35

Context, model selection, and fast feedback determine whether trust survives

Scott identifies three recurring failure points in AI adoption. The tool needs the right context, it needs an appropriate model for the task, and it needs to improve quickly as users find rough edges. He says users must provide more context as they move from autocomplete to orchestration, eventually including repositories, dependencies, and the wider organization. Kilo initially sent too much work to its most expensive model. Scott now recommends using a state-of-the-art model for architecture, then more cost-effective models such as Kimmy, MiniMax, or GLM for coding and debugging. Combining models works better than choosing one model for everything.

11:57

Broader repository context helped one engineer build Kilo's data model quickly

Scott describes Pedro, Kilo's one-person data team. Pedro built the company's dbt data model from the ground up in roughly a week or two, despite starting with little data infrastructure. He gave the agent access to the repository for the data project and to the repository containing the application code. That let the agent understand how the application created the data before developing warehouse transformations. Scott says this broader context was the major improvement. The agent could see the data's origin and the application structure instead of treating the warehouse as an isolated source.

14:31

Kilo measures trust through usage signals, though the signals weaken with autonomous work

Scott says trust can be measured by observing whether people use a feature and act on its results. Autocomplete offers a strong signal because Kilo can see whether a suggestion is accepted, and the events happen in high volume. Chat provides weaker evidence because copying, ignoring, or acting on a response may happen later. Long-running agents and orchestration are harder still because the result may arrive minutes, hours, or days after the request. Kilo invests in instrumentation to understand whether features are useful and trusted. Its fast release cycle lets the team respond when the data reveals a rough edge.

16:44

AI changes engineering work toward thinking, architecture, and judgment

Scott says AI has not reduced the amount of work at Kilo. It has changed the balance. He describes a shift from roughly 20 percent deep thinking and 80 percent coding toward about 80 percent thinking and 20 percent coding, or even less coding. Engineers must warm up to spending much more time as architects and orchestrators. He compares supporting 200 hours of agent coding with four hours of architecture and 36 hours of coding. At Kilo, developers set quality gates and decide how much of the generated work needs review, while agents handle typing, boilerplate, documentation, and routine searches.

18:11

Kilo organizes agents into roles that let one engineer own a full feature

Kilo provides an orchestrator, architect, code, ask, and debug agent. Scott explains that Suresh first used the ask agent to learn about the codebase, then the architect agent to design a feature, the code agent to implement it, and the debug agent when problems appeared. Engineers can also create custom agents and modes. One Kilo engineer created a Brian mode based on his own settings, and other developers use it. Scott compares the developer to the manager of a team of agents. The important design choice is making that team fit the work rather than giving every engineer one undifferentiated assistant.

"In the age of AI, I guess coding is the easy part, and the bottleneck is no longer the coding, it's kind of all the process."06:52
Who should watch
  • You run an engineering team that is experimenting with coding agents and need practical guidance on moving from autocomplete to multi-agent orchestration.
  • Your developers spend more time in meetings, handoffs, and reviews than in implementation, and you want to understand how Kilo reduces those delays.
  • You are designing an agent product and need examples of how latency, context, model choice, instrumentation, and user trust affect adoption.