Voice gives people a natural way to explain tasks, delegate work, and provide information to an agent.
2
Smaller models can reduce latency and cost when they handle specialized tasks under the direction of a larger model.
3
Production agents still face problems with context-specific data, long-horizon planning, infrastructure capacity, data quality, and developer access.
Summary
The panel examines how AI agents interact through voice, visual interfaces, text, and multiple models. Julia Kroll argues that voice lowers the barrier to using agents because people can speak naturally, change their minds, and provide information through conversation. Rogerio Bonatti describes a near-term role for computer agents that use screenshots and GUI elements, while predicting that mature systems will call APIs for greater precision. The panel also discusses architectures where a larger model routes work to smaller, specialized models. These can improve speed and cost, although smaller models are less capable on broad tasks. Examples include fine-tuning assistance, customer support, appointment scheduling, order taking, industrial control, and autonomous vehicles. The speakers are direct about the remaining problems. Agents need better context-specific data, the ability to plan several steps ahead, infrastructure that can handle large numbers of simultaneous workers, and simpler tools for developers.
Voice makes delegation easier for people who do not want to learn a new interface
Julia Kroll says voice is a natural human communication mode, so a person can explain what they need without learning a new system or interaction pattern. An agent can collect missing information through a fluid back-and-forth conversation before carrying out a task. She sees this applying to order taking, appointment scheduling, personalized customer support, and intake. These uses depend on the agent being able to draw on background information such as a calendar, medical history, or financial records. Voice also lets a user ask questions, change direction, and provide details in ordinary language.
Agents can communicate through more than one modality
Jazmia Henry describes agent-to-agent communication as a place where voice, text, video, and observation may all be useful. She connects this to reinforcement learning and swarm-style systems, where agents can observe one another, exchange information, and distill actions into reasoning patterns. Joshua Alphonse points to agent interactions in the Web3 space, including Truth Terminal, as an example of agents conversing and creating their own systems. The panel also argues that multimodal signals can reduce latency. Humans use visual context, body movement, voice, and prior experience together, and agents may gain similar advantages when these inputs work together.
Latency matters most when an agent is interacting with a person in real time
Rogerio Bonatti separates live human interaction from computer tasks that can happen offline. An agent that modifies or reviews documents does not necessarily need to act in front of the user, so precision can matter more than immediate response time. For live voice interaction, responsiveness remains important because long pauses disrupt the natural flow of conversation. Bonatti says smaller visual-language models are usually faster, while larger cloud models tend to perform better on agentic tasks. The panel treats this as a trade-off rather than a problem with one universal answer.
A larger model can route work to smaller specialized models
Joshua Alphonse describes PremAI's autonomous fine-tuning agent, which handles synthetic data generation, evaluation, and related steps for developers who may not have machine learning engineering resources. He also describes architectures with a larger model at the top and smaller models assigned to domain-specific tasks. Diego Oppenheimer summarizes the pattern as using a broad model for initial routing and more specialized models for subsequent work. This structure can reduce the cost and hardware demands of running every task through a large model. The panel still acknowledges that small language models are imperfect and require ongoing improvement.
Computer agents will likely move from GUIs toward precise API calls
Bonatti explains that current computer-agent research often combines image and text understanding. The agent identifies screen elements such as buttons, text boxes, and images, then emits actions against them. He expects this approach to remain useful in the short to medium term because existing systems are built around graphical interfaces. Over a longer period, he expects agents to call APIs directly. Ordering a pizza through a structured API with precise inputs and outputs is less prone to mistakes than navigating a website. Julia Kroll adds that a voice-facing agent can gather a user's flexible, conversational request and turn it into a structured REST API call.
Production adoption is limited by data, planning, capacity, and complexity
The panel names several barriers to wider use. Henry says agents need more context-specific data because people are still learning how they will interact with them. Bonatti identifies planning and reasoning across several possible future action chains as a major research problem. Kroll focuses on infrastructure capacity, especially for applications that may need thousands or millions of agents working at once. Alphonse adds the need for high-quality data, better decision-making, and more accessible implementation tools. Oppenheimer compares the coming operational challenge to the complexity of scaling Lambda functions and coordinating large agent swarms.
"It is much more precise for a computer agent to call a Domino's Pizza API with very precise inputs and outputs than it is for the agent to go to Domino's website."Rogerio Bonatti21:59
Who should watch
You are building a voice agent and need to understand where conversational flexibility helps and where latency becomes a problem.
Your team is deciding between GUI interaction and API-based tool use for computer agents.
You are planning to deploy many agents and need a grounded list of infrastructure, data, reasoning, and developer-experience problems to solve.
