# The Latency Goldilocks Zone Explained

Rafael Borger & Daniel Wolbert, iFood | MLOps Podcast | Episode 376 | 48:14
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=dH-1INvvELo
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/the-latency-goldilocks-zone-explained
Published: 2026-05-20
Tags: agents, recommender-systems, search, voice

## TL;DR
- ILO combines a user's preferences, purchase history, price sensitivity, location, and restaurant relationships to recommend food for complex requests.
- AI responses need a latency that fits the question and channel. Streaming, progress messages, and visual loading experiences can make a slower response feel faster.
- Building an agent at scale requires coordination with the owners of the data and APIs it uses, since an external bottleneck can become the agent's latency problem.

## Summary
Rafael Borger and Daniel Wolbert explain how iFood is building ILO, a conversational ordering experience for users with very different tastes, budgets, and habits. ILO can interpret requests such as a dinner for two where one person is vegetarian, then combine many signals to select suitable options. The system also tries to help undecided users through recommendations and a swipe-based discovery interface. A large part of the discussion concerns latency. Rafael says an answer can feel untrustworthy when it arrives instantly for a complex question, while a slow answer makes users think the system is stuck. The acceptable delay depends on the task and the channel, with voice needing faster, shorter responses than WhatsApp or an app interface. The guests also discuss data dependencies, agent-to-agent handoffs, and the difficulty of measuring product-market fit through survey scores in Brazil. Their account is practical and candid about scalability, cost, user education, and unresolved support workflows.

## Key ideas
### ILO recommends beyond a user's established tastes
[01:13](https://www.youtube.com/watch?v=dH-1INvvELo&t=73s)
Daniel Wolbert describes ILO as a conversational agent that can handle simple requests such as a pizza or richer ones such as a romantic dinner for two where one person does not eat onions. It uses knowledge of the user, their preferences, desires, and current context to return options. The team is also exploring proactive recommendations for undecided users, although the current experience is mainly reactive. The system is intended to help when a person does not know what to eat, rather than requiring them to begin with a precise product search.

### Recommendation requires exploration beyond a user's history
[06:26](https://www.youtube.com/watch?v=dH-1INvvELo&t=386s)
Rafael Borger says it is easy to recommend food that the system already knows a user likes. The harder problem is suggesting something unfamiliar without simply guessing. He gives the example of a user who often orders sweet food and receives a sweet pizza recommendation, then explains that this can still be wrong. ILO uses different techniques and repeated experiments to learn what a person might accept outside their normal pattern. The system considers how close a user is to other users, restaurants, and dishes, then tries to find a suitable path between them.

### Personalization includes money, distance, quality, and context
[07:59](https://www.youtube.com/watch?v=dH-1INvvELo&t=479s)
iFood cannot build one recommendation product that fits everyone, Rafael says. A user's food tastes are only one part of the decision. The system also needs to consider price sensitivity, available budget, previous purchases, distance from home, ratings, and the trade-off between quality and price. A person with only 10 reais to spend should not receive a 200-real pizza recommendation. ILO combines these signals instead of asking the user to work through separate filters. After a user chooses an item, the system treats that choice as feedback and updates its understanding when the recommendation was wrong.

### iFood uses small experiments to find larger businesses
[09:35](https://www.youtube.com/watch?v=dH-1INvvELo&t=575s)
Daniel explains iFood's 'jet skis' idea through the concept of an ambidextrous organization. The main business is a large ship that is difficult to turn, while small experimental teams can test ideas quickly and cheaply. If a thesis fails, the team moves to another one. If it works, iFood invests heavily. Daniel names the fintech business, grocery business, and iFood Clíp as businesses that began as jet skis. He describes the approach as starting small and fast while allowing successful experiments to grow into substantial revenue streams.

### ILO is shifting search toward suggestions and open-ended requests
[11:19](https://www.youtube.com/watch?v=dH-1INvvELo&t=679s)
Daniel says iFood is testing a move from users searching for an item toward having an item suggested to them. ILO can interpret a request containing delivery time, price, group size, and different dietary preferences, which would be difficult to express through ordinary filters. The team is also testing a Tinder-style food interface. Users swipe through recommendations, compare price and delivery time, and choose an item at the end. Daniel says the experience can expose users to food outside their usual ordering pattern while giving iFood more information for later recommendations.

### The right latency depends on task and channel
[21:52](https://www.youtube.com/watch?v=dH-1INvvELo&t=1312s)
Rafael describes a latency Goldilocks zone for AI. An answer to a complex question that arrives immediately can seem as if the system did not think enough. A response that takes too long can make the user believe the system is stuck or unreliable. He cites a 2022 study discussed as showing a more acceptable range for some interactions, while stressing that the right delay depends on the question. Answering a user's name should be fast, but a complex recommendation can take longer. WhatsApp supports more asynchronous waiting than voice, where the system must respond quickly.

### Perceived latency is part of the product experience
[24:48](https://www.youtube.com/watch?v=dH-1INvvELo&t=1488s)
Rafael separates actual latency from perceived latency. Streaming partial results, showing that the system is collecting information, or displaying a progress indicator can reassure users that work is happening. He describes different approaches for WhatsApp and the app interface, including messages that acknowledge the request and a visual loading experience. Demetrios Brinkmann mentions a you.com example where users were given trivia or other content while a page loaded. Rafael says iFood's intelligence and experience teams share goals such as retention and completed orders, so the experience team must design what users see while the model is working.

### Voice needs shorter answers and different system choices
[28:43](https://www.youtube.com/watch?v=dH-1INvvELo&t=1723s)
Daniel says voice ordering cannot present six products with long descriptions because the user would spend too long listening. A voice agent should make a concise recommendation, explain why it fits, and ask whether the user wants to continue. Text interfaces can show images, cards, and several options, while voice has one channel and needs more selective output. ILO keeps foundational intelligence configurable across channels, but the rendering and interaction change. Daniel also advises against unnecessarily chaining a voice-to-text model with another model for interpretation when a model can process voice and reasoning more directly.

### Agent scalability depends on shared data agreements
[39:55](https://www.youtube.com/watch?v=dH-1INvvELo&t=2395s)
Rafael says an agent usually consumes corporate data rather than generating all the information it needs. Before scaling, the team must align with the owners of those data models, APIs, and workflows. A slow finance report or unprepared API can create high agent latency even when the agent itself is working correctly. Rafael recommends asynchronous processing for information that can be prepared in the background. He frames scalability as an ecosystem issue, since the agent's performance depends on whether other parts of the company are ready to provide data at the required speed.

## Notable quotes
- Rafael Borger: "Recommending something that I know about the user is relatively easy. But how do I recommend something that I don't know if the user will like or not?" (06:46)
- Rafael Borger: "If you're really incredibly fast, you might, I think the general perception is that you did not think enough to bring me the answer." (23:20)
- Daniel Wolbert: "The intelligence like overall is almost the same, but the way we translate each into a user experience change very much from channel to channel." (38:00)
- Rafael Borger: "Before you build your agents, try to make sure that when you're going to scale your agent, you are aligned with the data owners." (40:42)
- Rafael Borger: "For the user, it's like one thing, iFood. He doesn't care if he's talking to an agent or any names." (46:11)

## Tools & references mentioned
- iFood
- ILO-Agent
- ILO
- LCM
- WhatsApp
- Pix
- Netflix
- TikTok
- you.com
- MCP apps
- Claude
- OpenAI
- computer-supported cooperative work
- Sean Ellis
- Chanel score

## Who should watch
- You are building a recommendation or ordering agent and need to combine user history with unfamiliar choices, budgets, delivery constraints, and dietary preferences.
- Your team is arguing about model speed without separating actual latency from what users see while they wait.
- You are scaling an agent across company data sources and need to plan for API ownership, asynchronous work, and support handoffs.

## Related talks

- [Context Engineering pitfalls for our e-commerce agent](https://mlopstalks.com/talks/context-engineering-pitfalls-for-our-e-commerce-agent) (Nishikant Dhanuka & Chiara Carateli, Prosus, 28:06)
- [The Latency Problem Nobody Talks About](https://mlopstalks.com/talks/the-latency-problem-nobody-talks-about) (Anoop Dawar, Deepgram, 20:38)
- [Real-Time Voice Agents in Production](https://mlopstalks.com/talks/real-time-voice-agents-in-production) (Panos Stravopodis, Elyos AI, 42:16)
- [Voice Agent Use Cases](https://mlopstalks.com/talks/voice-agent-use-cases) (Anurag Beniwal, ElevenLabs, 51:05)
- ["Real-Time" ML: Features and Inference](https://mlopstalks.com/talks/real-time-ml-features-and-inference) (Sasha Ovsankin & Rupesh Gupta, LinkedIn, 51:55)