# Goal Oriented Retrieval Agents

Zoe Weil, Faber Labs | Agents in Production 2024 | 25:09

Source: https://www.youtube.com/watch?v=cJ_sNYes9CA
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/goal-oriented-retrieval-agents
Published: 2024-12-18
Tags: latency, privacy, recommender-systems, reinforcement-learning

## TL;DR
- GORA optimizes retrieval and ranking around a client's measurable business goal, such as conversion rate, average order value, or lower surgical readmission rates.
- Faber Labs combines real-time feedback, end-to-end reinforcement learning, large event models, and conversational context to adapt rankings during a user session.
- Rust, parallel processing, component-level latency budgets, and GPU caching allow the system to support conversational retrieval with low response times.

## Summary
Zoe Weil presents GORA, or Goal-Oriented Retrieval Agents, as a retrieval and ranking system built around measurable client outcomes. Faber Labs uses it in e-commerce to optimize conversion rate and average order value, and in medical applications to help clinicians find alternatives to ineffective procedures while reducing readmission rates. The system uses client-specific goals, real-time user feedback, conversational context, large event models trained from client data, and an end-to-end reward layer. Weil explains that the team chose Rust for its backend because memory safety and concurrent processing helped reduce latency, infrastructure cost, and the difficulty of offering on-premise deployments. Intelligent GPU caching handles follow-up prompts, while parallel processing and component-level budgets control response times. She also describes training large event models from scratch and learning across clients without sharing private information. The talk is candid about the cost of adopting Rust and the engineering work required for privacy-sensitive deployments.

## Key ideas
### GORA applies retrieval to measurable business outcomes
[00:54](https://www.youtube.com/watch?v=cJ_sNYes9CA&t=54s)
Weil defines GORA as specialized agents that seek to maximize a chosen KPI. In retail, that can mean improving conversion rate or average order value. In value-based care, the goal can be reducing surgical readmissions. Faber Labs wants to provide this capability to marketplaces, retailers, and consumer-facing businesses as an embedded optimization layer. The system is intended for situations where changing what a user sees can change a measurable outcome, rather than only helping users discover information.

### The system puts real-time user feedback at the centre of relevance
[04:39](https://www.youtube.com/watch?v=cJ_sNYes9CA&t=279s)
GORA uses three design pillars: user behavior, contextual insights, and real-time adaptation. Historical behavior matters, but Weil says in-session feedback is especially important. The system can react to small signals from a user and adjust rankings during the interaction. Faber Labs also developed large event models to generalize from user event data and recognize event sequences that were not present in training data. Weil presents this as analogous to how language models generalize beyond the text seen during training, while also emphasizing the privacy challenge around behavioral and medical data.

### A shared client goal aligns the retrieval pipeline
[06:07](https://www.youtube.com/watch?v=cJ_sNYes9CA&t=367s)
GORA defines goals at the client level and optimizes its models end to end around the selected outcome. The pipeline includes embedding generators, reranking models, and agent models, which are jointly optimized in a final reinforcement learning step. Weil contrasts this with stacked ranking systems whose components may optimize different objectives. She gives the example that higher conversion can coincide with lower gross merchandise sales if customers buy cheaper products. A unified goal and feedback loop lets the system optimize related metrics together instead of treating each model independently.

### Conversational feedback changes context and agent decisions
[06:55](https://www.youtube.com/watch?v=cJ_sNYes9CA&t=415s)
The system is built around short, interactive feedback loops rather than static queries. Feedback affects context selection, agent behavior, and the overall optimization process. This matters because GORA handles conversations that can grow quickly as users send follow-up prompts. The system must retain enough conversational context to make effective decisions, while its purpose remains influencing a target outcome. Weil distinguishes this from a knowledge discovery platform, where the primary task is answering or retrieving information without directly optimizing a business or clinical metric.

### Latency is managed at the component and infrastructure levels
[09:00](https://www.youtube.com/watch?v=cJ_sNYes9CA&t=540s)
Weil describes latency as a direct user-experience concern and says Faber Labs assigns latency budgets to individual components. Parallel processing reduces the cost of communication between components. The backend is written almost entirely in Rust, a decision that followed a major internal debate because the team had stronger experience with Python and Scala. Weil says Rust's memory safety, concurrency, and low-level control improved efficiency, privacy, security, and cost. The system also uses intelligent GPU caching to avoid regenerating key-value caches for follow-up prompts.

### The team trains large event models from client data
[20:35](https://www.youtube.com/watch?v=cJ_sNYes9CA&t=1235s)
In the question period, Weil explains that Faber Labs trains its large event models from scratch. The models use data supplied by clients, including messy data that the system must handle without extensive cleanup. Open-source large language models are still used for components that connect the system and present results to users, while the main ranking work relies on the large event models. A reward-layer alignment function allows learning from feedback across clients without transferring direct private information between them.

### On-premise deployment supports privacy-sensitive customers
[18:37](https://www.youtube.com/watch?v=cJ_sNYes9CA&t=1117s)
Faber Labs offers an on-premise option for medical customers and is also using that approach with neo banks. Weil describes on-premise deployment as an additional engineering burden, especially for an early bootstrapped company. She credits the lightweight, fast backend with making this option practical. The deployment model keeps sensitive data within the customer's environment, while the broader system can still use feedback and event data without leaking direct information between clients.

### Moving to Rust requires learning a different way to structure code
[22:46](https://www.youtube.com/watch?v=cJ_sNYes9CA&t=1366s)
Weil says Rust was difficult because it required more than learning new syntax. It changed how she thought about code, especially after years of working with Python and Scala. Her advice is to use the available online learning communities and study code related to an existing area of expertise. She found it useful to inspect Hugging Face backend code because she understood what the code needed to do even when she did not yet understand Rust. She also describes the Rust community as welcoming and willing to answer questions on GitHub.

## Notable quotes
- "GORA, or goal-oriented retrieval agents, are the first of their kind specialized agents that autonomously seek to maximize any KPI." (00:54)
- "Our success hinges on these unified goals actually across different models." (06:07)
- "One of our best and most important decisions has been to run our back-end almost entirely in Rust." (09:44)
- "We can just use their messy data." (21:21)
- "Rust isn't just a new programming language. It's like you have to rethink how you think about code in a lot of ways as well." (23:08)

## Tools & references mentioned
- Faber Labs
- GORA
- Amazon
- Reverb
- Rust
- Python
- Scala
- Discord
- Hugging Face
- Marcin Meyran
- large language models

## Who should watch
- You are building search, recommendation, or retrieval for a product where ranking must affect a measurable customer or clinical outcome.
- Your agent system needs to use conversational feedback while staying within tight latency limits.
- You are considering Rust for an ML backend or need an on-premise design for privacy-sensitive customers.

## Related talks

- [RagSys: RAG is just RecSys in Disguise](https://mlopstalks.com/talks/ragsys-rag-is-just-recsys-in-disguise) (Chang She, LanceDB, 12:45)
- [Agents as Search Engineers](https://mlopstalks.com/talks/agents-as-search-engineers) (Santoshkalyan Rayadhurgam, Meta, 29:38)
- [System Design for Recommendations and Search](https://mlopstalks.com/talks/system-design-for-recommendations-and-search) (Eugene Yan, Amazon, 58:23)
- [Supercharging Your RAG System: Techniques and Challenges](https://mlopstalks.com/talks/supercharging-your-rag-system-techniques-and-challenges) (Tengyu Ma, Voyage AI and Stanford University, 40:20)
- [Using LLMs to Power Consumer Search at Scale](https://mlopstalks.com/talks/using-llms-to-power-consumer-search-at-scale) (Aravind Srinivas, Perplexity AI, 37:04)