# RagSys: RAG is just RecSys in Disguise

Chang She, LanceDB | AI in Production 2024 | 12:45

Source: https://www.youtube.com/watch?v=-wEvrIaldRg
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/ragsys-rag-is-just-recsys-in-disguise
Published: 2024-04-08
Tags: embeddings, rag, recommender-systems, reranking

## TL;DR
- Retrieval quality becomes a production concern when incorrect context can lead an AI system to produce costly errors.
- RAG pipelines resemble recommender systems because chunking works like feature engineering, retrieval methods work like recallers, and reranking selects the context passed to the language model.
- Hybrid search and reranking can improve retrieval, while user feedback and synthetic data can support later embedding-model fine-tuning.

## Summary
Chang She argues that production RAG has more in common with recommender systems than many teams acknowledge. A RAG pipeline experiments with chunking, embedding models, full-text search, vector search, and reranking in much the same way that a recommender system builds features, gathers candidates, and ranks them. He demonstrates this with LanceDB using an Airbnb SEC filing, showing how hybrid search combines vector and full-text scores, then how a Cohere reranker surfaces context about international operations in China. Chang also points to feedback as the next step for improving RAG retrieval, including an example where synthetic data was used to fine-tune an embedding model. He is careful about the limits of the analogy. RAG has a generation step, relevance is tied to a question rather than a user, and small rank differences may matter less when several retrieved passages fit within the context window.

## Key ideas
### Production RAG depends on retrieval accuracy
[02:26](https://www.youtube.com/watch?v=-wEvrIaldRg&t=146s)
Chang describes RAG as an open-book exam that extends a model's knowledge without changing the model. Building a demo is easy, but production systems depend on retrieving the right material. He gives an example of someone writing 500 basis points when they meant 50, after which a company's stock fell when the error was corrected. That example raises difficult questions about responsibility when an AI application makes a similar mistake. Chang's practical conclusion is that retrieval quality and accuracy matter before the retrieved context reaches the language model.

### RAG retrieval looks like a recommender pipeline
[03:19](https://www.youtube.com/watch?v=-wEvrIaldRg&t=199s)
Chang maps common RAG choices onto recommender-system concepts. Chunking experiments resemble feature engineering. Embeddings from different sources and full-text, SQL, or graph search act like recall methods that gather candidates. The system then combines those results and reranks them before producing an answer. In a content recommender, the analogous inputs can include user history, content metadata, demographics, posters, and subtitles. The final ranked output might determine what appears in a Netflix home grid or an Amazon product list.

### LanceDB supports a hybrid search workflow
[04:34](https://www.youtube.com/watch?v=-wEvrIaldRg&t=274s)
Chang walks through a notebook using an Airbnb financial dataset. A PDF loader reads an SEC filing, a recursive character splitter breaks it into chunks, and LanceDB stores the page content with vectors generated through an OpenAI embedding model. A vector query asks which factors contributed to Airbnb's increased operating expenses in the last fiscal year. Hybrid search adds a full-text index over the page content and combines normalized vector and text-search scores into a relevance score.

### Full-text search can change the retrieved evidence
[06:45](https://www.youtube.com/watch?v=-wEvrIaldRg&t=405s)
The vector-only search returns relevant material about financial headwinds. When Chang switches to hybrid search, the second result changes and includes a passage about a contractor delivering a pricing certificate. The phrase "fiscal year" in the query helps full-text search find that passage. The example shows why combining semantic similarity with exact text matching can alter which evidence reaches the next stage of the RAG pipeline.

### Rerankers can surface more direct context
[08:00](https://www.youtube.com/watch?v=-wEvrIaldRg&t=480s)
LanceDB supports several reranking approaches. Chang says the Cohere reranker performed best in his example without fine-tuning, and the API keeps a similar data-frame structure while changing the scores and contents. When he compares responses generated from vector-search context with responses generated from Cohere-ranked context, the latter identifies international operations, particularly in China, as the main factor. He also mentions ColBERT, cross-encoder, and OpenAI-based rankers, plus custom rankers that users can write and plug into the search process.

### Feedback can improve retrieval over time
[09:53](https://www.youtube.com/watch?v=-wEvrIaldRg&t=593s)
Chang says recommender systems improve through use because they collect feedback, and RAG systems are beginning to use the same idea. He cites a user who used about $10 of synthetic data to fine-tune an embedding model and make it better than generic best-in-class embedding models for the task. He presents feedback and task-specific training as an area LanceDB plans to work on next.

### The recommender analogy has limits
[10:40](https://www.youtube.com/watch?v=-wEvrIaldRg&t=640s)
Chang says RAG and recommender systems are not identical. RAG adds a generation step. Recommender relevance is defined for a particular user, while RAG relevance is defined for a particular question. Ranking precision can matter greatly on a recommender home page, whereas the difference between first, second, and third may matter less in RAG when the top passages all fit inside the model's context limit. The analogy remains useful for borrowing ideas, but quality measurement must account for these differences.

## Notable quotes
- "With RAG, it's sort of the open-book exam of AI where you can extend the model knowledge without having to touch the model itself." (02:26)
- "In production, quality of your retrieval really matters." (02:26)
- "The chunks are just features, and different chunking experiments are more like feature engineering." (03:19)
- "Analogies have limitations, right? So it's not a perfect one-to-one parallel between RAG and recommender systems." (10:40)

## Tools & references mentioned
- LanceDB
- Pandas
- TubiTV
- RAG
- recommender systems
- Airbnb
- OpenAI
- LangChain
- Cohere
- ColBERT
- cross-encoder
- ChatGPT
- Discord

## Who should watch
- You are building a RAG system and need to decide whether vector search alone is enough for production retrieval.
- You work on search or recommendations and want a concrete mapping between recall, feature engineering, hybrid search, and reranking.
- You want an example of using LanceDB with document chunks, embeddings, full-text search, and configurable rerankers.

## Editor's note

Chang She says production RAG depends on retrieving the right material before it reaches the language model, because incorrect context can produce a costly error. ZenML lets teams define retrieval workflows as Python pipeline steps and records each run's inputs, outputs, and code version, so a result can be traced to the data and code that produced it.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Supercharging Your RAG System: Techniques and Challenges](https://mlopstalks.com/talks/supercharging-your-rag-system-techniques-and-challenges) (Tengyu Ma, Voyage AI and Stanford University, 40:20)
- [Does AgenticRAG Really Work?](https://mlopstalks.com/talks/does-agenticrag-really-work) (Satish Bhambri, Walmart Labs, 1:01:40)
- [RAG Has Been Oversimplified](https://mlopstalks.com/talks/rag-has-been-oversimplified) (Yujian Tang, Zilliz, 48:56)
- [Retrieval Augmented Generation](https://mlopstalks.com/talks/retrieval-augmented-generation) (Syed Asad, KiwiTech, 44:10)
- [A Survey of Production RAG Pain Points and Solutions](https://mlopstalks.com/talks/a-survey-of-production-rag-pain-points-and-solutions) (Jerry Liu, LlamaIndex, 30:00)