# RAG in production

A pack of 8 sessions from the MLOps Community YouTube channel, in the order to watch them. 5h 26m of video.
Page: https://mlopstalks.com/packs/rag-in-production

The answer is somewhere in your documents, but retrieval misses it. Deleted files still appear in responses. A larger context window makes answers longer without making them better. Start by locating the failure in the path from source document to final answer. Then inspect extraction, document updates and chunk boundaries before changing the search model. The middle talks explain why nearby vectors are not necessarily relevant evidence, and when reranking earns its extra work. A documentation assistant shows how to compare retrieval and answer quality separately. Finish by questioning the machinery itself: some requests need an ordinary query, and extra retrieval stages deserve a place only when they improve representative questions. These 2023 and 2024 accounts supply design reasoning, not current product rankings.

## This pack is for you if

- Your assistant misses answers that are present in the document collection.
- Changed or deleted source documents leave stale passages in search results.
- You keep changing chunks and retrieval settings without understanding which stage fails.

## The talks, in order

### 1. A Survey of Production RAG Pain Points and Solutions

Jerry Liu, LlamaIndex | 30:00 | AI in Production 2024
Video: https://www.youtube.com/watch?v=pRhXoEXhWAM
Summary: https://mlopstalks.com/talks/a-survey-of-production-rag-pain-points-and-solutions.md

Why first: Liu separates missing source data, failed retrieval and a model that overlooks good context. That distinction gives you a way to choose the next experiment instead of replacing the whole stack. Begin with his diagnostic map, then follow the source documents through the less glamorous failures that can make every later stage look wrong.

### 2. False Starts and Dead Ends: Building a Retrieval Augmented Generation System

Wes Ladd, Train GRC | 11:44 | LLMs in Production 2023
Video: https://www.youtube.com/watch?v=_uPWW0ExiPw
Summary: https://mlopstalks.com/talks/false-starts-and-dead-ends-building-a-retrieval-augmented-generation-system.md

Why second: A file that claims to be a PDF may not parse, and a successful scraper may collect navigation instead of useful text. Ladd makes ingestion failure concrete. His warning about heading-based chunks also sets up the next question: how do you keep those extracted passages aligned with documents that change?

### 3. RAG Quality Starts with Data Quality

Adam Kamor, Tonic.ai | 59:34 | MLOps Podcast
Video: https://www.youtube.com/watch?v=n6e8BQUdnbI
Summary: https://mlopstalks.com/talks/rag-quality-starts-with-data-quality.md

Why here: Kamor follows documents beyond their first import. Source identifiers and incremental processing let a system replace changed files and remove deleted chunks. This gives the retrieval index a maintainable relationship with its inputs before you spend time comparing embeddings or search methods.

### 4. Embeddings and Retrieval for LLMs: Techniques and Challenges

Anton Troynikov, Chroma | 35:19 | LLMs in Production 2023
Video: https://www.youtube.com/watch?v=kZeOPapQ8yM
Summary: https://mlopstalks.com/talks/embeddings-and-retrieval-for-llms-techniques-and-challenges.md

Why here: Troynikov distinguishes a nearby vector from a result that matters to the user. With a usable corpus in place, compare models on your own examples and combine semantic matching with structure, keywords and metadata where needed. The next talk explains why a second model can judge the remaining candidates more carefully.

### 5. Supercharging Your RAG System: Techniques and Challenges

Tengyu Ma, Voyage AI and Stanford University | 40:20 | DE4AI 2024
Video: https://www.youtube.com/watch?v=_7h176mEv1M
Summary: https://mlopstalks.com/talks/supercharging-your-rag-system-techniques-and-challenges.md

Why here: A reranker reads the query and candidate together, gaining precision at a cost that broad vector search avoids. Ma makes that division of work clear, then shows how missing titles can make a chunk unintelligible. These are specific interventions to compare, not reasons to add every available retrieval stage.

### 6. Building RAG-based LLM Applications for Production

Philipp Moritz & Yifei Feng, Anyscale | 30:23 | LLMs in Production 2023
Video: https://www.youtube.com/watch?v=YuRFba27_1w
Summary: https://mlopstalks.com/talks/building-rag-based-llm-applications-for-production.md

Why here: This documentation assistant turns the preceding choices into a controlled comparison. Annotated source passages let the team test retrieval separately from generation, while chunk-size experiments expose different effects on retrieval and final answers. Use that separation before deciding that a higher search score has solved the user-facing problem.

### 7. RAG Has Been Oversimplified

Yujian Tang, Zilliz | 48:56 | MLOps Podcast
Video: https://www.youtube.com/watch?v=tz8xi3u0lu8
Summary: https://mlopstalks.com/talks/rag-has-been-oversimplified.md

Why here: Tang challenges two assumptions that the experiments can otherwise hide: retrieved context does not guarantee a correct answer, and some applications need no generated answer at all. Citations preserve a route back to evidence. His search-only examples prepare the final test of whether added machinery is doing useful work.

### 8. The Real E2E RAG Stack

Sam Bean, Rewind.ai | 1:10:07 | MLOps Podcast
Video: https://www.youtube.com/watch?v=8uZst7pgOw0
Summary: https://mlopstalks.com/talks/the-real-e2e-rag-stack.md

Why last: Bean ends the escalation of techniques with a stopping rule. Start from understandable search and evaluation baselines, then keep additions only when representative questions improve without sacrificing other cases. After inspecting ingestion, retrieval and generation separately, you can judge the complete system by what it helps a reader find.