Pack · 8 talks · 5h 26m to watch, 48 min to read

RAG in production

The answer is somewhere in your documents, but retrieval misses it. Deleted files still appear in responses. A larger context window makes answers longer without making them better. Start by locating the failure in the path from source document to final answer. Then inspect extraction, document updates and chunk boundaries before changing the search model. The middle talks explain why nearby vectors are not necessarily relevant evidence, and when reranking earns its extra work. A documentation assistant shows how to compare retrieval and answer quality separately. Finish by questioning the machinery itself: some requests need an ordinary query, and extra retrieval stages deserve a place only when they improve representative questions. These 2023 and 2024 accounts supply design reasoning, not current product rankings.

1
Jerry Liu, LlamaIndex · 30:00 · AI in Production 2024
A Survey of Production RAG Pain Points and Solutions

Why first: Liu separates missing source data, failed retrieval and a model that overlooks good context. That distinction gives you a way to choose the next experiment instead of replacing the whole stack. Begin with his diagnostic map, then follow the source documents through the less glamorous failures that can make every later stage look wrong.

2
Wes Ladd, Train GRC · 11:44 · LLMs in Production 2023
False Starts and Dead Ends: Building a Retrieval Augmented Generation System

Why second: A file that claims to be a PDF may not parse, and a successful scraper may collect navigation instead of useful text. Ladd makes ingestion failure concrete. His warning about heading-based chunks also sets up the next question: how do you keep those extracted passages aligned with documents that change?

4
Anton Troynikov, Chroma · 35:19 · LLMs in Production 2023
Embeddings and Retrieval for LLMs: Techniques and Challenges

Why here: Troynikov distinguishes a nearby vector from a result that matters to the user. With a usable corpus in place, compare models on your own examples and combine semantic matching with structure, keywords and metadata where needed. The next talk explains why a second model can judge the remaining candidates more carefully.

5
Tengyu Ma, Voyage AI and Stanford University · 40:20 · DE4AI 2024
Supercharging Your RAG System: Techniques and Challenges

Why here: A reranker reads the query and candidate together, gaining precision at a cost that broad vector search avoids. Ma makes that division of work clear, then shows how missing titles can make a chunk unintelligible. These are specific interventions to compare, not reasons to add every available retrieval stage.

6
Philipp Moritz & Yifei Feng, Anyscale · 30:23 · LLMs in Production 2023
Building RAG-based LLM Applications for Production

Why here: This documentation assistant turns the preceding choices into a controlled comparison. Annotated source passages let the team test retrieval separately from generation, while chunk-size experiments expose different effects on retrieval and final answers. Use that separation before deciding that a higher search score has solved the user-facing problem.

7
Yujian Tang, Zilliz · 48:56 · MLOps Podcast
RAG Has Been Oversimplified

Why here: Tang challenges two assumptions that the experiments can otherwise hide: retrieved context does not guarantee a correct answer, and some applications need no generated answer at all. Citations preserve a route back to evidence. His search-only examples prepare the final test of whether added machinery is doing useful work.

8
Sam Bean, Rewind.ai · 1:10:07 · MLOps Podcast
The Real E2E RAG Stack

Why last: Bean ends the escalation of techniques with a stopping rule. Start from understandable search and evaluation baselines, then keep additions only when representative questions improve without sacrificing other cases. After inspecting ingestion, retrieval and generation separately, you can judge the complete system by what it helps a reader find.