Pavol Bielik shows how aggregate accuracy can hide subsets of data where a model performs badly for systematic reasons.
2
LatticeFlow groups model errors and examines data splits, similar samples, and suspected failure modes so teams can find issues before deployment.
3
David Garnitz argues that reliable RAG systems require repeated experiments with extraction, chunking, embeddings, and retrieval rather than one fixed ingestion recipe.
Summary
This session combines two parts of the ML application pipeline. Pavol Bielik explains model blind spots, where a model looks strong on average but fails on a meaningful subset of examples. His examples include car damage images with fingers correlated with damage, image classes containing unusual examples, and data splits with poor representation or leakage. LatticeFlow analyzes errors and data structure to help teams test these hypotheses without manually labeling every item. David Garnitz then discusses VectorFlow, an open-source pipeline for extracting, chunking, embedding, and loading unstructured data into vector databases. He argues that RAG ingestion depends on the file type, use case, chunking method, metadata, embedding model, and language model, so teams need controlled experiments. His proposed evaluation method uses known questions and answers, then compares ingestion configurations while changing one variable at a time.
Average accuracy can hide systematic failures in a small subset of data
Pavol Bielik asks whether a model with a higher aggregate accuracy is truly better and what is keeping a team from improving past its current performance. He treats every dataset as a collection of individual predictions and errors. Those errors may come from rare examples, incorrect labels, spurious correlations, or ambiguity. The aim is to unwrap the collection of mistakes into groups that have an interpretable cause. On an ImageNet example, a model with 95% aggregate accuracy performs much worse on a subset of images containing children in particular clothing. The average number does not reveal that failure.
Spurious correlations can make a model use the wrong evidence
A car damage inspection system may learn that fingers signal damage because people naturally point at damaged areas when taking pictures. The fingers are correlated with the label, but they are the wrong reason for the prediction. Pavol also describes testing other hypotheses, such as whether images are zoomed in. The practical challenge is checking these ideas across a large dataset without manually labeling every image. LatticeFlow uses statistical analysis to identify subsets that support a hypothesis and to make the result useful for future, unlabeled examples.
Blind-spot analysis can expose large performance gaps behind a good overall score
In one aerial-image dataset, the aggregate performance is 88%, but the recall falls to 22% or 32% for a discovered subset. In another example, ambulance images have a subset where the model is 97% correct and another where it is only 25% correct. The unusual images include a helicopter, an ambulance far in the distance, a vehicle that is partly an SUV, and an ambulance without an open door. A watermark dataset produces a similar contrast, with one subset that is 98% correct and another where the model makes essentially no correct predictions.
Standard embeddings may not reveal the failure pattern a model is using
Pavol compares the discovered blind spots with ordinary embedding views. In one region of an embedding space, the model is 99.9% correct, while another region falls to 68%, but neither view exposes the much worse subset found by the automated analysis. He explains that embeddings are an unsupervised projection of the original data, so they can miss the depth needed for this analysis. Different models, including foundational models, produce different representations, which changes how errors are distributed. The embedding view is useful, but it is not enough on its own.
Dataset splits and similar samples can undermine evaluation
LatticeFlow compares dataset splits to find representativeness problems and leakage. Some clusters have no test images at all, so they cannot support a meaningful test of generalization. The tool also groups similar images automatically. Pavol shows how teams can filter for highly similar items in the test set and inspect whether duplicates or near-duplicates are concentrated in training and validation. In his example, the test set has relatively few similar images, while training and validation contain more. These checks help distinguish a model problem from a dataset construction problem.
Unstructured-data ingestion has many choices with no single best recipe
David Garnitz describes ingestion as a combination of source systems, target vector databases, file formats, extraction methods, chunking strategies, metadata, embedding models, and language models. A PDF with tables can be read as text, processed with OCR, or handled with a mixture of computer vision and NLP. A table also raises questions about whether to keep rows together, concatenate columns, or include headings. Large files may exceed one machine's capacity or a model's context window. Since the right approach depends on the data and use case, teams need to test configurations rather than assume a universal method.
Production infrastructure should be separated from ingestion experiments
Teams experimenting with ingestion may also need a message broker, queues, distributed compute, retries, logging, and Kubernetes orchestration. David argues that these pieces are not the part of the application where teams create value. VectorFlow provides an embedding pipeline with defaults for extraction and chunking, while allowing users to replace those defaults with their own Python logic. It can use connectors from Llama Hub and supports different vector databases. The goal is to let a team change an ingestion configuration and run the pipeline again without rebuilding the production infrastructure around every experiment.
RAG quality requires controlled experiments on representative data
David describes an evaluation process from a legal technology project that ingested as many as five million documents. The team selected a subset of well-understood PDFs, wrote about 25 questions and their answers, and used ChatGPT to judge whether generated answers were correct. They ran the evaluation multiple times and averaged the scores to reduce model variability. For a fair comparison, they changed one input at a time, such as the embedding model or chunking strategy. He recommends testing a substantial fraction of the data before processing everything, with enough vectors to judge how the vector search algorithms behave.
Open models and shared foundations may reduce lock-in over time
The speakers discuss open and closed models in the context of production, regulation, privacy, and maintenance. David says his users often choose closed models because they are easy to adopt or fit an existing cloud stack, while open-source embeddings are used by a smaller share of users during experimentation. He also describes cases where open models were chosen for security, compliance, or cost reasons. Pavol argues that accessible models make it easier for people to inspect systems and report vulnerabilities. Their closing advice is to build shared tooling and repeatable practices instead of chasing every new model release.
"It is very important to build the common foundation on top of which everybody can build and would allow us to make systematic progress."Pavol Bielik54:22
Who should watch
You are evaluating a vision model whose overall score looks good, but you suspect it fails on particular conditions or groups of examples.
You are building a RAG system and need to compare chunking, extraction, metadata, embedding, and vector-search choices before processing all your data.
Your team is spending time on queues, retries, distributed workers, and connectors when the harder problem is finding an ingestion configuration that gives good answers.
