Adam Kamor argues that RAG quality depends first on giving the language model the right context, which requires preparing and structuring source data well.
2
Chunking should match the document type, and a single strategy may not work across an entire corpus.
3
Sensitive information should be removed or replaced before it reaches the vector database, while access control belongs in the database layer.
Summary
Adam Kamor explains why RAG projects often fail before retrieval begins. The source documents need to be extracted correctly, converted into a usable structure, chunked according to their format, and kept up to date as files change. Tonic.ai's Textual uses named entity recognition to find sensitive and useful entities in unstructured text. It can redact information or synthesize safe replacements while preserving the meaning of a chunk. Kamor separates this data preparation work from row-based access control, which he sees as a database responsibility. He also describes incremental processing for added, modified, and deleted files, plus the use of metadata to remove stale chunks from a vector database. Near the end, he discusses evaluation through Tonic Validate, which measures retrieval context, answer quality, privacy indicators, latency, cost, and other signals. He is direct about the limits of current evaluation methods, especially for complex, agentic, and multimodal RAG systems.
RAG answers depend on the context supplied to the model
Adam Kamor says Tonic.ai started building Textual after seeing that data quality mattered more than other parts of their internal RAG systems. If the language model receives the wrong context, it will not reliably answer questions about private data. Textual is designed to build pipelines that produce chunks suited to retrieval. Kamor agrees that documentation quality is a prerequisite, although he also points out that companies usually have large amounts of existing data, sometimes more than they need. Their early RAG use cases were internal assistants for HR questions and company handbooks, while newer systems increasingly use customer data in externally facing applications.
Named entity recognition can improve safety and retrieval together
Textual runs unstructured data through in-house named entity recognition models. These models can identify sensitive entities such as Social Security numbers and credit card numbers, which should generally not enter a RAG system. They can also find useful context, such as the product discussed in a customer service transcript or the company where someone works. Kamor says those entities can be used to improve chunks as well as to remove or protect sensitive information. The data pipeline therefore addresses both the risk of leakage and the quality of retrieval metadata.
Chunking has to follow the document and the use case
Kamor rejects the idea that one chunking method will work for every dataset. Textual includes chunking techniques, but it also lets users insert their own purpose-built algorithms into the pipeline. He gives FAQs as a simple example: each question and answer pair can be kept as its own chunk. That approach would not automatically fit other document types. A corpus may need different methods for different classes of documents, and the right choice depends on the data, the retrieval task, and other system variables.
Document extraction is a separate engineering problem for each file type
Before chunking, teams need to understand what documents they have, where they live, and how they change. Office files can be traversed programmatically because Word documents, spreadsheets, and presentations contain structured content. PDFs often require optical character recognition, while images and plain text need other extraction paths. Kamor describes converting complex files into Markdown so that text, lists, tables, headings, and titles can remain represented in a plain-text format. Textual also creates a structured JSON counterpart for each source document, giving downstream code a consistent schema regardless of the original file type.
Tables and cross-references often need additional processing
Tables are especially difficult in PDFs because they may be regular grids or highly stylized designs. Kamor mentions table detection and extraction through tools such as PaddleOCR, Azure Document Intelligence, and AWS Textract. Another approach combines OCR with a language model that reformats extracted text and identifies tables. A separate pass may be needed to connect a paragraph with the table it references. Kamor admits that sending data through another language model is not always ideal, but says it can be faster than training a specialized model for a particular task. Tonic.ai instead uses fine-tuned BERT and RoBERTa models for its named entity recognition work.
Incremental processing keeps changed source data aligned with retrieval data
Textual tracks whether a source file has been added, modified, or deleted. For storage systems such as Amazon S3, this can use checksums, while systems such as Jira expose last-updated information. The first run processes the full backlog. Later runs calculate a delta and process only what changed. For a modified file, Kamor recommends deleting all of its existing chunks and recreating them rather than trying to update individual chunks. Deletions require the vector database to identify which chunks came from the removed file, so the stored chunk metadata needs a pointer or other relationship back to its source.
A vector database still needs ordinary database design
Kamor says teams should treat vector databases as databases, even when they store embeddings. They still need ordinary operations such as filtering, deletion, and access control. A source identifier stored alongside each chunk makes it possible to remove chunks when a file disappears. For smaller or familiar deployments, he describes using PostgreSQL with the PGVector extension and advises teams to start with tools they already know until they reach a real limitation. He does not assume that every RAG application needs a specialized vector database immediately.
Privacy filtering and row access control belong in different parts of the system
Kamor distinguishes between deciding what data may enter a vector database and deciding who may retrieve a row. Tonic.ai's role is to detect sensitive entities and redact or synthesize them before indexing. In synthesis, a real name or identifier can be replaced with a plausible substitute while preserving the meaning of the text. This allows customer service conversations to provide general help without exposing names, credit card details, or Social Security numbers. User-specific visibility, including private channels and documents, is a row-based access control problem that Kamor assigns to the database and application architecture.
RAG evaluation needs several metrics and still has open problems
Kamor describes Tonic Validate as an open-source Python SDK and web interface that treats an application as a black box. It evaluates both the retrieved context and the generated answer. Answer similarity can compare a generated response with a human-written reference, although that method is expensive. Other metrics examine context relevance, how much of the answer comes from the context, the presence of personally identifiable information, latency, cost, token use, and exact strings. Kamor says these measurements make trends easier to observe, but current methods become less satisfactory for multi-step, agentic, and multimodal RAG systems.
"Data quality really trumps everything else when it comes to getting good quality answers from your RAG system."Adam Kamor05:04
Who should watch
You are building an internal or customer-facing RAG assistant and need a practical data-ingestion plan before choosing embeddings or a retrieval framework.
Your source material includes PDFs, office files, tables, customer conversations, or frequently changing tickets, and you need to handle extraction and updates safely.
You need to separate privacy filtering from database access control and want an honest account of what current RAG evaluation can measure.
