Watch on YouTubeA good retrieval-augmented generation system should improve over time as a data flywheel, with the underlying knowledge becoming better as the system is used.
Anthony Alcaraz argues that knowledge graphs are a better foundation for retrieval than vector databases because they combine structured relationships with graph-based reasoning.
Graphs can support LLM applications in three ways: as context providers, as reasoning topologies, and as targets for transformer and LLM-based graph algorithms.
Anthony Alcaraz presents knowledge graphs as the foundation for retrieval-augmented generation systems that need to reason over business data. He argues that RAG should be designed as a data flywheel, where the system and its knowledge improve over time. Knowledge graphs provide structured entities and relationships, while graph algorithms add capabilities such as clustering, centrality analysis, and causal reasoning. Alcaraz describes three interactions between graphs and LLMs: graphs can provide context, define a topology for reasoning, or become the object of an LLM-based graph task. He also discusses using LLMs to build knowledge graphs from documents and domain data. His proposed architecture separates knowledge retrieval from reasoning. Retrieval should rely mainly on a knowledge graph and several forms of graph augmentation, while reasoning can use fine-tuned LLMs, NLP models, or classifiers trained on the graph. He connects these ideas to Fribl's work on automating HR reasoning.
Anthony Alcaraz describes retrieval-augmented generation as a system design pattern for extracting value from LLMs in business. He says a RAG system should improve over time rather than remain static. The data and knowledge behind retrieval need a process that automates this improvement. He sees the quality of the data supplied to the model as a future differentiator, especially when that data contains structure and relationships.
Alcaraz says fine-tuning may increase a model's reasoning capabilities, but knowledge graphs provide a separate set of tools for reasoning over data. These include Cypher queries, vectorization, and graph algorithms. He describes the interaction between an LLM and a knowledge graph as agentic RAG, where an agent reasons about what it retrieved and what it still needs from the graph.
Alcaraz contrasts graph retrieval with vector search and cites research from Microsoft and researchers associated with MIT. He says graph-based RAG produced more diverse viewpoints per answer in one comparison, with the cited result being 1.8 more diverse viewpoints per answer. He connects that diversity to answers that are more comprehensive.
Alcaraz identifies three forms of interaction. A graph can provide context for an LLM, with graph algorithms such as clustering helping augment the retrieved context. A graph can define a reasoning topology, as in Graph of Thoughts or approaches using Monte Carlo algorithms. LLMs and transformers can also perform graph tasks, such as finding centrality or influential parts of a graph.
Alcaraz discusses a model called RARE, or Retrieval Augmented Thought Process, which extends the Graph of Thoughts idea by retrieving knowledge for each thought. The LLM scores answers, creating a way to improve the process over time. He also describes combining graph retrieval, graph reasoning, parallel querying, planning, and cyclical graphs. In a cyclical graph, a model can return to an earlier point and try another option after failing.
Alcaraz argues that the cost of building a knowledge graph can be reduced because LLMs can assist with its construction. He mentions turning documents directly into graphs, extracting domain information, generating graph data models from CVs, fine-tuning models for domain extraction, and using an agent system called AutoKG. These techniques allow graph creation to become part of the same LLM and data workflow.
Alcaraz lists several ways to retrieve and augment context from a knowledge graph. These include Cypher queries with prior ontology grounding, vector similarity search, graph algorithms, and generative knowledge graphs. His recommendation is to chain these forms of context augmentation rather than depend on only one retrieval method.
Alcaraz divides a RAG system into knowledge retrieval and reasoning. He recommends using a knowledge graph for the retrieval side. The reasoning side can use different LLMs fine-tuned for reasoning tasks, as well as NLP models and classifiers trained on the knowledge graph. He presents this separation as a way to choose different models for finding knowledge and making judgments.
In his final example, Alcaraz describes research combining an LLM with a graph algorithm for causal reasoning over a causal knowledge graph. He says the method reduces complexity and lowers LLM usage by dividing work between the model and the graph algorithm. He presents this as an example of using both technologies together as a reasoning engine for business applications.
"My system, you need to think a RAG system into two parts. You have a knowledge retrieval part and you have a reasoning part."11:45