LLMs can fabricate sources and produce faulty reasoning, so applications that need factual answers must ground responses in accessible evidence.
2
A useful knowledge assistant combines a knowledge base with guard rails that keep users within supported workflows and make the system say when it cannot answer.
3
Retrieval, knowledge evaluation, citations, and evals help measure and improve a grounded system, although conflicting or incomplete source data remains difficult.
Summary
Scott Mackie explains how to build an assistant that answers from a controlled knowledge base instead of relying only on an LLM's training data. He uses a Bevy game-development chatbot as the example. ChatGPT gave him outdated Bevy APIs, so he collected tutorials, documentation, code examples, and GitHub FAQ data. The system retrieves relevant material, places it in the response context, and tells the model to use only that material. Guard rails reject unrelated questions, while knowledge evaluation decides whether the retrieved evidence is sufficient and returns citations when it is. Scott also describes evals for expected questions, system capabilities, edge cases, user-driven cases, and unsupported workflows. He is direct about the remaining problems: temporal questions, conflicting documents, poor source data, and retrieval across many systems. His main recommendation is to provide grounded knowledge and guide users toward the areas the system can answer.
Hallucinations include invented facts and faulty reasoning
Scott separates hallucinations into fabrication of facts and faulty reasoning. His fabricated-source example uses GPT-3.5 Turbo, which recommends a behavioral economics paper with a citation that does not point to a real paper. In the reasoning example, the model is asked to compare two pounds of feathers with one pound of bricks and incorrectly says the feathers weigh less. He expects reasoning errors to improve as models improve, especially with techniques such as self-critique and tree of thought. His main focus is fabricated facts, where a system needs evidence that a user can inspect.
Real applications need retrieval because model weights do not contain live facts
Scott argues that larger context windows and better reasoning will not remove the need for information retrieval. A model cannot know the current basketball score from its weights, so an application has to place fresh data in context through retrieval or an API. He also rejects the idea that future models will provide complete trustworthiness on their own. When a system produces a fact, users should be able to trace it to a source. Retrieval makes that connection easier because the application knows which documents supplied the information.
A knowledge base gives the assistant material it can point to
Scott's Bevy chatbot collects several kinds of information in one place. He uses web tutorials, documentation and code examples, plus a GitHub FAQ accessed through an API. The downloaded material is a snapshot, while the GitHub source can provide live data. The knowledge base can contain lists of facts, documents, files, or website content. The purpose is to give the chatbot material it can reference when answering questions. This addresses the problem he encountered with outdated Bevy information, since the framework had changed many times since the model's training data.
Guard rails should limit unsupported answers without trapping users
The chatbot should answer questions about Bevy and Rust that its knowledge base supports, such as adding keyboard controls to a game. Scott uses the example of a geography question, the capital of Canada, to show why a correct general answer can still be wrong for the product. The system should explain that it cannot answer and direct the user back toward Bevy questions. He also uses math and action requests as examples of capabilities that may be outside the system. Tool or function calling can help classify requests and separate supported paths from unrelated ones.
Evals test whether the system covers its intended workflows
Scott describes evals as tests that compare inputs with outputs, while acknowledging that LLMs can produce many valid answers. Tests should isolate the parts that matter. For a keyboard-input question, he checks whether the answer explains key presses and connects them to game state. His eval categories include intended user questions, system capabilities, edge cases such as missing or conflicting data, user-driven cases, and unsupported workflows. He starts with simple text matching against a live staging or production-like system. The purpose is to measure changes over time rather than declare one run perfect.
Retrieval grounds generation when the prompt restricts the evidence
Scott describes an embedding-based approach with semantic search, while also mentioning Elasticsearch as an option. The retrieval layer takes a user query and returns relevant knowledge. The response-generation step receives those documents and is instructed to use only the supplied knowledge, rather than information from the model's training data. In his Bevy example, this lets the system return the current version found in the source material instead of the outdated version held by the model. The same pattern can support documentation, support centers, and other collections of facts.
Knowledge evaluation can reject weak evidence and attach citations
After retrieval, a separate knowledge-evaluation step decides whether the results are enough to answer. If the evidence is insufficient, the system can say it does not know and suggest where the user might look. If the evidence is useful, the evaluator identifies the documents that should be used in the generated response. The chatbot can then return a citation pointing to the source file. Scott also describes a conflicting-sources path, where the system shows both sources and lets the user inspect them instead of pretending it knows which one is correct.
Retrieval and source quality remain the hardest parts
Scott sees the most room for improvement in retrieval. Systems may need to search many APIs, document stores, or databases, and deciding where to look can make responses slow. He discusses hybrid retrieval, query generation against structured data, reranking, preprocessing documents, temporal queries, and multi-step planning. Knowledge graphs can help query structured entities, but extracting reliable triples from unstructured data is difficult. If the source data is wrong or conflicting, the assistant cannot simply make the answer trustworthy. Scott recommends making the uncertainty visible to the user.
"The two things that are really important when you're building these kind of systems providing access to the knowledge base and that's just to help bring the user to the spots where you have knowledge."31:51
Who should watch
You are building a support or documentation assistant and need answers tied to source material.
Your LLM application must refuse unrelated questions instead of answering every prompt from general model knowledge.
You need a practical starting point for testing retrieval, citations, guard rails, and response quality.
