The paper defines context engineering as assembling and managing a dynamic set of instructions, retrieved knowledge, tools, memory, state, and user queries rather than writing one static prompt.
2
RAG systems can become more selective by decomposing questions, choosing retrieval methods, checking relevance, and using graph structure to represent entities and relationships.
3
Tool use and multi-agent systems require models to reason about when to call functions, how to chain tools, and how specialized agents should coordinate their context and work.
Summary
A Survey of Context Engineering for Large Language Models is the paper discussed in this reading group. Adam Becker presents the paper's framework, which treats context as a dynamically assembled set of instructions, external knowledge, tools, memory, state, and user input. He groups the work into context generation and retrieval, processing, and management. Rohan Prasad then examines modular and agentic RAG, with a focus on graph-based retrieval through LightRAG, and explains the difference between stateless RAG and stateful memory. Matt Squire covers tool-integrated reasoning and multi-agent systems. He connects tool use with function calling, environment interaction, and the ability to chain operations such as reading logs, inspecting source code, and opening a pull request. The discussion also covers graph updates, multimodal retrieval, memory evaluation, and the operational difficulty of maintaining these systems. The presenters leave evaluation and dataset research mostly aside because of time.
Context engineering assembles a structured system input
Adam Becker says prompt engineering traditionally treats context as one static string. The survey instead describes a dynamically structured set of components assembled for the model. These can include system instructions, external knowledge, tools and function calls, persistent memory, the user's state, the world's state, a multi-agent system, and the user's query. The aim remains to improve system performance, but the assembly must work within computational and memory limits that grow with sequence length. This is the paper's basic distinction between writing prompts and engineering the information environment around a model.
The survey organizes context work into generation, processing, and management
Adam Becker compares the three foundational areas to preparing food. Context generation and retrieval source the ingredients, context processing cooks them together, and context management decides what stays available, what is stored, and what is removed. Generation includes zero-shot, one-shot, and few-shot prompting, chain-of-thought methods, tree or graph structures for reasoning, and retrieval from external knowledge. Processing includes long-sequence architectures, prompt refinement, and multimodal integration. Management includes hierarchical memory systems that move information between active context and longer-term storage.
Retrieval research is moving beyond one-shot vector search
The survey examples Adam Becker gives include compositional RAG, recursive retrieval and summarization, systems that decide whether retrieval is needed, and retrieval from knowledge graphs. A compositional system can break a question into smaller parts and retrieve evidence for each part. RAPTOR recursively clusters and summarizes text into a tree with different levels of abstraction. Other work prepends facts from a knowledge graph to the input question. These approaches treat retrieval as a process that can decompose, select, and organize information before it reaches the model.
Agentic and modular RAG make retrieval an iterative decision process
Rohan Prasad describes basic RAG as retrieving data and passing it into context, then contrasts it with modular and agentic systems. The system can first decide whether retrieval is needed, break a question into subquestions, choose between dense retrieval and keyword search, compare the relevance of returned information, and repeat the process. Dependencies between subquestions matter. An answer to one part can become an input to the next. Rohan uses a question about the plague, Venus, and a place of worship to show why decomposition is useful when a single query contains several linked questions.
Graph RAG adds entity and relationship structure to retrieval
Rohan Prasad explains LightRAG as a system that segments documents, uses an LLM to identify entities and relationships, stores entity and relationship vectors, and builds a graph that can merge identical entities and relationships. Low-level retrieval finds a particular entity and nearby nodes. High-level retrieval traverses the graph to answer broader questions about how concepts connect. In his example, asking for a character's immediate relationships differs from asking for a book's main character, which may require examining a wider set of connections. He also demonstrates indexing A Christmas Carol and comparing simpler retrieval with graph traversal.
RAG is stateless while memory preserves an agent's state
Rohan Prasad draws a basic distinction between the two systems. RAG is the agent's input and output connection to the outside world, while memory is the agent's internal state. Short-term memory contains the active conversation context. Long-term memory preserves preferences, history, and user state. Sending an entire conversation history to a model creates its own problem because models struggle with extended context, so a useful system must select the relevant portion. Rohan describes Mem0 as maintaining factual, episodic, and semantic memory, resolving conflicts such as a user first saying they like pizza and later saying they do not.
Tool-integrated reasoning lets models act through deliberate tool sequences
Matt Squire describes tool-integrated reasoning as a model's ability to interact with the real world. The components include function calling, reasoning about which function to use, and interaction with an environment. Tools can search, calculate, query databases, ask users for information, or call APIs. Matt gives a site reliability example from Fuzzy Labs: an agent receives a production alert in Slack, reads application logs, retrieves source code from GitHub, diagnoses the issue, and may create an issue or pull request. The important capability is reasoning about the order and purpose of these tool calls.
Multi-agent systems divide work across specialized agents
Matt Squire presents multi-agent systems as multiple autonomous agents that communicate and coordinate to complete tasks. Each agent can have its own specialist knowledge, context, and tools. He frames this as deliberately applying Conway's law, so the architecture mirrors the specialist roles in an organization. One agent might handle advertising while others handle finance or hiring. The paper discusses communication protocols, orchestration, workload coordination, MCP, and agent-to-agent communication. Matt says these systems face familiar distributed-systems problems, including shared context, task dependencies, coordination, and agreement on ordering.
"RAG is the agent's IO into the world. Memory is the agent's internal state and agentic reasoning is the CPU which decides how it uses these various tools."Rohan Prasad33:00
Who should watch
You are building an LLM application and need a clearer model of how retrieval, memory, tools, and prompt assembly fit together.
Your RAG system needs question decomposition, graph structure, or a better way to select relevant history instead of passing everything into the context window.
You are designing agents that call tools or coordinate with other agents and want concrete examples of the reasoning and distributed-systems problems involved.
