Tool use lets language models retrieve current or proprietary information and take actions outside text generation.
2
Clear tool descriptions, repeated instructions, and tool retrieval can improve when a model selects and uses tools.
3
Structured output parsers can turn model responses into tool calls, but repairs need the original prompt when meaning is missing.
Summary
Harrison Chase explains why language models need tools and how LangChain approaches the practical problems. Tools can provide current events, proprietary data, calculations, code execution, and access to APIs. They can also let a model take actions such as writing to a database. Chase focuses on three problems: getting a model to use a tool in the right situation, stopping it from using tools when ordinary conversation is enough, and parsing its response into a valid tool invocation. His advice includes clear instructions, detailed tool descriptions, reminders at the end of prompts, and retrieving a smaller set of relevant tools when many are available. For structured responses, he describes output parsers and repair parsers. He also explains why a parser that only fixes JSON syntax cannot restore missing meaning. For ChatGPT plugins, LangChain wraps each API endpoint in a separate chain so the agent can interact with complex endpoints through simpler natural-language inputs.
Tools give language models access to information and actions outside text generation
Chase gives two main reasons to connect language models to tools. A tool can retrieve current events, proprietary information, or information from complex data structures. It can also let the model act in the outside world, such as pushing something to a database. Since a language model is roughly text in and text out, tools add capabilities that the model does not have by itself. He lists search engines, calculators, retrieval systems, coding environments, arbitrary functions, and APIs as examples.
The prompt must explain when a tool should be used and what it returns
Chase's short answer is to tell the model when to use a tool, how to use it, and what the tool returns. He says the prompt or system message should name the available tools and describe what they do. A search tool should explicitly say that it is for current events, because otherwise the model may guess incorrectly about when to use it. Detailed descriptions often address cases where a model selects a tool incorrectly.
Repeating instructions can keep older models from losing the tool rules
Chase recommends putting tool instructions at the beginning of the prompt and repeating the important constraints at the end. He has observed that some older models lose track of early instructions by the time they reach the end of a long prompt. His example of the closing reminder is to format the output correctly or use tools only when needed. He describes this short reminder as surprisingly helpful.
Tool retrieval narrows the choice when an application has many tools
Putting every available tool into one prompt becomes impractical when an application has around 100 tools. Chase describes a first retrieval step based on embeddings or semantic similarity. The system retrieves relevant tools, places a smaller group, such as the top five, in the prompt, and asks the language model to choose among those. This reduces the number of tool descriptions the model must consider at once.
A model needs an explicit way to answer without calling an external tool
A conversational bot may have access to a tool without needing to use it for every message. Chase recommends stating this in the system message and repeating it at the end of the instructions. He also describes adding a tool whose purpose is simply to respond to the user. That explicit option can help when general instructions alone do not stop the model from making unnecessary tool calls.
Structured output parsers turn model responses into tool invocations
Chase describes using structured response types such as JSON and TypeScript to make tool calls easier to parse. In LangChain, an output parser can generate format instructions from a schema, insert those instructions into the prompt, and parse the model's response back into the defined object. He shows this with a schema defined in Python and says the JavaScript library has a different schema-definition mechanism.
A repair parser can send malformed output to another language model and ask it to correct JSON decoding errors. Chase points out that syntax repair has limits. If a response omits an argument, the repair model may insert a blank string because it does not know the intended value. A parser that retries with the original prompt has the missing context and can make a more informed correction.
Wrapping each API endpoint in its own chain simplifies complex plugin calls
For ChatGPT plugins and other Open API tools, Chase says LangChain can use prompts and output parsers to communicate JSON and TypeScript parameters. Language models still struggle with complex parameters and function definitions. His approach wraps each endpoint in its own chain, so that chain handles one endpoint and its parameters. An agent routes between these chains, receiving and sending natural-language strings instead of directly managing every complex endpoint definition.
