# LangChain: Enabling LLMs to Use Tools

Harrison Chase, LangChain | LLMs in Production 2023 | 11:43

Source: https://www.youtube.com/watch?v=XTczX82wzLQ
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/langchain-enabling-llms-to-use-tools
Published: 2023-04-28
Tags: prompt-engineering, structured-outputs, tool-use

## TL;DR
- Tool use lets language models retrieve current or proprietary information and take actions outside text generation.
- Clear tool descriptions, repeated instructions, and tool retrieval can improve when a model selects and uses tools.
- Structured output parsers can turn model responses into tool calls, but repairs need the original prompt when meaning is missing.

## Summary
Harrison Chase explains why language models need tools and how LangChain approaches the practical problems. Tools can provide current events, proprietary data, calculations, code execution, and access to APIs. They can also let a model take actions such as writing to a database. Chase focuses on three problems: getting a model to use a tool in the right situation, stopping it from using tools when ordinary conversation is enough, and parsing its response into a valid tool invocation. His advice includes clear instructions, detailed tool descriptions, reminders at the end of prompts, and retrieving a smaller set of relevant tools when many are available. For structured responses, he describes output parsers and repair parsers. He also explains why a parser that only fixes JSON syntax cannot restore missing meaning. For ChatGPT plugins, LangChain wraps each API endpoint in a separate chain so the agent can interact with complex endpoints through simpler natural-language inputs.

## Key ideas
### Tools give language models access to information and actions outside text generation
[02:20](https://www.youtube.com/watch?v=XTczX82wzLQ&t=140s)
Chase gives two main reasons to connect language models to tools. A tool can retrieve current events, proprietary information, or information from complex data structures. It can also let the model act in the outside world, such as pushing something to a database. Since a language model is roughly text in and text out, tools add capabilities that the model does not have by itself. He lists search engines, calculators, retrieval systems, coding environments, arbitrary functions, and APIs as examples.

### The prompt must explain when a tool should be used and what it returns
[03:51](https://www.youtube.com/watch?v=XTczX82wzLQ&t=231s)
Chase's short answer is to tell the model when to use a tool, how to use it, and what the tool returns. He says the prompt or system message should name the available tools and describe what they do. A search tool should explicitly say that it is for current events, because otherwise the model may guess incorrectly about when to use it. Detailed descriptions often address cases where a model selects a tool incorrectly.

### Repeating instructions can keep older models from losing the tool rules
[06:19](https://www.youtube.com/watch?v=XTczX82wzLQ&t=379s)
Chase recommends putting tool instructions at the beginning of the prompt and repeating the important constraints at the end. He has observed that some older models lose track of early instructions by the time they reach the end of a long prompt. His example of the closing reminder is to format the output correctly or use tools only when needed. He describes this short reminder as surprisingly helpful.

### Tool retrieval narrows the choice when an application has many tools
[06:56](https://www.youtube.com/watch?v=XTczX82wzLQ&t=416s)
Putting every available tool into one prompt becomes impractical when an application has around 100 tools. Chase describes a first retrieval step based on embeddings or semantic similarity. The system retrieves relevant tools, places a smaller group, such as the top five, in the prompt, and asks the language model to choose among those. This reduces the number of tool descriptions the model must consider at once.

### A model needs an explicit way to answer without calling an external tool
[07:29](https://www.youtube.com/watch?v=XTczX82wzLQ&t=449s)
A conversational bot may have access to a tool without needing to use it for every message. Chase recommends stating this in the system message and repeating it at the end of the instructions. He also describes adding a tool whose purpose is simply to respond to the user. That explicit option can help when general instructions alone do not stop the model from making unnecessary tool calls.

### Structured output parsers turn model responses into tool invocations
[08:09](https://www.youtube.com/watch?v=XTczX82wzLQ&t=489s)
Chase describes using structured response types such as JSON and TypeScript to make tool calls easier to parse. In LangChain, an output parser can generate format instructions from a schema, insert those instructions into the prompt, and parse the model's response back into the defined object. He shows this with a schema defined in Python and says the JavaScript library has a different schema-definition mechanism.

### Fixing invalid JSON does not fix missing meaning
[09:19](https://www.youtube.com/watch?v=XTczX82wzLQ&t=559s)
A repair parser can send malformed output to another language model and ask it to correct JSON decoding errors. Chase points out that syntax repair has limits. If a response omits an argument, the repair model may insert a blank string because it does not know the intended value. A parser that retries with the original prompt has the missing context and can make a more informed correction.

### Wrapping each API endpoint in its own chain simplifies complex plugin calls
[10:33](https://www.youtube.com/watch?v=XTczX82wzLQ&t=633s)
For ChatGPT plugins and other Open API tools, Chase says LangChain can use prompts and output parsers to communicate JSON and TypeScript parameters. Language models still struggle with complex parameters and function definitions. His approach wraps each endpoint in its own chain, so that chain handles one endpoint and its parameters. An agent routes between these chains, receiving and sending natural-language strings instead of directly managing every complex endpoint definition.

## Notable quotes
- "Tool usage itself can actually be used to interact with data structures." (02:20)
- "You tell them when to use them, you tell them how to use them, and then you want to tell them what they return as well." (03:51)
- "The answer is kind of like beef up the tool description." (06:00)
- "If you have 100 tools, you probably can't put them all on the prompt and ask the language model to choose between them." (06:56)
- "If we ask another language model to just fix this thing it won't actually know how it should fix it." (09:39)

## Tools & references mentioned
- LangChain
- ChatGPT
- ChatGPT plugins
- AutoGPT
- BabyAGI
- Python
- TypeScript
- JSON
- Pydantic
- Demetrios Brinkmann

## Who should watch
- You are building a language-model application that needs search, retrieval, code execution, APIs, or actions outside text generation.
- Your model chooses tools at the wrong times or ignores tools that should be available.
- You need to parse structured model output and want to understand why syntax repair alone can produce the wrong arguments.

## Related talks

- [DevTools for Language Models: Unlocking the Future of AI-Driven Applications](https://mlopstalks.com/talks/devtools-for-language-models-unlocking-the-future-of-ai-driven-applications) (Diego Oppenheimer, Factory, 29:55)
- [Reasoning Machines](https://mlopstalks.com/talks/reasoning-machines) (Justin Uberti, Fixie.ai & Jon Turow, Madrona Venture Group, 14:11)
- [Impact of LLMs on the Tech Stack and Product Development](https://mlopstalks.com/talks/impact-of-llms-on-the-tech-stack-and-product-development) (Anand Das, Bito, 55:31)
- [Linguistically-informed LLMs Perform Better](https://mlopstalks.com/talks/linguistically-informed-llms-perform-better) (Chris Brousseau, Mastercard, 19:34)
- [The Emerging Toolkit for Reliable, High-quality LLM Applications](https://mlopstalks.com/talks/the-emerging-toolkit-for-reliable-high-quality-llm-applications) (Matei Zaharia, Databricks, 31:01)