Production tools need generated, constrained schemas so language models can choose arguments reliably.
2
Exploratory tools should gather information without write access, while operational tools should have narrow privileges and a small set of verifiable outcomes.
3
Teams should evaluate individual tools across multiple language models and limit each agent to the tools it needs.
Summary
Sam Partee explains that production agentic systems depend on the design and testing of individual tools, rather than only on evaluating the agent as a whole. He recommends generating schemas from annotated parameters, restricting difficult argument types, documenting pagination defaults, and assigning authorization scopes to every tool. He divides database tools into exploratory and operational types. Exploratory tools help an agent search and gather context, usually with a human involved, while operational tools perform tightly scoped actions with predictable outcomes. The two types should have different privileges. Partee also describes returning prompts from tools so an agent can recover from errors or refine a search. Tool evaluations should run regularly across several language models. In the question period, he warns against exposing too many tools through one gateway, describes MCP as an incomplete layer, and argues that API specifications often need an agent-optimized layer before they work well for language models.
Tool schemas need to be generated and deliberately constrained
Partee calls generated schemas a bare minimum for production tools. His team generates schemas from annotated non-context parameters and does not allow unannotated tool arguments. He recommends restricting data types because language models have an easier time choosing from a small enum than from an effectively unlimited set of values, such as arbitrary floating-point numbers. Pagination and limits should exist like they do in regular APIs, with their defaults explained inside the parameter annotation. He also says database connection strings should not live in environment files as the production method for handling dependencies.
Every tool should carry its own authorization requirements so the least privilege can be applied at tool level. Partee compares reading, writing, editing, and deleting a Google Drive file, which have different levels of risk. An enterprise deployment will expect those distinctions to be explicit. A tool that acts with a user's credentials or company secrets needs scopes that match the specific action it can perform, rather than inheriting broad access from the surrounding agent.
Teams should evaluate tools separately from agents
Partee says teams often evaluate the agent while rarely evaluating the tools that the agent calls. Arcade maintains an evaluation gauntlet for each tool and runs it across 15 language models every night. That gives the team a CI/CD-style process for detecting changes in tool behavior. He also recommends evaluating tool descriptions whenever they change, since a description mutation can alter how different models respond when calling the tool.
Database tools fall into exploratory and operational types
Exploratory tools gather context through open-ended actions such as discovering databases, collections, schemas, and documents. They can search progressively and work alongside a human as the agent narrows the answer. Operational tools have tightly scoped privileges and a limited set of outcomes. Partee's payment-plan example accepts an enum of plans, which makes the action easier to constrain and verify. He says exploratory tools should be read-only, while operational tools can act when their outcomes are controlled.
Enums and returned prompts make tool calls easier for models
Partee prefers semantic enum values such as yesterday, last week, or last month over separate start-date and end-date arguments. He says models understand those choices more reliably. Tools can also return a prompt along with an error or normal result. In his search example, a tool can tell the model that it searched too narrowly and should retry with a medium scope. He calls this a useful shortcut because the returned instruction directly guides the next model response.
Exploration and action work better as separate stages
Partee's central recommendation is to keep exploratory tools from having write privileges and to make operational tools narrow enough that their outcomes are verifiable. Used together, the two types let an agent first search and understand a situation, then act after it has enough information. He presents this split as especially useful for database systems, where an exploratory tool may inspect sensitive data while an operational tool changes it.
A gateway should expose only selected tools to each agent
Arcade's gateway concept places multiple MCP servers behind one access point while letting a team select individual tools for each agent. Partee says an agent should not use more tools than it has been assigned. In the question period, he advises against putting every available tool into one gateway because too many tools reduce response quality. A server with many integrations can still be narrowed to a small selection for a particular agent.
MCP is a tool protocol, not the complete agent abstraction
Partee says MCP is not all that production systems need. He expects another layer above tools that packages them into more comprehensive actions and carries more human intuition. He does not claim to know exactly what that layer will look like, and he is skeptical of editing Markdown files as the whole approach to skills. He also warns that MCP deployments have security loopholes and need careful review, especially when exposed to the internet.
Raw API specifications need an agent-optimized layer
Partee says converting an OpenAPI specification directly into tools often leaves too many parameters, high-cardinality values, and date formats that models do not handle naturally. His recommendation is to write on top of generated API tools, then improve their descriptions and schemas and evaluate them for language-model use. He calls these wrapped versions agent-optimized tools because they cover corner cases better than an unmodified API conversion.
"If you take anything away from the talk is that if you want to take LLM tools to prod, operational tools should be different than your exploratory tools."13:06
Who should watch
You are building agents that need to read or modify customer, financial, or other sensitive data and need a clear split between discovery and action.
Your team has an MCP or API integration that works in demos but has weak schemas, broad permissions, or no tool-level evaluation.
You are deciding how many tools to expose to an agent and want practical guidance on gateways, authorization scopes, and model-facing parameters.
