# AI Agents: The Future of Productivity, or Just a Fad?

Sam Partee, Arcade AI | MLOps Community | 35:18
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=mmW_1ok3JW0
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/ai-agents-the-future-of-productivity-or-just-a-fad
Published: 2024-12-02
Tags: agents, deployment, evals, tool-use

## TL;DR
- Sam Partee defines an agent as a language-model system that can call and execute tools, while retrieval alone is only a more capable search system.
- Tool execution should run in a dedicated, scalable runtime instead of inside the orchestration framework that manages the language model.
- Agents need user-authorized actions, software-style tool packaging, testing, versioning, monitoring, and runtime management before they can perform useful work.

## Summary
Sam Partee explains what tool calling does and why he thinks it is the part that makes an AI system agentic. A language model predicts tool parameters, but the developer still has to execute the function and send its results back to the model. He argues that tool execution needs its own runtime because orchestration frameworks often make tools into concurrent HTTP requests inside a Python process, which limits computational work and scaling. Tools should be packaged, versioned, tested, monitored, and deployed like software. Partee also says agents need authorization to act as the user, since API-key access to public services limits what they can do. He then presents Arcade AI, which provides a tool SDK, authentication and authorization APIs, a distributed tools API, worker runtimes, and an OpenAI-compatible language-model API. Arcade is intended to make tool development, execution, authorization, scaling, and evaluation easier across different models and agent frameworks.

## Key ideas
### An agent needs tools that can take actions
[04:50](https://www.youtube.com/watch?v=mmW_1ok3JW0&t=290s)
Partee draws a line between retrieval and tool calling. A retrieval system adds information to a model's input, while an agent can select a function and act on its result. He says, "an agent is not an agent unless it can call tools." In his view, a system with a good retrieval pipeline is still "Fancy Search" if it cannot perform actions. Tool calling is the model's prediction of parameters for a function. It does not mean that the function has already run, even when an API describes the feature as parallel tool calling.

### Tool calling predicts parameters, while developers still execute the function
[05:23](https://www.youtube.com/watch?v=mmW_1ok3JW0&t=323s)
A tool minimally has typed inputs, possible typed outputs, a description of what the function does and when it should be used, plus a way to execute it. The client sends a prompt and these definitions to a language model. The model returns a selected tool and predicted parameters, usually as JSON. The application then runs the function and sends the result back in another model request. Partee stresses that execution belongs to the developer or an orchestration system. Multiple predicted tool calls can run concurrently only when they have no shared state or dependency.

### Tool execution needs a runtime separate from model orchestration
[10:55](https://www.youtube.com/watch?v=mmW_1ok3JW0&t=655s)
Partee argues that frameworks such as LangChain, LlamaIndex, and CrewAI should coordinate model work without owning every tool's execution environment. He points to Python's GIL as one reason. Computational work such as BM25 search, website scraping, or text processing can block the process, while collocating orchestration and tools also mixes their resource needs. A tool might need a GPU on Modal, while the agent itself does not. Separating the two lets teams scale a busy tool without scaling the whole agent and gives each tool a runtime suited to its work.

### Tools should be managed like production software
[13:05](https://www.youtube.com/watch?v=mmW_1ok3JW0&t=785s)
Partee says tools are currently often buried as subpackages inside a larger repository. He wants them packaged, versioned, tested, and deployed with pinned dependencies, as ordinary software would be. Agents also need monitoring and scalable runtimes. He mentions agent operations products such as Logfire and LangSmith, but says the ecosystem still needs stronger systems for managing tools. His recommendation is to treat agent development more like a software project than a Jupyter notebook experiment, because better processes should make agents more stable and improve their performance.

### User authorization determines whether agents can do useful work
[14:38](https://www.youtube.com/watch?v=mmW_1ok3JW0&t=878s)
Most tools, in Partee's example, make an HTTPS request to a service protected by an API key and return text. That model works for search and crawling, but it does not let an agent act as a user across the services that matter in daily work. Partee says agents need to act on behalf of the user without handing over broad access to an entire account. He describes authorization as one of the main limits on useful agent actions. If an agent cannot use services while properly authorized as the user, many practical tasks remain out of reach.

### Tool orchestration must select and run only the tools an agent needs
[17:17](https://www.youtube.com/watch?v=mmW_1ok3JW0&t=1037s)
Partee places tool orchestration above ordinary agent orchestration in his proposed hierarchy. An agent with hundreds or thousands of tools could have much greater capability, but putting all of those definitions into one context window can overwhelm a model. The system therefore needs to decide which tools to include for a given context. It also needs to run tools in different environments and handle failures and retries. He says smaller models can begin to fall off at around 20 tools, which makes grouping and managing toolkits an operational problem.

### Arcade AI combines tool packaging, authorization, execution, and model access
[19:47](https://www.youtube.com/watch?v=mmW_1ok3JW0&t=1187s)
Arcade AI is presented as a platform for the problems Partee describes. Its tool SDK uses a CLI to create a Python package and requires parameter annotations. Its authentication and authorization API associates provider scopes with a user's unique ID and can keep token storage inside the user's VPC. The tools API manages, enables, disables, and executes tools, while actors provide worker runtimes for those tools. An OpenAI-compatible language-model API can inject and handle tools through a single request. Partee also describes pre-built integrations, including email actions and services such as Gmail, Outlook, Slack, Salesforce, and Atlassian.

### Tool calling needs continuous evaluation and model-change testing
[23:13](https://www.youtube.com/watch?v=mmW_1ok3JW0&t=1393s)
Arcade's SDK includes an evaluation framework built for tool calling. It uses critics and rubrics to test how changes affect a tool or an agent. Partee wants this to run in a CI/CD pipeline whenever a new model, prompt, or package changes, so teams can see whether production tool-calling behavior has shifted. In the question period, he says Arcade uses fake accounts and fake provider accounts for nightly tests. Community packages and Arcade packages have different testing treatment, with Arcade packages tested through that process.

## Notable quotes
- Sam Partee: "I believe an agent is not an agent unless it can call tools otherwise it's just Fancy Search and it's trading Precision for generality and user experience." (04:50)
- Sam Partee: "When I say tool calling what I mean is prediction of parameters not execution." (05:43)
- Sam Partee: "Tools are code they need to be packaged like code versioned like code tested like code." (13:05)
- Sam Partee: "Agents should be able to act as you and this is really hard." (14:18)
- Sam Partee: "If agents can take actions on behalf of you the world of the things that they can do for us opens up drastically." (19:20)

## Tools & references mentioned
- Arcade AI
- Redis
- LangChain
- LlamaIndex
- CrewAI
- Modal
- Logfire
- LangSmith
- BM25
- Google Search
- SerpApi
- Crawl
- Gmail
- Outlook
- Slack
- Salesforce
- Atlassian
- Twilio
- Notion
- OpenAI
- Python
- ECS
- PostgreSQL
- Superbase
- Pest

## Who should watch
- You are building an agent that needs to call APIs or perform actions for users, and you need a clearer distinction between model orchestration and tool execution.
- Your tools run in the same process as the agent and you are running into scaling, computational, dependency, or resource-isolation problems.
- You need authorization, evaluation, and CI/CD practices for tools rather than relying on manual testing or vibe checking.

## Related talks

- [Sub-Agent Architectures: What You Can Leverage](https://mlopstalks.com/talks/sub-agent-architectures-what-you-can-leverage) (Sidd Seethepalli, Vellum, 14:48)
- [The Agent Landscape - Lessons Learned Putting Agents Into Production](https://mlopstalks.com/talks/the-agent-landscape-lessons-learned-putting-agents-into-production) (Paul van der Boor & Floris Fok, Prosus Group, 1:08:41)
- [Web Agents: The Cutting Edge of AI is Here?](https://mlopstalks.com/talks/web-agents-the-cutting-edge-of-ai-is-here) (Paul van der Boor & Chiara Caratelli, Prosus Group, 45:53)
- [AI Agent Development Tradeoffs You NEED to Know](https://mlopstalks.com/talks/ai-agent-development-tradeoffs-you-need-to-know) (Sherwood Callaway, 11X, 57:07)
- [How Agentic Workflows Will Change Everything](https://mlopstalks.com/talks/how-agentic-workflows-will-change-everything) (Raj Rikhy, Microsoft, 49:13)