# Deploying Executable Agent Workflows

Gal Peretz, Carbyne | MLOps Community | 18:50

Source: https://www.youtube.com/watch?v=2N-mnKLE0Lk
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/deploying-executable-agent-workflows
Published: 2025-08-25
Tags: guardrails, tool-use, workflows

## TL;DR
- CodeAct lets an LLM write and execute Python for tool use instead of emitting JSON tool calls.
- CodeAct handles branching, loops, nested data, composition, and on-the-fly aggregation more naturally than fixed tool schemas.
- JSON tool calling remains a better fit for simple, low-latency, plug-and-play tasks, while CodeAct needs a controlled execution environment.

## Summary
Gal Peretz compares traditional JSON tool calling with CodeAct, where an LLM writes Python and runs it in a sandbox. He argues that JSON schemas become awkward for large branches, nested objects, function composition, and tasks that require aggregation. For example, asking an agent to deactivate every active account can fail when the user list grows because the model must keep and process too much data in its context. CodeAct lets the model use ordinary programming structures, including loops, conditionals, pandas, and libraries that were not explicitly defined as tools. Peretz stresses that code execution needs limits. Teams should define what the environment can access, inspect generated code with abstract syntax trees and linters, and block forbidden functions before execution. He presents CodeAct as a useful choice for complex workflows when a team is willing to customize its environment. JSON remains suitable for simple tasks and low-latency API calls.

## Key ideas
### JSON tool calling makes the executor carry out the model's intent
[01:53](https://www.youtube.com/watch?v=2N-mnKLE0Lk&t=113s)
In the traditional pattern, the LLM receives a user task and tool schemas, then emits a JSON object containing a tool name and parameters. The application executes that function and sends the result back to the model. The model may emit several tool intents, which the application can execute in parallel. The JSON describes what the model wants done, while the application remains responsible for calling the actual function and changing the environment.

### Large branches can overwhelm a model's context
[04:13](https://www.youtube.com/watch?v=2N-mnKLE0Lk&t=253s)
Peretz uses a task that gets all users and deactivates every active account. With five users, the model can request the deactivation calls one by one. With 10,000 or 100,000 users, the full list and the repeated actions create a context problem. He says the model's attention cannot reliably handle that amount of information, so it may deactivate some users and miss many others.

### Fixed JSON schemas describe inputs more easily than workflows
[08:48](https://www.youtube.com/watch?v=2N-mnKLE0Lk&t=528s)
Peretz says JSON tool schemas are tied to fine-tuned output formats and are difficult to use for nested objects. A company can contain departments, departments can contain employees, and employees can contain addresses. Representing those types in a schema creates a large structure that competes with the task, conversation history, and other context. The model also does not know a function's output shape in the same way it knows the declared inputs, which makes it harder to connect one function's result to another function's input.

### CodeAct gives the model ordinary programming control
[10:35](https://www.youtube.com/watch?v=2N-mnKLE0Lk&t=635s)
CodeAct changes the action format from JSON to Python. The model plans the task, writes code, and runs it in a sandboxed environment. It can inspect errors and printed output, revise the code, and execute it again. Peretz presents this as a way to express loops, conditionals, and multi-step workflows directly instead of asking the model to emit many separate tool calls.

### Code can compose data and use libraries during execution
[12:26](https://www.youtube.com/watch?v=2N-mnKLE0Lk&t=746s)
Functions can be represented as normal code definitions rather than nested metadata objects. The generated program can use loops and conditional logic, call functions, and pass one result into another. Peretz also says the model can use tools such as NumPy and pandas for work that was not defined as a separate function, including aggregating data during execution.

### The execution environment must limit CodeAct's freedom
[13:38](https://www.youtube.com/watch?v=2N-mnKLE0Lk&t=818s)
Giving the model the ability to write code creates too much freedom unless the environment sets boundaries. Peretz describes system-level restrictions such as whether the model can access the internet or read and write files. The goal is to reduce the search space so the generated program can converge, while preserving enough flexibility to express the workflow.

### CodeAct can make a smaller model competitive on workflow tasks
[14:20](https://www.youtube.com/watch?v=2N-mnKLE0Lk&t=860s)
Peretz shows results where a smaller model using CodeAct is competitive with a larger model using JSON schemas on workflow-based tool-calling datasets. He presents this as a reason to consider executable workflows when the task needs branching or dynamic behavior. The comparison is specific to the datasets and setting he describes.

### CodeAct needs code inspection before execution
[17:02](https://www.youtube.com/watch?v=2N-mnKLE0Lk&t=1022s)
In the discussion, Peretz recommends abstract syntax trees and linters to inspect generated code before it runs. An abstract syntax tree can expose typing errors and forbidden functions. The system can then reject or revise the code. He recommends pairing a prompt that restricts the model with technical checks that enforce those restrictions.

## Notable quotes
- "Instead of output JSON let's write code." (10:35)
- "LLM is great in writing code so let's do that." (10:35)
- "You can use, for example, a tool like abstract syntax tree and linters to understand before you even execute the code if there were a typing error or stuff like that." (17:02)
- "When do you want to use JSON? When you want the plug-and-play solution, when you don't have any complex task, and when you want a low latency." (15:05)

## Tools & references mentioned
- Carbyne
- OpenAI
- Anthropic
- Model Context Protocol
- MCP
- CodeAct
- Python
- pandas
- NumPy
- JSON
- OpenAI schema
- LangTalks
- trustcall
- GPT-4o
- abstract syntax tree

## Who should watch
- You are building an agent that needs loops, branching, or data processing across many records, and fixed JSON tool calls are becoming difficult to manage.
- Your team is considering executable code for agents and needs practical controls before generated programs run.
- You need a simple tool-calling agent with low latency and do not want to maintain a sandbox or code inspection layer.

## Related talks

- [AI Agents: The Future of Productivity, or Just a Fad?](https://mlopstalks.com/talks/ai-agents-the-future-of-productivity-or-just-a-fad) (Sam Partee, Arcade AI, 35:18)
- [How Agentic Workflows Will Change Everything](https://mlopstalks.com/talks/how-agentic-workflows-will-change-everything) (Raj Rikhy, Microsoft, 49:13)
- [Sub-Agent Architectures: What You Can Leverage](https://mlopstalks.com/talks/sub-agent-architectures-what-you-can-leverage) (Sidd Seethepalli, Vellum, 14:48)
- [How to Make a Coding Agent a General Purpose Agent](https://mlopstalks.com/talks/how-to-make-a-coding-agent-a-general-purpose-agent) (Harrison Chase, LangChain & Sam Partee, Arcade, 36:55)
- [The Agent Landscape - Lessons Learned Putting Agents Into Production](https://mlopstalks.com/talks/the-agent-landscape-lessons-learned-putting-agents-into-production) (Paul van der Boor & Floris Fok, Prosus Group, 1:08:41)