AI agents need isolated computers when they can use shells, write code, and execute arbitrary tools.
2
Agent evaluation should measure complete executions in known environments, rather than individual model calls.
3
Coding agents work well partly because Git, pull requests, CI, and deterministic checks provide a workflow for proposing, reviewing, and reverting changes.
Summary
Jonathan Wall argues that the main infrastructure problem for AI agents is giving them a safe computer to use. A sandbox isolates an agent's files, processes, network access, and resource usage while still giving it the tools and context needed to work. This matters when agents can run Bash, compile code, download files, or modify repositories. Wall describes how sandboxes can be configured with images, code mounts, objects, agents, and scoped MCP credentials. He also explains why agent evaluation needs full environments and scoring functions that inspect the resulting state. The discussion then moves to harnesses, which provide context management, planning, and tool use around capable agents. Wall's broader point is that coding has an advantage because Git and pull requests support staging, review, CI, and rollback. Other industries will need similar workflows before agents can operate safely in production.
Jonathan Wall defines a sandbox as an isolated environment where an agent can operate without directly threatening the host system. It includes the tools and context needed for the task, and in practice gives the agent something like its own computer. Runloop.ai lets users choose memory, compute, disk, and the number of environments to run in parallel. Default images include common tools such as Python and Node. Users can then create a blueprint, which Wall describes as a Dockerfile plus extra configuration for objects, agents, code mounts, and Docker-in-Docker. Developers can assemble the environment dynamically first, then save the working recipe as a blueprint.
Isolation makes powerful agent capabilities safer to use
Demetrios Brinkmann asks why an agent should run in a sandbox instead of behind an ordinary API. Wall says the answer changes when the agent needs shell access, can author and execute code, or can use dangerous tools. Agents have unpredictable workloads, and a mistake could kill the server or remove its root filesystem. A sandbox limits the blast radius while allowing the agent to clone repositories, download files, inspect the filesystem, write plans, and store intermediate work. Wall describes this as two sides of the same decision: the computer gives the agent more capability, while isolation makes that capability acceptable.
Sandbox resources and credentials can be scoped to each agent
Wall describes several ways to give an agent safe access to data. A copy of selected database tables can be placed in object storage and added to a sandbox, or an MCP server can expose an external database through scoped, read-only credentials. Resource limits also contain runaway work. A sandbox can use only the CPU, memory, and disk assigned to it, and Runloop.ai adds alerts when usage gets high. At scale, customers may launch thousands of sandboxes at once. Wall also connects each sandbox to an identity, so an agent ID, sandbox ID, and specific MCP credential can be associated with one execution.
End-to-end benchmarks are better than judging isolated model calls
Wall says Runloop.ai provides observability, debugging, auditing, and optional trajectory collection for agent runs. Its benchmarks start from sandboxes in a known state, run an agent against a task, and apply scoring functions to the result. This fits agents because a task may involve dozens of model calls and several tool calls. In his dependency-update example, evaluation can inspect the changed dependency file and check whether the repository builds, since a version can look correct while not existing. Benchmarks let teams compare models and track whether an agent is improving over time. Wall also describes agents that judge other agents' work and send failed work back for another attempt.
Shared state could support recovery and multi-agent workflows
Runloop.ai sandboxes currently start without network access, though users can open tunnels for HTTP, WebSockets, and communication between agents. Wall says the company is exploring shared filesystems, volume mounts, and saved output directories. These could preserve an agent's notes or intermediate work after its sandbox disappears, or let separate agents pass work through a shared location. He gives a possible pipeline in which agent A writes a result, agent B reads and extends it, and agent C continues the process. He presents these as prototypes and open product questions rather than finished features.
Harnesses provide the agent machinery around custom skills
Wall agrees with Harrison Chase's distinction between frameworks and harnesses. A harness such as the Claude Agent SDK or LangChain Deep Agents already handles context, planning, tool calls, and conversation compaction. A user can add skills or tools without implementing those mechanics from scratch. Wall's example is an agent that reads two PDFs and creates a database entry. The developer supplies the PDF-reading skill and database schema, while the harness manages context, calls tools, and updates the plan. He expects this batteries-included approach to make simpler agents easier to build, while specialised startups may still implement their own systems.
Coding agents benefit from composable tools and deterministic checks
The conversation turns to Bash and verification. Wall says decades of Unix tooling make Bash a powerful, composable tool for agents. An agent can combine commands such as grep, pipes, sed, and git instead of needing a custom tool for every file operation. He connects this to verification: software can be compiled, linted, formatted, tested, or run to check its output. Those checks give an agent fast feedback about whether its work functions. Wall contrasts this with legal documents, where there is no equivalent of a compiler that can immediately verify correctness.
Other industries need a GitHub-like workflow for agents
Wall says coding has an unusual advantage because Git and pull requests already support proposed changes, human review, CI, approval, and reversion. He asks what the equivalent of a pull request would be for an insurance claim, a marketing task, or a change in Salesforce or Workday. Even if sandboxes, MCP access, and harnesses become easy to use, agents still need a way to stage work, have a person review it, accept or reject it, and send failures back for correction. He connects this workflow to training data because pull requests contain both the motivation for a change and the human-approved solution. Deterministic scoring and fast feedback help coding agents learn through self-play, while other fields may need new verification functions.
"When you're going to give them bash or shell access or access to dangerous tools, if that's running next to your computer, there's nothing to stop your agent from killing the server it's running on or rm-rfing the root file system where your server is running."Jonathan Wall09:09
Who should watch
You are building agents that need shell access, code execution, repository access, or internet access and need to limit what a mistake can damage.
Your team is evaluating agents by looking at individual model calls, while the real task depends on many tool calls and the state of a filesystem or database.
You work outside software engineering and need a way for agents to stage, review, verify, approve, and revert changes before they reach production systems.
