AI agents create bursty, stateful workloads whose resource use depends on reasoning depth and task complexity, rather than predictable request volume.
2
Agent infrastructure needs to separate persistent context from short-lived compute, provision resources around interactions, and move execution between devices, the edge, and the cloud.
3
Existing distributed-systems practices still matter, but identity, memory boundaries, audit trails, governance, and cost attribution need new designs for machine-speed decisions.
Summary
Diego Oppenheimer argues that AI agents are exposing a mismatch between traditional infrastructure and software that can perceive, reason, act, and remember. Conventional systems assume predictable request-response flows, known resource use, and mostly stateless execution. Agents can instead enter reasoning loops, fan out into many sub-agents, move large amounts of context, and consume resources according to task complexity. Oppenheimer describes tensions between ephemeral compute and persistent memory, between serverless efficiency and conversational latency, and between resource uptime pricing and the cost of reasoning. His thought experiment, which he calls "the agenda," proposes separating state from compute, provisioning around interaction complexity, making context persistent and searchable, allowing fluid movement across device, edge, and cloud, and allocating resources according to workflow value. He is careful not to present this as a finished blueprint. The unanswered issues include agent identity, dynamic permissions, compliance, shared-memory security, auditability, and deciding which department pays for machine reasoning.
Agent workloads can break conventional scaling assumptions within minutes
Oppenheimer asks what happens when an autonomous agent is released into production. In his example, it can create 12,000 forward passes in under two minutes and fork 15 sibling agents with separate credential scopes. Model queues spike, schedulers scale rapidly, caches are consumed, and the system may stream 10,000 tokens into a user interface. At the memory layer, the agent pulls 2 gigabytes of embeddings and documents, emits 8-megabyte reasoning log cycles, and triggers cold storage reads. Logs may be truncated, making debugging useless. Credentials can also fan out across tenants. His point is that traditional scale-up and scale-down playbooks assume predictable CRUD workloads, while agents can change the shape of the system very quickly.
Agents differ from CRUD software because they perceive, act, reason, and remember
Traditional applications usually follow a linear input-output path with mostly predictable resource consumption and explicit persistence. Oppenheimer says agents behave differently in four ways. They can continuously observe their environment, act without a human trigger, apply inference to incoming information, and retain large amounts of context across interactions. Agent workflows therefore include cyclical reasoning and feedback. Their resource use depends on the complexity of the task rather than simply on elapsed time or request count. They also require persistent memory to carry context through the workflow. Oppenheimer describes this pattern as nearly orthogonal to the systems built around CRUD operations, because the old assumptions do not simply patch onto it.
Persistent agent memory conflicts with the desire for disposable compute
Agent infrastructure has to keep context while using compute only when it is needed. Oppenheimer compares this with a serverless function that starts, does work, and disappears. That model is attractive for compute because it avoids keeping unused capacity alive, but an agent may need the full history of previous conversations, goals, and reasoning chains. The result is a tension between ephemeral processing and persistent state. Oppenheimer also says cost is beginning to depend on the cognitive depth of a workload rather than only on throughput or response time. Existing cost models therefore have trouble predicting how much a task will consume. He identifies persistent, searchable, versioned context as one of the hardest parts of fitting agent workloads into current infrastructure.
Agent collaboration creates dense communication patterns that isolation-oriented systems do not expect
Single-agent systems can use relatively clear resource boundaries and one main conversation thread. Multi-agent systems introduce fan-out and fan-in at machine speed, shared context, and coordinated decisions. Oppenheimer calls this a move from compute isolation toward compute intimacy. Agents need to exchange substantial memory, not just send ordinary HTTP requests. That communication density can clog orchestration and networking layers. The security model also becomes harder because systems designed to isolate components must now allow controlled cooperation between them. The challenge is not only starting more processes. It is managing the context, permissions, and coordination that pass between agents while their workflows expand.
The economics of infrastructure may shift from paying for uptime to paying for thought
Traditional capacity planning focuses on peak load and improving utilization of reserved resources. Oppenheimer says agent workloads introduce a different cost question: how many reasoning cycles are needed to produce value? The relevant measure becomes reasoning depth and the capability produced per dollar. This changes how teams might optimize systems because a workload that looks small by request count can still consume substantial resources if the agent performs extensive reasoning or tool use. It also complicates cost attribution. Later, Oppenheimer asks which department should pay for an agent's thinking and where that cost should be recorded. He treats this economic shift as a design problem for infrastructure, governance, and organizational budgeting.
Serverless efficiency conflicts with the latency and context needs of conversations
Oppenheimer describes a cold-start paradox. Serverless execution can be efficient because resources are created only when needed, but starting a function takes time and the agent may need to load a large context before it can respond. Human conversation expects a quick response, while meaningful agent interactions may require context ranging from hundreds of megabytes to a gigabyte. Keeping everything warm improves responsiveness but burns money across the system. This creates a direct tradeoff between efficiency and conversational continuity. The problem is wider than model inference because an agent may also call tools and perform other processing whose shape is difficult to predict from a simple API request.
Oppenheimer's proposed agenda separates state, interaction demand, and execution location
Oppenheimer proposes a first-principles thought experiment that he calls "the agenda." Its first principle is to decouple state from resources, allowing compute, memory, and storage to operate as pools that agents consume as needed. The second is interaction-driven provisioning, where resources scale with cognitive demand and systems recognize usage patterns before allocating capacity. Context should persist across execution boundaries, with versioning, search, rollback, and continuity between environments. Compute should also move between device, edge, and cloud based on privacy, latency, compliance, coordination, and computational needs. Finally, Oppenheimer suggests market mechanisms in which agents bid for resources according to task priority or potential value. He presents these as a lens rather than a finished architecture.
Identity, governance, and memory security remain unresolved constraints
Oppenheimer lists several issues that the infrastructure proposal does not solve. Systems need to know which agent is acting, whose authority it represents, how access is granted and revoked, and how machine-speed decisions are audited. Regulated data adds compliance requirements. Dynamic permissions also need an audit history that records what was created and later revoked. In the discussion, he identifies context engineering and memory management as the first difficult framework problem, followed by authentication and authorization. Separating memory from compute may help technically, but shared memory boundaries are difficult to secure. Agents need to hand off context, yet communal physical memory could create serious isolation problems. He does not claim to have a final answer.
"The future of infrastructure is not about fast servers. It's about systems that support software that reason."Diego Oppenheimer21:42
Who should watch
You are designing production systems around agents and need to understand why request-based scaling may fail when agents fan out or spend longer reasoning.
Your team is deciding whether current serverless, container, and orchestration patterns can handle persistent context and machine-speed collaboration.
You need to assign ownership, permissions, audit trails, or budgets for agent workflows and want a clear account of the unresolved design questions.
