# Write Reliable Software with Temporal

Johann Schleier-Smith, Temporal | MLOps Podcast | Episode 364 | 1:00:37
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=umdiwQbkwlY
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/write-reliable-software-with-temporal
Published: 2026-03-17
Tags: agents, reliability, workflows

## TL;DR
- Durable execution lets regular programs recover from failures and continue to completion without developers writing recovery logic throughout the application.
- Temporal separates workflow control flow from activities that perform I/O, recording activity results so a workflow can replay deterministically and resume after failures.
- Long-running agents benefit from durable state, timers, human interactions, signals, updates, and queries, while workers run the code inside the user's trusted environment.

## Summary
Johann Schleier-Smith describes durable execution as a programming model that makes software crash-proof. Temporal stores workflow state and activity results so a program can replay and continue when servers fail, APIs go down, compute disappears, or a region becomes unavailable. The application is written as ordinary code, with deterministic workflows controlling execution and activities handling I/O such as tool calls, API requests, and LLM calls. Johann contrasts this with hand-written retries, event-driven systems, and coarse checkpoints. He explains why the model fits long-running agents, including timers, human approval steps, interruptions, and stateful conversations. Temporal has a server, workers, and clients, and workers execute application code in the user's environment. The discussion also covers Airflow-like pipelines, hyperparameter optimization, serverless computing, memory isolation, streaming, deployment versioning, and Temporal integrations with agent frameworks. Johann recommends learning the programming model, separating workflows from activities, and using the Temporal UI and coding-agent skill while building.

## Key ideas
### Durable execution moves failure handling out of business logic
[00:55](https://www.youtube.com/watch?v=umdiwQbkwlY&t=55s)
Johann defines durable execution as software continuing from its start to its end despite failures. Cloud systems commonly have flaky servers, overloaded services, unavailable APIs, and rate limits. Temporal records the state needed to recover from those conditions, so the programmer does not have to add recovery code everywhere. Reliability and application logic can be treated as separate concerns. Temporal is open source under the MIT license, can use Cassandra or Postgres for persistence, and also has Temporal Cloud. Johann's point is about making reliable software easier to write, since retries and distributed-systems logic are possible to implement by hand but become messy as systems grow.

### Workflows are deterministic control flow and activities perform I/O
[03:06](https://www.youtube.com/watch?v=umdiwQbkwlY&t=186s)
Temporal gives developers programming-language-level building blocks instead of making servers the main abstraction. Workflows and activities are functions, but they have different rules. Activities can contain arbitrary code and perform I/O. Workflow code must behave the same way when replayed with the same inputs, which is deterministic execution. An agent loop can therefore be workflow code when it repeatedly decides to call an LLM or a tool. The LLM call and tool call are activities. When an activity returns, the Temporal SDK saves the result to the Temporal server, allowing the workflow to replay its control flow without repeating already-recorded work.

### Temporal records fine-grained progress instead of relying on coarse checkpoints
[06:43](https://www.youtube.com/watch?v=umdiwQbkwlY&t=403s)
Johann contrasts durable execution with systems that repair broken transactions through database scripts, event-driven designs with queues and dead-letter handling, and checkpoint-based frameworks. Checkpoints save the entire program state at selected moments, which can be awkward around concurrent tool calls or human interactions. Temporal records incremental state changes as the program interacts with tools and LLMs. Long-running workflows can still use 'continue as new' to start from a full snapshot when their history becomes too large. Johann says this model is especially useful for branching programs such as hyperparameter optimization, where clean checkpoints across concurrent work are difficult to create.

### Durable execution fits long-running and interactive agents
[20:02](https://www.youtube.com/watch?v=umdiwQbkwlY&t=1202s)
Johann explains that an LLM's output can be non-deterministic while the agent loop remains durable. The LLM decides what happens next, such as calling a tool, and the result is captured as activity state. Temporal is useful when an agent runs on timers, sleeps between checks, or waits for an outside interaction. A workflow can wake every half hour or at a scheduled time without holding compute resources while it sleeps. It can also pause for an e-signature, send a reminder after a time limit, or respond to a retraction. Signals, updates, and queries let external systems interact with a running workflow and its state.

### Temporal separates agent state from the agent framework
[33:01](https://www.youtube.com/watch?v=umdiwQbkwlY&t=1981s)
Johann describes Temporal as protecting the program's state and execution path, while an agent framework handles agents, guardrails, coding tools, and file systems. He names integrations with the OpenAI agents SDK, Pydantic AI, and AI SDK by Vercel, along with observability integrations for BrainTrust and LangFuse. He also says organizations including OpenAI's Codex on the web, Replit, and Lovable use Temporal. For many agents that call APIs, LLMs, and databases, Temporal can hold the state without a file system. Code-generating or computer-use agents may still need a file system. Cross-session memory and context sharing remain areas the ecosystem is exploring.

### Workers execute private application data while the Temporal server tracks state
[46:10](https://www.youtube.com/watch?v=umdiwQbkwlY&t=2770s)
The basic architecture has a Temporal server, a worker, and a client. The client starts a workflow, the server tracks durable state, and the worker executes the code. In production, the client can encrypt application data before sending it to the Temporal server, leaving the encryption keys with the customer. Because the server cannot compute on that encrypted data, the worker runs inside the customer's VPC or trusted environment and decrypts it there. Workers can run on a server, in containers, or on Kubernetes. Temporal can provide signals for autoscaling, while the customer controls how workers are deployed.

### Temporal applies one programming model to pipelines, agents, and distributed jobs
[18:58](https://www.youtube.com/watch?v=umdiwQbkwlY&t=1138s)
Johann places Temporal as a general programming model next to more domain-specific tools such as Airflow. Teams can build Airflow-like frameworks or DAG executors on Temporal, but ordinary Python code can be more natural for processing legal documents, media files, and vector indexing steps. He also gives hyperparameter optimization as an example. A Python driver can run overnight or for a week while Temporal stores which iteration and rollout the program reached. If compute disappears, another resource can resume from the saved state. The model treats compute as replaceable and keeps the program state durable.

### The Temporal UI and coding-agent skill shorten development and repair
[37:52](https://www.youtube.com/watch?v=umdiwQbkwlY&t=2272s)
Johann recommends starting with the Temporal CLI locally or with Temporal Cloud, then using an SDK and a small hello-world project. He suggests asking a coding agent to teach Temporal or build the first example, using Temporal's skill to handle edge cases. Developers should understand the worker, client, server, workflows, and activities, then put operations that perform I/O into activities. The Temporal UI displays workflow execution, activities, arguments, return values, retries, and failures. During local development, a developer can fix a bug, save the file, restart the server, and let the workflow finish from where it stopped. The same model can help recover a long-running job without writing a separate continuation program.

## Notable quotes
- Johann Schleier-Smith: "So, durable execution basically just means that your software does what it's supposed to do." (00:21)
- Johann Schleier-Smith: "What makes durable execution work is that the workflow code is a restricted program model." (03:51)
- Johann Schleier-Smith: "What matters is the state and the state of that program which iteration is it on, on which rollout and all that is going to be saved in the server." (22:33)
- Johann Schleier-Smith: "At the end of the day, it's just like the thing does what it's supposed to do. That's all the serverless is, that I don't need to worry about server nonsense." (51:29)

## Tools & references mentioned
- Temporal
- Temporal Cloud
- Cassandra
- Postgres
- Airflow
- Argo Workflows
- Kubernetes
- OpenAI agents SDK
- Pydantic AI
- AI SDK by Vercel
- BrainTrust
- LangFuse
- OpenAI's Codex
- Replit
- Lovable
- S3
- Redis streams
- Crystal DBA

## Who should watch
- You are building an agent that needs to run for hours, days, or indefinitely and want it to recover without custom checkpoint and retry code.
- Your team is comparing Temporal with Airflow-like pipelines, event-driven workflows, or agent frameworks and needs to understand where each model fits.
- You want application code to run in your own environment while a separate service tracks encrypted state, retries, timers, and human interactions.

## Related talks

- [Loop Engineering](https://mlopstalks.com/talks/loop-engineering) (David DeStefano & Sam Christensen, EvolutionIQ & Valdimar Eggertsson, Snjallgögn (Smart Data inc.) & Sparsh Jain, CentralAgent AI, 56:10)
- [Building AI that Doesn't Break](https://mlopstalks.com/talks/building-ai-that-doesnt-break) (Elliot Gunton, Pipekit & Qian Li, DBOS, Inc. & Alan Nichol, Rasa, 1:01:03)
- [Building AI Agents That Survive Production](https://mlopstalks.com/talks/building-ai-agents-that-survive-production) (Haytham Abuelfutuh, Union.ai, 31:59)
- [Decomposing the Agent Orchestration System: Lessons Learned](https://mlopstalks.com/talks/decomposing-the-agent-orchestration-system-lessons-learned) (Niels Bantilan, Union.ai, 30:13)
- [How to Build Execution Layers That Don't Burn Out](https://mlopstalks.com/talks/how-to-build-execution-layers-that-dont-burn-out) (Tanmay Tiwari, 10:07)