# Ship Agents: A Virtual Conference Track 2

Adam Boaz Becker & Sarmad Absil, Trial Cyber & Divia Mahajan, Amazon Alexa | Ship Agents 2026 | 1:38:26

Source: https://www.youtube.com/watch?v=nKDKhbPcFcs
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/ship-agents-a-virtual-conference-track-2
Published: 2026-04-10
Tags: agents, inference, observability, security

## TL;DR
- Manufacturing agents can connect IoT alerts, ERP data, equipment manuals, inventory, work orders, and shift logs so engineers can act on failures with better context.
- Production agents often fail silently because successful HTTP responses and completed tool calls do not prove that the agent understood the task or changed the real world correctly.
- Snapshotting initialized GPU and model state can reduce cold starts for supported inference workloads to under one second, while avoiding the cost of keeping every GPU warm.

## Summary
Track 2 covers four production problems for agents. Sarmad Absil presents a manufacturing system that turns IoT anomalies into shared Microsoft Teams alerts, then uses specialized agents to retrieve maintenance history, inventory, manuals, and work-order data. The system keeps humans responsible for physical repairs. Kamal presents a security workflow that moves threat modeling and software supply-chain checks into the developer environment, with security skills evolving through an open-source community. Divia Mahajan explains why agents can return successful status codes while losing intent, drifting from goals, fabricating around failed tools, or acting on an incorrect view of the world. She recommends deterministic state machines and lineage tracking. Brad presents snapshot-based inference cold starts. His system saves initialized CPU and GPU state, including captured graphs and compiled kernels, then restores it from high-bandwidth storage. The talks share a practical concern: production systems need explicit checks around agent reasoning, tools, state, and actions.

## Key ideas
### Manufacturing agents connect operational data to human decisions
[01:18](https://www.youtube.com/watch?v=nKDKhbPcFcs&t=78s)
Sarmad Absil says manufacturing already produces large amounts of IoT data, but the hard part is acting on it in real time. His proposed system connects sensors, ERP systems, equipment knowledge, and a communication channel such as Microsoft Teams. A triage agent routes questions to inventory, work-order, or knowledge agents. The goal is to move from reactive troubleshooting toward earlier diagnosis and maintenance. Humans still perform physical repairs, while the agents gather the information needed to fix the right equipment without repeated back-and-forth.

### A manufacturing workflow needs context from several systems
[08:08](https://www.youtube.com/watch?v=nKDKhbPcFcs&t=488s)
The proposed workflow starts with sensor data entering Azure IoT Hub and passing through Stream Analytics, where anomaly detection can trigger a Microsoft Teams alert. Data is also stored in Cosmos DB for transactional and analytical use. An orchestration agent can call inventory, work-order, and knowledge agents. In the example, an engineer asks what happened, checks possible fixes from a product manual, retrieves past work orders, checks stock in the ERP, selects an available work slot, and creates a work order. Shift logs give the next team the context left by the previous team.

### Security checks can move into the coding environment
[25:07](https://www.youtube.com/watch?v=nKDKhbPcFcs&t=1507s)
Kamal argues that coding agents have compressed the time between vulnerability discovery and exploitation, while fixing security issues can still take weeks or months. His approach puts threat modeling and software supply-chain checks inside the developer's environment, including Visual Studio Code and Cursor. Developers can inspect a threat model before pushing a feature, generate an implementation for an identified issue, and examine dependencies for vulnerabilities. He also describes security skills stored as markdown files, with proposals and updates contributed through a community and open-source pull requests.

### Agent success metrics miss semantic failures
[41:12](https://www.youtube.com/watch?v=nKDKhbPcFcs&t=2472s)
Divia Mahajan opens with an invoice agent that returned successful responses while matching invoices to the wrong ledger. She explains that agents differ from ordinary software because they reason through multiple steps, depend on finite and lossy context, and can produce semantically wrong results with valid HTTP responses. Monitoring that checks uptime, latency, and exceptions can miss lost intent, gradual goal drift, confident false claims, and state changes that never actually happened. Her examples include a support agent forgetting a cancellation request, a reservation agent wandering through irrelevant research, and an agent approving its own incorrect code.

### Agent failures compound across tools and turns
[49:09](https://www.youtube.com/watch?v=nKDKhbPcFcs&t=2949s)
The failure patterns include empty or erroneous tool results being treated as useful data, system instructions losing influence over long conversations, an agent using its own output as evidence, and an incorrect world model after a partially completed action. Divia recommends validating tool responses before the model sees them, checking instruction compliance over time, separating generation from validation, and confirming the real state after every action. For example, a ticket workflow should create, retrieve, and confirm the ticket before assigning it or notifying the user.

### State machines limit the damage from free-form reasoning
[56:03](https://www.youtube.com/watch?v=nKDKhbPcFcs&t=3363s)
Divia recommends separating reasoning from control flow. A deterministic pipeline can define stages such as gather, analyze, validate, and deliver. Each state has entry and exit conditions, explicit failure transitions, and a maximum number of iterations. The model reasons within a stage, but code decides which stage comes next. This gives each stage a scoped context, makes retries and escalation explicit, and prevents goal drift. Divia's point is that state machines do not remove the model's reasoning ability; they reduce how far a bad decision can spread.

### Lineage makes agent claims traceable
[58:27](https://www.youtube.com/watch?v=nKDKhbPcFcs&t=3507s)
Lineage tracking records what data was produced, which inputs produced it, a confidence score, and a unique identifier for each reasoning step. Later claims carry their parent lineage and grounding status. This lets a team identify where an ungrounded claim first entered a workflow instead of reconstructing the entire failure from logs. Divia places this alongside infrastructure monitoring, structural monitoring, semantic monitoring, and outcome validation. Infrastructure health alone cannot show whether an agent's meaning or final result is correct.

### GPU snapshots reduce inference cold starts
[1:03:03](https://www.youtube.com/watch?v=nKDKhbPcFcs&t=3783s)
Brad explains that serverless inference avoids idle GPU costs, but GPU cold starts can take more than 30 seconds. The delay includes framework initialization, model loading, CUDA graph capture, kernel compilation, and movement of large model files. His snapshot-based approach performs the normal initialization once, saves CPU and GPU state to high-bandwidth storage, and restores that state for later requests. He demonstrates a supported 32B model with a first-token cold start of about 800 milliseconds, followed by a much faster warm request.

### Snapshot inference trades setup work for faster scaling
[1:13:25](https://www.youtube.com/watch?v=nKDKhbPcFcs&t=4405s)
The snapshot approach requires an initial normal cold start before a snapshot exists, high-bandwidth storage, and compatibility between the serving framework and the model. Brad says the system uses H100 hardware, supports vLLM configurations, and can scale replicas when requests exceed an instance's parallel capacity. Replicas can scale back in when traffic stops, so customers do not pay for idle GPUs. Larger multi-GPU models remain harder because transferring state through available bandwidth takes longer.

## Notable quotes
- Sarmad Absil: "The real challenge is not collecting that IoT data. It is actually acting on it in real time." (01:18)
- Divia Mahajan: "Traditional monitoring tells you if your agent is running, not if it's thinking straight." (42:58)
- Divia Mahajan: "State machines don't limit your agents intelligence. They limit the blast radius." (58:00)
- Brad: "With this way we can achieve 30 a 1 second course start for 32B model." (1:02:47)

## Tools & references mentioned
- Microsoft Teams
- Azure IoT Hub
- Azure Stream Analytics
- Cosmos DB
- Azure AI Foundry
- MCP
- Visual Studio Code
- Cursor
- GitHub
- Discord
- vLLM
- Hugging Face
- Google Cloud Run GPU
- NVIDIA
- H100
- AWS
- OpenAI
- OpenRouter
- McKinsey

## Who should watch
- You are building an agent for a factory, plant, or other operation where the agent must combine sensor alerts with ERP and maintenance data before a person acts.
- Your agent returns clean responses and successful status codes, but you have little evidence that it preserved intent, used trustworthy tool results, or changed the real system state.
- You are weighing managed inference against self-hosting and need a practical explanation of GPU cold starts, snapshot restoration, scaling, and idle cost.

## Editor's note

Divia Mahajan shows that an agent can return successful responses while matching invoices to the wrong ledger. ZenML records each run's steps, inputs, outputs, and code version, so a result can be traced to the data and code that produced it. That gives teams a concrete record when they need to find where an unsupported claim entered a workflow.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [The Next Evolution of AI Agents](https://mlopstalks.com/talks/the-next-evolution-of-ai-agents) (Alon Horev, Vast Data, 13:38)
- [AI Agents: The Future of Productivity, or Just a Fad?](https://mlopstalks.com/talks/ai-agents-the-future-of-productivity-or-just-a-fad) (Sam Partee, Arcade AI, 35:18)
- [Sub-Agent Architectures: What You Can Leverage](https://mlopstalks.com/talks/sub-agent-architectures-what-you-can-leverage) (Sidd Seethepalli, Vellum, 14:48)
- [The Future of Compute: How AI Agents Are Reshaping Infrastructure](https://mlopstalks.com/talks/the-future-of-compute-how-ai-agents-are-reshaping-infrastructure) (Diego Oppenheimer, Guardrails AI, 38:34)
- [Everything Hard About Building AI Agents Today](https://mlopstalks.com/talks/everything-hard-about-building-ai-agents-today) (Shreya Shankar & Willem Pienaar, Cleric, 47:03)