Vast Data built an operating system that combines data storage, compute services, and applications for AI workloads.
2
Alon Horev says production agents need confidence, security, and efficiency, with fine-tuning and RAG helping improve their reliability and usefulness.
3
Reactive pipelines can watch for events such as video uploads, invoke a vision-language model, index the result, and trigger an agent only when needed.
Summary
Alon Horev describes Vast Data's operating system for AI, which combines storage, a database, data movement across locations, and compute services. He explains why large training systems need high-throughput checkpointing because hardware failures interrupt workloads, then turns to the practical problems of deploying agents. Customers need confidence that agents produce trustworthy answers, security controls that define identity and permissions, and enough efficiency to control cost and latency. Fine-tuning adapts a model to application feedback, while retrieval-augmented generation brings current information and citations into the workflow. Horev also describes vector embeddings, vision-language models, reactive programming, and observability. His example uses NVIDIA's Cosmos Reason to analyse uploaded video, index its meaning, and trigger downstream actions through Vast Data's Data Engine. The talk presents infrastructure choices as part of agent design, especially when data, GPUs, models, and audit requirements span multiple locations.
Vast Data combines storage and compute into an AI operating system
Horev defines Vast Data's operating system as a platform that knows how to hold data and run compute services. Those capabilities support AI applications together. The company started with storage for big data and later saw customers storing metadata, statistics, metrics, and vector embeddings. That led to the Vast database, launched three years before the talk. Data Space moves data across geographies and data centers when training or inference needs it elsewhere. Data Engine adds compute capabilities. Horev's point is that AI infrastructure has to keep data and computation close because an LLM depends on both its architecture and the data it uses.
Large training jobs need checkpoint storage because hardware fails
Horev describes training as many GPUs working together like one distributed brain. He gives Meta's reported experience with more than 16,000 GPUs used for Llama training, where a GPU was lost every three hours. Checkpoints protect the work by dumping GPU memory to storage at intervals. If a GPU fails, the system restores the checkpoint and continues. In his example, checkpoint writes enter the system at about a terabit per second, followed by read spikes of around one terabyte per second during recovery. He says newer workloads also process raw video and require exabytes of capacity, which makes the storage path even more demanding.
Production agents have to earn confidence, pass security checks, and control cost
Horev groups the problems his customers face when bringing agents into production into confidence, security, and efficiency. Confidence means deciding whether an application can speak to customers or give advice about medicine. Teams add guardrails and fact-based decision-making to improve trust. Security teams want proof that an agent does not leak information, follows its intended behavior, and leaves a complete audit trail. Efficiency concerns cost, speed, scale, and whether a company can run an agent for every customer or employee. He says customers can address these concerns through model choices, data controls, monitoring, and infrastructure that can move workloads to suitable GPU locations.
Fine-tuning and RAG solve different problems and can work together
Fine-tuning uses examples from an application, including good and bad interactions, to continue training a model for a particular use case. Horev says this can improve performance and address decay, which happens when a model's training data becomes less relevant to current users. Retrieval-augmented generation brings information into the application at request time, so it can reflect current data. It also supports citations, such as legal precedents or medical research. Horev treats the methods as complementary. Fine-tuning changes the model through feedback, while RAG supplies live context and evidence during an interaction.
Embeddings let agents search by meaning across text, images, and video
Horev contrasts older literal search with semantic search from modern language models. An embedding model converts text, paragraphs, or other inputs into vectors, which are groups of numbers carrying semantic meaning. A query such as a request for pictures of cats can then find images whose embeddings indicate that a cat appears in them. Newer embedding models support vision, so vision-language models can create applications that interpret images and video. Horev connects this capability to robotics, safe cities, and retail systems. The same approach lets an agent retrieve information based on what content means rather than only on the exact words used.
Agent security requires identity, permissions, auditing, and one policy across data types
Horev says an agent needs an identity and a permission model that limits what it can do. Teams also need to observe and audit every action. Vast Data stores unstructured material such as documents and images alongside structured information such as vector embeddings and metadata. Horev says the platform applies a unified security model across those modalities. This matters because an agent may retrieve a document, use metadata or embeddings to find it, and then call a tool or write a result. Security teams need controls and records that cover the whole path rather than only the final answer.
Reactive programming prevents agents from wasting GPU time
Horev warns that a continuously running agent can create an infinite loop and keep a GPU busy. He uses a hypothetical stock-monitoring agent that repeatedly checks news about Apple as an example. Reactive systems wait for something to happen, then invoke the agent or language model when it is needed. This approach limits expensive model calls and gives the application a clear event-driven structure. The same idea applies to data and video pipelines. A new clip, a changed record, or another defined event can trigger inference and a decision instead of forcing the system to poll constantly.
A video pipeline can turn an upload into indexed meaning and an agent action
Horev presents an application that accepts a video and sends it through NVIDIA's Cosmos Reason vision-language model. The model produces a paragraph describing what appears in the video. When a clip enters the system, a function is invoked in real time, the model derives meaning, and that meaning is indexed in the database. An agent can then react to the upload, perform inference, and make a decision. Horev identifies this pipeline with Vast Data's Data Engine, where developers connect triggers and functions. The system supports a graphical experience and YAML, so developers can define the workflow through the interface or code.
Observability has to cover dynamic agent behavior in real time
Horev says AI systems produce new information on every run, so developers need to observe and monitor them as they operate. He describes an integrated software development kit that lets application developers create logs and traces and monitor the system in real time. That information supports changes to the application while it is running in production. In his framing, observability is part of the agent architecture rather than a separate report added afterward. It connects the triggers, functions, model calls, and resulting actions so teams can understand what happened and adjust the workflow.
