High-stakes systems such as fraud detection, underwriting, and search ranking need auditable, low-latency predictions, so agents should improve their models rather than make decisions directly.
2
Agentic experimentation lets an agent create features, train and evaluate a model, and deploy an isolated version to dev before a human reviews and ships it.
3
Branch-based isolation, partial aggregate caching, and semantic hashing keep experiments away from production, reduce repeated computation, and make results reproducible.
Summary
Varant Zanoyan argues that agents should improve the models behind fraud detection, underwriting, search ranking, and other high-stakes systems instead of replacing those models at decision time. These systems need auditability, low latency, high throughput, and stable performance. His proposed pattern is agentic experimentation: an agent creates features, trains and evaluates a model, and deploys a complete pipeline to dev. A human reviews the result and decides whether to run an A/B test or ship it. Chronon provides the data foundation by turning feature definitions into consistent training and serving pipelines. Branch-based resource isolation keeps agent work away from production. Partial aggregate caching avoids recomputing unchanged features, while semantic hashing makes reruns produce the same data. Zanoyan is direct about the limits of the agent layer: Chronon supplies the tools, while downstream teams decide how agents generate ideas and manage experiments. Resource limits provide the practical guardrail when experiment volume grows.
Chronon was built to make fraud-model iteration faster
Chronon began at Airbnb with payments fraud detection. The team could not get new features and models into production quickly because streaming features were difficult to build, feature logic was spread across Spark, Flink, and services, and training data did not always match online inference. Chronon introduced one API for defining features and an automated engine for training-data generation, streaming jobs, state management, serving indices, and inference. Zanoyan says the workflow went from months to about a day or a few days. The same platform later supported account takeover detection, trust models, search ranking, personalization, customer support, and real-time product statistics.
High-stakes decisions need models and rules that agents can improve
Zanoyan groups Chronon's use cases around fraud detection, trust and safety, underwriting, personalization, and customer support. They need real-time predictions, low latency, high QPS, and auditability. A company must be able to explain why it accepted or rejected an insurance claim, for example. Sending thousands of listings and user context to an agent to rank them at request time would break those requirements. The safer pattern is to let an agent improve the rules and machine-learning models that make the decision. The agent can create features, add them to a model, train it, evaluate results, and deploy an end-to-end pipeline to dev.
An agent's output must be ready for review and safe to run
The agent's work cannot be a rough experiment that a human must rebuild from scratch. Zanoyan says the output needs to be production ready, reviewable, and safe. A reviewer should be able to understand the change and decide whether it belongs in a high-stakes system. The agent can prepare a model and pipeline, but it must not interfere with live infrastructure during experimentation. This is why the dev boundary matters. The human reviews the changes, modifies them if needed, decides whether to launch an A/B test, and decides whether the result goes to production.
A semantic feature API prevents infrastructure sprawl
Without the right abstraction, an agent would need to create training data, configure serving pipelines, train a model, deploy it to dev, and keep training and serving data consistent. That could involve SQL in Spark, Snowflake, or BigQuery, real-time work in Flink and Kafka, Kubernetes configuration, and Airflow code across multiple repositories. Zanoyan says this would be difficult to review, easy to get wrong, hard to reproduce, and risky for production. Chronon exposes a semantic API instead. The agent describes the data it wants, how to transform it, and how to connect it to a model. Chronon automates the underlying training and serving infrastructure.
Branch-based isolation keeps experiments off production resources
Chronon associates an agent experiment with a Git branch and routes the branch's jobs to isolated resources. The experiment gets separate compute, storage, and serving from production. This allows an agent to add or modify features and prepare an endpoint without increasing load on the live system. The isolation is managed by the infrastructure rather than by custom code written by the agent. Zanoyan presents this as a safety requirement. The agent can interact with production-oriented infrastructure through the Chronon API, while its jobs and outputs remain separated from the resources handling production traffic.
Isolation alone would be expensive if every experiment recomputed every feature. Zanoyan uses a feature-window example: if 30-day and 60-day aggregates already exist and an agent adds a 7-day window, Chronon can use cached partial aggregates from the raw data. It copies unchanged 30-day and 60-day features into the dev table and computes the new 7-day feature without scanning all raw data again. The agent only asks to add the window and run a backfill. This keeps the result isolated while reusing existing work. Zanoyan says the same reuse matters as experimentation grows, because faster iteration without compute reuse would raise infrastructure costs sharply.
A shared feature repository lets experiments benefit other teams
Chronon is a shared repository, so a feature created by one agent can be used by other agents or human teams. If the required computation has already run and can be reused, later experiments can use it rather than repeating the work. This allows teams to share features across models and use cases. Zanoyan describes the benefit as an economy of scale that grows as more experiments run. The repository therefore supports collaboration as well as caching. It also gives agent builders a common data foundation instead of asking every agent to invent its own cross-system data-engineering logic.
Semantic hashing makes agent experiments reproducible
A reviewer needs to be able to rerun an agent's branch and get the same result. Zanoyan says that arbitrary SQL, copied data, and ad hoc joins make this difficult to understand and reproduce. Chronon uses semantic hashing based on the meaning of the pipelines and their outputs. When an agent changes a job and runs it, the resulting data is guaranteed to be the same when the job is rerun. At column level, Chronon can identify what changed and what did not. It recomputes changed data in an isolated asset and copies unchanged data from production. This supports reproducible dev branches without clobbering production outputs.
Resource limits are the practical guardrail for large-scale experimentation
In the Q&A, Zanoyan says Chronon does not define the ideation process for agents. It provides tools, while downstream teams decide how an agent generates ideas and what business logic it follows. When asked about running 2,000 experiments at once, he points to resource allocation. Teams, agents, or users can receive limits on the compute and data-generation resources they may consume. If too many experiments are requested, they get stuck rather than overwhelming the system. That turns an infrastructure limit into an organizational discussion about cost. The limit can be changed when the responsible team agrees to spend more.
"The human is responsible to review changes, make modifications if it sees fit, decide on whether or not we want to launch an AB test, and decide on whether or not we go to prod with this change that the agent cooked up."07:58
Who should watch
You are building agents that need to change fraud, ranking, underwriting, or other business-critical ML systems without putting live traffic at risk.
Your current agent workflow produces SQL and infrastructure changes across several systems, making experiments hard to review or reproduce.
You need to run many model experiments while controlling repeated computation, production isolation, and infrastructure cost.
