Ketan Umare argues that machine learning workflows need to preserve data and time context because backfills can change the meaning of past decisions.
2
Flyte focuses on broad workflow needs, while Union adds managed features such as dynamic multi-cluster support, single sign-on, task-level monitoring, and resource management.
3
Ketan Umare says large language model infrastructure still follows the same basic pattern as classical machine learning, with data preparation, training or fine-tuning, evaluation, and deployment.
Summary
Ketan Umare and Stephen Batifol discuss where ML orchestration differs from ordinary data workflow orchestration, using Flyte and Union as the main examples. Ketan explains why machine learning systems need to treat time, data, experiments, and backtesting as things users must be able to change together. He also describes how the Flyte community grew through open development and direct support, while Union takes on enterprise features that would add too many dependencies or security risks to the open-source project. The conversation covers real-time ML, where Ketan argues that noisy signals often need buffering rather than immediate reactions. He describes Union features including dynamic cluster management, single sign-on, task-level resource monitoring, and cost visibility. On large language models, he says the infrastructure is familiar, although GPU use, prompting, checkpointing, and evaluation create specific demands. He is especially cautious about training models from scratch when an existing open model can be fine-tuned.
Building a company gave Ketan a useful kind of failure
Ketan Umare says he would be happy at 50 even if Union failed, because he had tried to build a difficult product, company, and community. He describes startup life as aging him by about ten years and says he warned his wife that the company might not make money at first. His motivation came from wanting to do something different rather than chasing the excess he had seen in the technology industry. He tells people to choose a startup because they want the experience, not because they expect money. The point is personal rather than financial: he sees the company as the best experience of his life and still wants to come to work and keep building.
Flyte's community grew through support and sustained open-source work
Ketan says communities are harder to build than companies because they need passion and care rather than money. He describes a Slack community where questions continue to receive answers, including from him. The team did not build a toy, make promises they could not keep, or stop improving the project. One sign of lasting adoption is that people leave a company, join another, and ask to bring Flyte with them. Stephen Batifol adds that newcomers receive patient explanations and empathy, even for questions that may seem simple to experienced users. Ketan says the community includes Kubernetes users, ML specialists without data engineering experience, data engineers moving toward ML, and people who have never used Docker.
ML workflows need control over time and historical data
Ketan says people coming from ETL often expect ML orchestration to behave like their existing data workflows. In ML, time is not just another parameter. Users may need to change time, data, models, ideas, and code while backtesting or testing a hypothesis. He warns that deleting a table and backfilling it can erase the evidence behind an earlier decision. His bank-loan example shows the risk: a customer could be denied one day and approved the next, while a later data correction makes it impossible to determine what happened. ML products often begin as experiments with uncertain value, so systems need to support exploration without losing the context of previous outcomes.
Real-time predictions often need dampening before they need faster infrastructure
Ketan uses Lyft's estimated-time-of-arrival work to explain why real-time data is not automatically better. The team considered using real-time traffic to improve predictions, but collecting, processing, and analyzing that data was difficult. A single stopped car could look like a traffic jam, causing the estimate to jump unnecessarily. The system needed buffering and additional models to avoid reacting to noisy signals. Ketan says hourly predictions were already strong, and traffic mainly helped with the tail cases. In fraud detection, he gives a similar example: one fraudulent event should not immediately cause the system to reject everyone else. The right interval depends on the signal and the application.
Open-source scope depends on installation cost and dependency size
When users request features, Ketan weighs the benefit of a specialized capability against the cost it adds to every Flyte installation. A feature for very fast workflows might require hard container startup, distributed queues, leader elections, and many supporting systems. Large companies can assign teams to manage that stack, while a small company may have only a few engineers. Union can take on some of these needs as a managed product, while Flyte keeps a smaller surface area and uses plugins or integrations where possible. Ketan says the team listens to requests, explains tradeoffs, and sometimes recommends another tool when it is a better fit.
Security-sensitive enterprise features are difficult to standardize in open source
Ketan uses single sign-on and role-based access control to explain why some features belong in Union. Customers ask for integrations with many identity providers, but the team would need to support different systems and sometimes build missing authorization pieces. He says this creates security risks and makes it hard to offer one open-source implementation that works everywhere. Union provides managed SSO and role-based access control down to an individual workflow. Ketan is direct that this is not because the team ignored the requests. The challenge is that there is no common standard across the systems customers want to use, and secure operation requires ongoing engineering and management.
Union adds managed cluster and resource visibility to Flyte workflows
Ketan describes Union as an inverted model in which customers can connect clusters from different clouds and let the service manage their lifecycle. Flyte can scale a single Kubernetes cluster to very large workflow volumes, while Union supports dynamic clusters for teams that need separate projects, cloud accounts, or GPU capacity. He also describes task-level monitoring for CPU, GPU, memory, and storage use. A broader dashboard lets platform teams see usage by project, domain, or workflow and investigate spending. Ketan says ML teams often request more resources than they use, so this visibility can reveal substantial unused capacity. Union also works on lineage and artifact sharing.
LLM infrastructure is familiar, but evaluation and GPU use make the work harder
Ketan breaks LLM infrastructure into preparing data, fine-tuning or training, updating prompts, evaluating the result, and deploying it. He questions whether most companies should train a model from scratch when open models such as Llama 2 are available, unless they have a strong reason. Training still needs distributed GPU orchestration, efficient interconnects, checkpointing, and metrics. Flyte integrates with Hugging Face so users can checkpoint training and use spot machines, and it supports Ray for teams that prefer it. Ketan says prompting and evaluation are especially difficult because even detecting a hallucination may require checking an external body of knowledge. A benchmark or golden set can help, and human review makes many applications more practical.
