Rafael Pierre says Kubernetes gives ML teams power, flexibility, isolation, and reproducibility, but it requires enough organizational maturity and support.
2
Databricks Model Serving V2 provides serverless real-time endpoints, reducing the operational work needed to deploy and run ML models.
3
Machine learning teams must track code, models, and data, including model lineage, permissions, training data, metrics, and promotion history.
Summary
Rafael Pierre describes his path from software engineering in Brazil's stock exchange to data-intensive systems, machine learning, and his work at Databricks. He recalls building real-time vehicle tracking systems in 2010 with TCP sockets and bare-metal infrastructure, before cloud platforms offered elasticity. Rafael explains why Kubernetes is useful for ML: it provides flexible resource allocation, workload isolation, and reproducible environments. He is also direct about its costs. Teams need Kubernetes knowledge, security practices, scaling plans, and ongoing operational ownership. For smaller or less mature teams, Docker on a virtual machine may be enough. Rafael compares this approach with Databricks Model Serving V2, which offers serverless real-time endpoints and removes much of the platform management burden. He also discusses promotion pipelines, regulated fraud detection, access control, model lineage, and the need to manage code, models, and data as separate artifacts.
Rafael started with real-time data systems before cloud platforms were available
Rafael Pierre began as a software engineer in the Brazilian Stock Exchange and later worked on vehicle and fleet tracking in Israel. In 2010, cars and trucks sent data to TCP sockets running on bare-metal servers in a data center. The system supported real-time geofencing, such as detecting when a FedEx truck driver left an assigned area. It also supported longer-term analysis of fuel use and driver behavior. Rafael describes this as an early IoT system built before the cloud, with scaling handled through infrastructure planning rather than elastic capacity.
Cloud elasticity changed how Rafael thought about infrastructure
Rafael says the cloud's biggest benefit for him is elasticity. His earlier systems ran on bare metal, so teams had to predict when they needed more capacity and physically manage the data center. Kubernetes initially felt difficult because of its terminology and networking concepts. Once he understood it, he saw that code could control an entire data center. A horizontal autoscaler could be created and left to adjust capacity, which meant he no longer had to deal directly with bare-metal infrastructure.
Kubernetes gives ML teams control, isolation, and reproducibility
Rafael says Kubernetes is powerful for ML because it combines elasticity, flexibility, and isolation. Those properties help with production workloads and with reproducibility. A data scientist can package a model in a Docker image, then Kubernetes can allocate the memory and CPU needed for that workload instead of tying it to one fixed virtual machine. He contrasts this with a Docker engine running inside a VM, where the available stack and machine resources impose tighter limits.
Kubernetes is a poor fit when teams cannot support its operational demands
Rafael has seen organizations where individual teams adopted Kubernetes without anyone around them who understood or could support it. He says teams either need a build-it-and-own-it commitment or support from a specialized group. One machine learning center of excellence he worked with acted as a Kubernetes gatekeeper and internal consultancy, providing blueprints, CI/CD pipelines, Helm charts, and fully automated deployments. Smaller teams without the required scale, maturity, or investment may be better off running Docker on one or more virtual machines.
Healthy ML organizations blur the boundaries between job functions
Rafael says company culture affects whether ML work becomes collaborative or remains divided into silos. In a healthy organization, data scientists, data engineers, ML engineers, and SREs understand enough of one another's work to share responsibility. He describes code reviews between data scientists and ML engineers as a productive form of collaboration. In a siloed organization, a data scientist may hand over a Jupyter notebook and avoid responsibility for what happens after training, while an engineer becomes responsible for production issues without shared context.
Databricks Model Serving V2 removes much of the platform management work
Rafael compares running ML infrastructure on Kubernetes with Databricks Model Serving V2. The Databricks offering provides a serverless real-time endpoint, so teams do not need to manage the underlying serving platform themselves. He compares this choice with using a managed Kafka service instead of operating an Apache Kafka cluster. Databricks also brings model serving into the same platform as other data and ML work, which can reduce the number of separate systems a team must operate.
Release controls should match the cost of model failure
Rafael says there is no single best process for promoting models from experimentation to production. A fraud detection model in a tightly regulated setting may require unit tests, integration tests, reviews by several people, A/B testing, and more advanced methods such as Thompson sampling. A less critical model may need fewer controls. He connects this process to mature software development, while adding that ML has more artifacts to manage. Code, models, and data each have different quality and governance concerns.
Model lineage needs to include training data, changes, and approvals
Rafael argues that teams need visibility into how a model was created and changed. A model registry should expose metadata, features, the data version used for training, metrics, and hyperparameters. Teams should also know who trained or changed a model, who approved its promotion, and when it moved from development to staging and production. He calls model lineage as important as data lineage, especially when people leave teams or data changes. Rafael names MLflow as a mature option and notes that automatic logging can capture much of this information with one line of code.
"If you don't have enough maturity and at the same time if you don't require that level of scale, you might be pretty okay running Docker in a virtual machine."Rafael Pierre16:36
Who should watch
You are deciding whether to run ML workloads on Kubernetes and need a practical view of the staffing, support, and scaling requirements.
Your team is moving models from notebooks into production and needs to think through release controls, testing, approvals, and model lineage.
You are comparing self-managed ML infrastructure with a managed platform such as Databricks Model Serving V2.
