# Creating MLOps Standards

Alex Chung, Social Good Technologies & Srivathsan Canchi, Intuit | MLOps Coffee Sessions | Episode 50 | 47:45

Source: https://www.youtube.com/watch?v=5lRpq0SNUr8
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/creating-mlops-standards
Published: 2021-08-12
Tags: deployment, developer-experience, governance, platform-teams

## TL;DR
- MLOps teams need common interfaces because enterprise machine learning combines many tools across data, training, deployment, monitoring, metadata, and artifacts.
- Internal standards can hide infrastructure differences from data scientists while allowing platform teams to enforce security, compliance, logging, and monitoring requirements.
- mlctl aims to make jobs portable across SageMaker, Kubernetes, Kubeflow Pipelines, Azure, and other systems, with platform changes controlled through configuration rather than rewritten model code.

## Summary
Alex Chung and Srivathsan Canchi describe why enterprise ML teams struggle with tooling sprawl. A company may have many data science groups, each using different orchestrators, model servers, metadata systems, and cloud services. The result is repeated boilerplate and long handoffs between model code and production infrastructure. Canchi describes Intuit's platform, where mlctl gives teams a common interface across training, batch hosting, online hosting, and workflow tools such as Airflow and Kubeflow Pipelines. Chung explains the related work of Social Good Technologies, which focuses on education, standards discussions, and open source projects. Their proposed direction is a control layer that lets teams choose tools for specific jobs while keeping a shared experience for data scientists and consistent operational rules for the company. They expect the approach to matter most for organizations with many data scientists and multiple infrastructure systems.

## Key ideas
### MLOps has a larger integration problem than ordinary application deployment
[10:10](https://www.youtube.com/watch?v=5lRpq0SNUr8&t=610s)
Alex Chung describes ML production as a lifecycle with data collection, processing, feature engineering, labeling, model design, training, optimization, deployment, and monitoring. Metadata, experimentation logs, and artifacts often belong to separate tools. Kubeflow Pipelines may handle workflow orchestration while teams use MLflow for experimentation, which shows how systems are combined instead of used as one complete platform. Chung says enterprise teams need flexibility to choose tools for different use cases while still sharing governance, spend tracking, and resource allocation.

### Kubernetes provides a useful model for portable interfaces
[07:40](https://www.youtube.com/watch?v=5lRpq0SNUr8&t=460s)
Srivathsan Canchi compares the MLOps problem with the earlier cloud infrastructure problem. Before Kubernetes, teams had different ways to build and operate applications on AWS, Azure, Google Cloud, OpenStack, or local systems. Kubernetes introduced a control plane and common interfaces, so an application could move between environments without requiring a completely new set of commands. Canchi sees a similar opportunity in ML, where common interfaces could make models and workflows easier to move across infrastructure and allow more people to use machine learning.

### Enterprise scale makes tooling sprawl expensive
[17:59](https://www.youtube.com/watch?v=5lRpq0SNUr8&t=1079s)
Chung says large companies often contain many independent data science teams with different tooling approaches. One group may use Airflow, another Kubeflow Pipelines, and others may use different model servers or metadata services. Internal standards can extract the repeated integration work from model code. The data scientist should not have to relearn how a training container receives parameters or how an artifact reaches production every time the company's infrastructure changes. This matters most when a platform supports dozens or hundreds of data scientists.

### Intuit built a platform around a large production ML estate
[15:43](https://www.youtube.com/watch?v=5lRpq0SNUr8&t=943s)
Canchi says Intuit had roughly 250 to 300 data scientists and about 150 engineers supporting them. More than 400 models were in production, compared with about 30 when he joined the team. Intuit also had a feature store with more than 10,000 features, many shared across the company. The platform connected data exploration, feature engineering, model training, evaluation, and production serving. Models could run in online applications or in offline batch workloads.

### mlctl gives different infrastructure systems one interface
[27:04](https://www.youtube.com/watch?v=5lRpq0SNUr8&t=1624s)
Canchi explains that mlctl began as an internal Intuit project. Data scientists use a common interface even when model training runs on SageMaker, batch hosting runs on Kubernetes, and online hosting runs on SageMaker. The open source version is a smaller form of the internal library, with added work for Azure, Kubernetes, and SageMaker. It can create boilerplate for a new ML project, import that setup into a notebook, and provide consistent ways to publish experiment metrics, persist metadata, and store artifacts.

### The platform layer lets teams keep their workflow preferences
[39:01](https://www.youtube.com/watch?v=5lRpq0SNUr8&t=2341s)
At Intuit, one business unit uses Airflow and another uses Kubeflow Pipelines. The platform team allows both choices while requiring shared standards for compliance, security, logging, and monitoring. mlctl is integrated into both orchestration systems, and a third group uses it from Jenkins. The same library applies the platform's operational requirements regardless of where a job is triggered. This gives teams a choice of workflow tool without giving up the production controls supplied by the ML platform.

### The proposed target is configuration-level portability
[35:16](https://www.youtube.com/watch?v=5lRpq0SNUr8&t=2116s)
Chung says the near-term goal is support for several commonly used ML platforms, including SageMaker, Azure ML, Kubeflow Pipelines, and Databricks. A user should be able to define a job once and switch the underlying platform or orchestrator by changing configuration settings. The model code should not need to change. He expects larger enterprises with multiple tools to benefit from a shared library because the basic needs are similar: infrastructure portability, a simpler developer experience, and flexible composition of tools.

### Open source standards will need both vendors and independent projects
[42:08](https://www.youtube.com/watch?v=5lRpq0SNUr8&t=2528s)
Chung expects different kinds of open source projects to participate. Some projects are closely tied to a vendor, while others began inside companies to solve a specific problem and were later released publicly. Commercial open source companies may promote standards that also support their products. A project such as mlctl can address a different need because it is aimed at enterprises using several tools and does not have to favor one vendor. Social Good Technologies is intended to provide a forum for these discussions and for incubating projects.

## Notable quotes
- Alex Chung: "The biggest challenge of selling SageMaker into enterprises was that there was no common integration point." (04:52)
- Srivathsan Canchi: "Kubernetes said, 'We will create this control plane which can operate across any cloud.'" (08:34)
- Srivathsan Canchi: "The platform team comes in and says, 'You use the orchestration tool of your choice, but we would like to make sure that these certain basic tenets are followed.'" (39:48)
- Alex Chung: "The jobs themselves should be portable because in a multi-cloud setting that's actually very valid." (30:27)
- Alex Chung: "It should just be config settings that change." (36:21)

## Tools & references mentioned
- Social Good Technologies
- AWS SageMaker
- Kubernetes
- Kubeflow Pipelines
- MLflow
- Weights & Biases
- Airflow
- Jenkins
- Azure ML
- Databricks
- mlctl
- Baklava
- Sriracha
- Feast
- Metaflow
- Prefect
- Seldon
- Snorkel
- Demetrios Brinkmann
- Morgan Stanley

## Who should watch
- You run an enterprise ML platform and several data science groups keep choosing different orchestration, serving, and metadata tools.
- Your model teams spend weeks connecting model code to infrastructure after the model itself is finished.
- You want to understand how a common SDK could preserve team choice while enforcing production requirements across Airflow, Kubeflow Pipelines, SageMaker, Kubernetes, or similar systems.

## Editor's note

Srivathsan Canchi describes how Intuit used mlctl to give teams one interface while training, batch hosting, and online hosting ran on different systems. ZenML lets teams write pipelines as Python steps and choose the orchestrator and other infrastructure through stack configuration. The same pipeline code can run on Kubernetes, Airflow, Kubeflow, or a cloud service.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Packaging MLOps Tech Neatly for Engineers and Non-engineers](https://mlopstalks.com/talks/packaging-mlops-tech-neatly-for-engineers-and-non-engineers) (Jukka Remes, Haaga-Helia University of Applied Sciences, 8wave AI, 55:31)
- [DevOps, Security, and Observability in ML](https://mlopstalks.com/talks/devops-security-and-observability-in-ml) (Luke Marsden, MLOps Consulting, 32:46)
- [Designing ML Infra for ML & LLM Use Cases](https://mlopstalks.com/talks/designing-ml-infra-for-ml-llm-use-cases) (Amritha Arun Babu, Klaviyo & Abhik Choudhury, IBM, 1:00:18)
- [SGT Model Deployment Working Group](https://mlopstalks.com/talks/sgt-model-deployment-working-group) (Srivathasan Canchi, Intuit, 46:51)
- [Doing MLOps](https://mlopstalks.com/talks/doing-mlops) (Noah Gift, Pragmatic AI Labs, 1:01:22)