mlctl gives data scientists one SDK and CLI for running machine learning jobs across different infrastructure providers.
2
The library uses adapters to connect common project code and commands to systems such as SageMaker, Azure ML, Kubernetes, and Kubeflow Pipelines.
3
Intuit designed mlctl to work with existing enterprise tools incrementally, rather than requiring teams to replace their current platforms.
Summary
Srivathasan Canchi introduces mlctl, an open-source MLOps toolkit developed at Intuit. The library provides a common SDK and CLI for creating projects, packaging code, building containers, running training jobs, storing artifacts, and logging metrics across different infrastructure providers. Its adapters connect the same data scientist workflow to systems such as SageMaker, Azure ML, Kubernetes, and Kubeflow Pipelines. The demonstration starts with a scaffolded Python project, then adds training adapters for input data, metrics, model artifacts, and provider-specific paths. A second demonstration shows declarative YAML files for configuring Azure ML jobs. The related Baklava library builds universal machine learning containers from Python packages and adds hosting interfaces. The presenters describe mlctl as an interconnection layer for enterprises that mix managed services, open-source tools, and internal systems. They argue that workflow orchestrators may become interchangeable, while reusable job components and the connections between tools matter more.
MLOps is a set of connected stages with feedback loops
Canchi describes MLOps as a sequence of data collection and processing, feature engineering, labeling, model design, training, optimization, deployment, and monitoring. The process is not linear. A team may train a model, discover that its features are inadequate, and return to feature engineering before deploying. Intuit uses a mix of vendor products, open-source libraries, and internal tools at these stages, so the problem is connecting them without exposing data scientists to every underlying system.
mlctl gives different tools one common control layer
The mlctl idea came from the need to provide one standardized layer for launching jobs, updating them, checking their status, and connecting systems across the MLOps lifecycle. Canchi compares this role with how Terraform abstracts cloud infrastructure and how other tools manage Kubernetes or data pipelines. mlctl provides a platform SDK that turns model code into a universal MLOps container and handles enterprise concerns such as data versioning, metadata tracking, artifact storage, and authentication.
The same project can route training to different backends
At Intuit, Kubeflow Pipelines invokes mlctl, while training and deployment can happen on different infrastructure. The data scientist uses one interface even when one job runs on Kubernetes and another runs on SageMaker. The open-source release initially focused on SageMaker, with beta support planned for Azure ML and Kubeflow Pipelines. The presenters describe the release as early, with the basic functions working while provider-specific integrations continue to develop.
Training adapters hide provider-specific data and artifact paths
The training adapter connects model code to the infrastructure running the job. Instead of reading a fixed local CSV path, the code calls adapter helpers such as input-as-data-frame and names a channel such as training, test, or validation. The adapter supplies the provider-specific location and environment details. Similar functions log metrics, resolve the correct model artifact path, and send artifacts to the underlying provider or to MLflow when that integration is configured.
The CLI scaffolds projects and packages them into deployable jobs
The demo uses mlctl init to create a project folder with a name, slug, description, and version. The scaffold includes Python packaging files, train.py, and predict.py. A basic scikit-learn decision tree example is adapted to the mlctl format. The project can then build a training container, push it to a cloud container repository, and run it through commands such as mlctl train build, mlctl train push, and mlctl train run.
Baklava builds universal containers from Python packages
Baklava is a separate open-source library that works closely with mlctl. It extends the Python setup tools packaging system so Python code and its dependencies can become Docker containers. The container can target SageMaker, Azure, or other environments. For hosting, Baklava adds the interfaces required by the target system, such as health checks and expected request paths. The presenters say this hides web-server and scaling details from data scientists.
mlctl is intended to connect existing systems rather than replace them
The presenters describe mlctl as a neutral library for interconnection. Its adapters let an enterprise keep existing components while adding connections to other backends. They argue that large companies will not remove systems they have already built and invested in, so an incremental translation layer is more practical than introducing a new domain-specific language or replacing the entire platform. The intended users include smaller teams tied to one cloud and large enterprises planning to combine cloud, on-premise, open-source, and internal systems.
Workflow orchestrators may become interchangeable while job components matter more
The discussion broadens the meaning of MLOps orchestration beyond constructing workflow graphs. The speakers observe that companies use different workflow systems, including Airflow and Kubeflow Pipelines, while recreating similar hooks around the jobs themselves. They expect workflow orchestrators to become more interchangeable over time. In that model, mlctl and Baklava provide reusable job packaging and connections to services such as monitoring, metadata tracking, model serving, and feature systems.
"The core differentiation here is really going incrementally and solving each one of the steps where you can pack it using existing tools like Kubeflow Pipelines and then also reuse your existing component code."36:25
Who should watch
You are building an internal ML platform and need one workflow across managed services, Kubernetes, and open-source tools.
Your team has existing training, serving, or metadata systems and wants to connect them without replacing everything.
You want to see how a common project scaffold, training adapter, container builder, and YAML configuration fit together in an enterprise MLOps workflow.
