# Bring Your On-Prem ML Use Cases to Production on Google Cloud using Kubeflow

Chanchal Chatterjee, Google | MLOps Meetup | Episode 35 | 20:28
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=JZAc_yZgByg
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/bring-your-on-prem-ml-use-cases-to-production-on-google-cloud-using-kubeflow
Published: 2020-09-28
Tags: model-serving, open-source, orchestration

## TL;DR
- Chanchal Chatterjee created the open-source ML Pipeline Generator to reduce the effort of moving existing models into production on Google Cloud.
- The tool uses configuration files and generated components to build Kubeflow pipelines for scikit-learn, XGBoost, and TensorFlow models.
- The implementation covers data preparation, hyperparameter tuning, training, deployment, prediction, and explainability, but it was tested only on Google Cloud.

## Summary
Chanchal Chatterjee presents ML Pipeline Generator, an open-source project created from Google customer engagements. One retail customer had a churn model that took a team four to six weeks to move onto Google Cloud. Chatterjee wanted a lower barrier for customers with hundreds of models. The project accepts model and preprocessing code, a configuration file, and a selected set of pipeline steps. It then generates Kubeflow pipeline components without requiring users to learn Kubeflow's domain-specific language. The supported frameworks are scikit-learn, XGBoost, and TensorFlow. The generated pipeline can include data preparation, hyperparameter tuning, training, deployment, prediction, and explainability. Chatterjee also describes a possible hybrid or multi-cloud arrangement through Anthos, while being clear that the current implementation contains Google Cloud-specific code and has only been tested there. The session is a hurried overview rather than a live demonstration, with the presenter offering to provide a later demo video.

## Key ideas
### The project targets the time required to move existing models into production
[02:40](https://www.youtube.com/watch?v=JZAc_yZgByg&t=160s)
Chanchal Chatterjee says Google customer engagements often require a program manager, a software engineer, and several weeks of work to move one or two models onto Google Cloud. One retail customer had built a churn prediction model. The process took four to six weeks, which does not scale when a customer has hundreds of models. The ML Pipeline Generator was created to lower that barrier and make the process much faster. Chatterjee describes the goal as reducing the work to hours in some cases, while admitting that model-specific complexity can still make the process harder.

### ML Pipeline Generator supports three model frameworks
[04:56](https://www.youtube.com/watch?v=JZAc_yZgByg&t=296s)
The open-source solution generates ML pipelines for XGBoost, scikit-learn, and TensorFlow. Chatterjee distinguishes it from TFX, which primarily handles TensorFlow, because his team wanted one more comprehensive solution across the three frameworks. The project uses a templating framework called Jinja. Existing model files or notebooks can be connected to the generated pipeline. The repository contains examples for each supported framework, so users can see how the configuration and generated Kubeflow pipeline fit together.

### The generated pipeline covers the full path from data to serving
[06:57](https://www.youtube.com/watch?v=JZAc_yZgByg&t=417s)
Chatterjee lists the blocks implemented by the solution: data preparation, hyperparameter tuning, model training, model deployment, model prediction, and model explainability. He wants the resulting system to be scalable, composable, secure, and orchestrated by Kubeflow. The design can also separate parts of the workflow. For example, data could remain in an on-premises environment while prediction runs elsewhere. Chatterjee says Anthos provides a framework for a multi-cloud setup, although the implementation shown in the talk does not itself cover that arrangement.

### A configuration file drives pipeline generation
[08:44](https://www.youtube.com/watch?v=JZAc_yZgByg&t=524s)
The proposed workflow starts with the Git repository and a config.yml file. The file identifies the data and model, and it can describe anything from a simple SVM to a more complex deep learning model. Users can add their own model.py and preprocessing.py modules. The repository's library files then generate the Kubeflow pipeline components. Chatterjee presents this as a way for software engineers to package an existing model without first learning Kubeflow's domain-specific language.

### Users compose only the pipeline steps they need
[11:45](https://www.youtube.com/watch?v=JZAc_yZgByg&t=705s)
After running the generator program, users get framework-specific modules such as scikit-learn, TensorFlow, or XGBoost modules. They then combine them with simple calls such as add_train, add_deploy, and add_predict. Hyperparameter tuning can be omitted when it is unnecessary. Chatterjee gives another example that combines data preparation, training, deployment, and explainability. The generator turns those selections into a pipeline rather than requiring users to write every component themselves.

### The session does not include a live demo
[14:10](https://www.youtube.com/watch?v=JZAc_yZgByg&t=850s)
Chatterjee cannot run the demonstration during the meetup because he joined with little preparation time and has another meeting shortly afterward. He says the repository provides the step-by-step material and that the process should be simple for a software engineer. He offers to send the presentation as a PDF and make a separate demo video for the audience. The talk therefore explains the repository workflow and its intended abstractions instead of showing a completed run.

### Portability requires adapting Google Cloud-specific parts
[16:56](https://www.youtube.com/watch?v=JZAc_yZgByg&t=1016s)
In response to a question from Demetrios Brinkmann about cloud-provider independence, Chatterjee says the early parts of the project contain Google Cloud-specific constructs. Some later code is more cloud-independent when a provider already offers a Kubeflow framework, but training and related modules can still include Google Cloud Platform items. The hyperparameter tuning configuration and deployment examples were tested on Google Cloud only. Supporting Azure or AWS would require generalizing and modifying the code. The repository is open source, so users can submit pull requests for unsupported cases.

## Notable quotes
- Chanchal Chatterjee: "The difficulty is to bring machine learning into production." (02:40)
- Chanchal Chatterjee: "We should be able to do it in hours because many customers have hundreds of models." (04:11)
- Chanchal Chatterjee: "You don't even have to learn much about Kubeflow." (11:20)
- Chanchal Chatterjee: "We only tested it on the Google Cloud, we did not test it on Azure or AWS." (18:58)

## Tools & references mentioned
- Google Cloud
- ML Pipeline Generator
- Kubeflow
- Kubeflow Pipelines
- Jinja
- XGBoost
- scikit-learn
- TensorFlow
- TFX
- Anthos
- GitHub

## Who should watch
- You have existing scikit-learn, XGBoost, or TensorFlow models on-premises and need a starting point for moving them into Google Cloud.
- Your team wants to generate Kubeflow pipelines without writing every pipeline component or learning Kubeflow's domain-specific language first.
- You need to assess whether an open-source Google Cloud implementation can be adapted to another cloud provider.

## Editor's note

Chanchal Chatterjee says moving one retail customer's churn model to Google Cloud took four to six weeks. ZenML lets teams write workflows as Python steps and run the same pipeline on a laptop, Kubernetes, Airflow, Kubeflow, or a cloud provider by changing the configured stack. That gives teams a common workflow as infrastructure changes.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Optimizing Your ML Workflow with Kubeflow 1.0](https://mlopstalks.com/talks/optimizing-your-ml-workflow-with-kubeflow-1-0) (Josh Bottum, Arrikto, 1:03:41)
- [Re-Platforming Your Tech Stack](https://mlopstalks.com/talks/re-platforming-your-tech-stack) (Michelle Marie Conway & Andrew Baker, Lloyds Banking Group, 51:15)
- [Kubeflow vs MLflow](https://mlopstalks.com/talks/kubeflow-vs-mlflow) (Byron Allen, Servian, 54:57)
- [Team Aurora: Accelerating ML with Kubeflow](https://mlopstalks.com/talks/team-aurora-accelerating-ml-with-kubeflow) (Maurizio Vitale, Vinay Anantharaman & Ankit Aggarwal, Aurora, 54:57)
- [Setting up an ML Platform on GCP: Lessons Learned](https://mlopstalks.com/talks/setting-up-an-ml-platform-on-gcp-lessons-learned) (Mefta Sadat, Loblaw Digital, 39:54)