# Building an ML Platform from Scratch: Live Coding Session

Alon Gubkin, Aporia | MLOps Meetup | Episode 67 | 1:57:24
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=s8Jj9gzQ3xA
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/building-an-ml-platform-from-scratch-live-coding-session
Published: 2021-06-11
Tags: deployment, experiment-tracking, model-serving, platform-teams

## TL;DR
- An ML platform should be designed around an organisation's specific problems, with each tool evaluated against its alternatives.
- The workshop combines DVC, MLflow, FastAPI, Kubernetes, Pulumi, AWS services, Traefik, Poetry, Docker, GitHub Actions, Cookiecutter, and S3 to create a working platform.
- The demo covers experiment tracking and model deployment, while leaving production needs such as HTTPS, authentication, environments, validation, and pull-request workflows unfinished.

## Summary
Alon Gubkin builds a small ML platform on AWS during a live coding workshop. He starts with the problems that grow when an organisation has more data scientists or begins putting models into production: data versioning, experiment tracking, deployment, serving, and shared infrastructure. The platform uses DVC for datasets, MLflow for runs and model artifacts, FastAPI for HTTP model servers, Kubernetes on EKS for running services, Pulumi for infrastructure as code, S3 and Postgres for storage, Traefik for routing, and GitHub Actions for deployment. A Cookiecutter template gives data scientists a starting repository. The demo trains an Iris classifier, logs metrics and a model to MLflow, packages a serving API, and deploys it to Kubernetes. Gubkin is clear about what the workshop does not solve, including HTTPS, authentication, staging and production environments, validation, common libraries, and pull-request workflows.

## Key ideas
### An ML platform becomes useful when teams move beyond small experiments
[03:27](https://www.youtube.com/watch?v=s8Jj9gzQ3xA&t=207s)
Gubkin says one or two data scientists can often work without much ML infrastructure. The need changes when an organisation has more data scientists or begins applying models in production. At that point, teams need support across the pipeline, including data versioning, experiment management, deployment, and model monitoring. He describes the combination of these systems as an ML platform and focuses the workshop on how to build that structure yourself rather than adopting a single packaged product.

### Platform design should start with problems and priorities
[04:40](https://www.youtube.com/watch?v=s8Jj9gzQ3xA&t=280s)
The platform diagram includes data collection and versioning, a possible feature store, training orchestration, experiment management, packaging, deployment, and model serving. Gubkin says some of these areas may be irrelevant to a particular organisation, while other local problems may be missing from the diagram. His process is to understand the needs deeply, prioritise them, and choose a solution for each problem. The workshop focuses on data versioning, experiment management, deployment, and serving, using AWS while noting that the approach can work with other cloud providers.

### Tool choices need comparison rather than imitation
[06:11](https://www.youtube.com/watch?v=s8Jj9gzQ3xA&t=371s)
The workshop uses DVC and MLflow, but Gubkin warns against selecting them simply because they appear in the demo. He names Pachyderm, Weights & Biases, Comet ML, and Neptune as alternatives in related categories. He also shows MLOps.toys, an open-source website that compares MLOps tools by area, including model serving tools such as BentoML, Valohai, and KServe. The intended workflow is to compare tools against the team's problems and needs, then change tools later if a better fit appears.

### Kubernetes provides a shared place to run replicated model services
[07:47](https://www.youtube.com/watch?v=s8Jj9gzQ3xA&t=467s)
Gubkin explains Kubernetes by comparing it with an operating system that runs across multiple machines. If a model service runs out of memory or receives too many predictions, more machines can be added. Applications usually run in Docker containers, and replicas can be created. Helm is used as the Kubernetes package manager, so applications such as MLflow can be installed on the cluster. In AWS, the workshop uses EKS as managed Kubernetes, S3 for file storage, RDS with Postgres for the metadata database, and Route 53 for DNS.

### The shared platform separates training infrastructure from model projects
[12:43](https://www.youtube.com/watch?v=s8Jj9gzQ3xA&t=763s)
The architecture has a shared infrastructure repository and a model template repository. A data scientist clones the template, changes the dataset and parameters, and trains experiments. MLflow runs on Kubernetes, stores model files in an S3 artifact bucket, and stores experiment metadata such as metrics and history in Postgres. For serving, each model gets a FastAPI server on Kubernetes. Traefik exposes the services and routes requests such as a URL beginning with "/my_model" to the matching server.

### Pulumi turns cloud infrastructure changes into reviewable code
[17:16](https://www.youtube.com/watch?v=s8Jj9gzQ3xA&t=1036s)
Instead of configuring resources through cloud provider interfaces, Gubkin writes the infrastructure in Pulumi using TypeScript. Pulumi creates the EKS cluster, database, buckets, permissions, Helm installations, routes, and DNS records. Running "Pulumi up" creates or updates the cloud resources, while commenting out a resource removes it on a later run. He says this makes infrastructure changes reviewable in source control and gives the team a history of how the platform changed. Terraform is presented as a comparable alternative.

### MLflow connects training runs to a repeatable serving path
[1:03:12](https://www.youtube.com/watch?v=s8Jj9gzQ3xA&t=3792s)
The model template uses Poetry to define Python dependencies and expose commands such as training. MLflow autologging records the LightGBM run, while custom metrics such as log loss and accuracy are also logged. The trained model is stored in the S3 artifact bucket and can be inspected from the MLflow dashboard. The serving package loads a selected MLflow model, accepts flower measurements through a FastAPI POST endpoint, creates a dataframe, and returns a predicted class. The Docker image then packages the server for Kubernetes.

### The demo uses CI/CD and data versioning, while leaving production work open
[1:27:41](https://www.youtube.com/watch?v=s8Jj9gzQ3xA&t=5261s)
The model repository includes a Makefile and GitHub Actions workflow. A push to the main branch installs dependencies, trains the model, obtains its MLflow run ID, builds the container, and deploys the model with Pulumi. DVC is configured with an S3 remote so the Iris dataset is stored outside Git while a small DVC file is committed to the repository. Gubkin ends by listing unfinished production work: HTTPS, authentication, staging and production environments, shared preprocessing code, model and input validation, and a stronger pull-request workflow.

## Notable quotes
- Alon Gubkin: "When designing your ML platform, it's super important to deeply understand your needs and your problems." (05:01)
- Alon Gubkin: "It's really important not to take these tools for granted and evaluate the alternatives." (06:11)
- Alon Gubkin: "Instead of defining your infrastructure in the UI, we're going to write code that represents that infrastructure." (18:11)
- Alon Gubkin: "When we run the model in production, we don't want to use a constant MLflow run ID. We want it to be dynamic." (1:21:01)
- Alon Gubkin: "We didn't configure HTTPS, which is obviously really important. We didn't configure authentication." (1:33:47)

## Tools & references mentioned
- Aporia
- DVC
- MLflow
- FastAPI
- Pulumi
- Cookiecutter
- Kubernetes
- AWS
- S3
- EKS
- RDS
- Postgres
- Route 53
- Traefik
- Helm
- Docker
- Poetry
- GitHub Actions
- LightGBM
- MLOps.toys
- Pachyderm
- Weights & Biases
- Comet ML
- Neptune
- BentoML
- Valohai
- KServe
- Terraform
- Seldon Core
- KFServing
- Cortex
- Triton Inference Server
- DVC remote

## Who should watch
- You are choosing tools for experiment tracking, data versioning, or model serving and need a concrete comparison-driven starting point.
- Your team wants to connect cloud infrastructure, Kubernetes deployments, MLflow runs, and model repositories into one workflow.
- You want to see how a small platform can be assembled in code, while also understanding which production concerns the demo leaves unresolved.

## Editor's note

Alon Gubkin says an ML platform becomes necessary when an organisation has more data scientists or starts putting models into production, with teams needing support across data versioning, experiment management, deployment, and serving. ZenML records each pipeline run's steps, inputs, outputs, and code version, so teams can trace an artifact or model to the data and code that produced it.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Building an ML Platform from Scratch: Live Coding Session - Part 2](https://mlopstalks.com/talks/building-an-ml-platform-from-scratch-live-coding-session-part-2) (Alon Gubkin, Aporia, 1:12:53)
- [Building an ML Platform: Insights, Community, and Advocacy](https://mlopstalks.com/talks/building-an-ml-platform-insights-community-and-advocacy) (Stephen Batifol, Wolt, 45:49)
- [How to Leverage ML Tooling Ecosystem](https://mlopstalks.com/talks/how-to-leverage-ml-tooling-ecosystem) (Mariya Davydova, Neu.ro, 55:57)
- [Tecton Round-table // Get your ML Application Into Production](https://mlopstalks.com/talks/tecton-round-table-get-your-ml-application-into-production) (Kevin Stumpf, Derek Salama, Eddie Esquivel & Isaac Cameron, Tecton, 55:42)
- [MLOps Engineering Labs Recap, Part 2](https://mlopstalks.com/talks/mlops-engineering-labs-recap-part-2) (Laszlo Sranger & Artem Yushkovsky, Neuro & Paulo Maia, Nilgai, 1:04:16)