An ML platform should be designed around an organisation's specific problems, with each tool evaluated against its alternatives.
2
The workshop combines DVC, MLflow, FastAPI, Kubernetes, Pulumi, AWS services, Traefik, Poetry, Docker, GitHub Actions, Cookiecutter, and S3 to create a working platform.
3
The demo covers experiment tracking and model deployment, while leaving production needs such as HTTPS, authentication, environments, validation, and pull-request workflows unfinished.
Summary
Alon Gubkin builds a small ML platform on AWS during a live coding workshop. He starts with the problems that grow when an organisation has more data scientists or begins putting models into production: data versioning, experiment tracking, deployment, serving, and shared infrastructure. The platform uses DVC for datasets, MLflow for runs and model artifacts, FastAPI for HTTP model servers, Kubernetes on EKS for running services, Pulumi for infrastructure as code, S3 and Postgres for storage, Traefik for routing, and GitHub Actions for deployment. A Cookiecutter template gives data scientists a starting repository. The demo trains an Iris classifier, logs metrics and a model to MLflow, packages a serving API, and deploys it to Kubernetes. Gubkin is clear about what the workshop does not solve, including HTTPS, authentication, staging and production environments, validation, common libraries, and pull-request workflows.
An ML platform becomes useful when teams move beyond small experiments
Gubkin says one or two data scientists can often work without much ML infrastructure. The need changes when an organisation has more data scientists or begins applying models in production. At that point, teams need support across the pipeline, including data versioning, experiment management, deployment, and model monitoring. He describes the combination of these systems as an ML platform and focuses the workshop on how to build that structure yourself rather than adopting a single packaged product.
Platform design should start with problems and priorities
The platform diagram includes data collection and versioning, a possible feature store, training orchestration, experiment management, packaging, deployment, and model serving. Gubkin says some of these areas may be irrelevant to a particular organisation, while other local problems may be missing from the diagram. His process is to understand the needs deeply, prioritise them, and choose a solution for each problem. The workshop focuses on data versioning, experiment management, deployment, and serving, using AWS while noting that the approach can work with other cloud providers.
Tool choices need comparison rather than imitation
The workshop uses DVC and MLflow, but Gubkin warns against selecting them simply because they appear in the demo. He names Pachyderm, Weights & Biases, Comet ML, and Neptune as alternatives in related categories. He also shows MLOps.toys, an open-source website that compares MLOps tools by area, including model serving tools such as BentoML, Valohai, and KServe. The intended workflow is to compare tools against the team's problems and needs, then change tools later if a better fit appears.
Kubernetes provides a shared place to run replicated model services
Gubkin explains Kubernetes by comparing it with an operating system that runs across multiple machines. If a model service runs out of memory or receives too many predictions, more machines can be added. Applications usually run in Docker containers, and replicas can be created. Helm is used as the Kubernetes package manager, so applications such as MLflow can be installed on the cluster. In AWS, the workshop uses EKS as managed Kubernetes, S3 for file storage, RDS with Postgres for the metadata database, and Route 53 for DNS.
The shared platform separates training infrastructure from model projects
The architecture has a shared infrastructure repository and a model template repository. A data scientist clones the template, changes the dataset and parameters, and trains experiments. MLflow runs on Kubernetes, stores model files in an S3 artifact bucket, and stores experiment metadata such as metrics and history in Postgres. For serving, each model gets a FastAPI server on Kubernetes. Traefik exposes the services and routes requests such as a URL beginning with "/my_model" to the matching server.
Pulumi turns cloud infrastructure changes into reviewable code
Instead of configuring resources through cloud provider interfaces, Gubkin writes the infrastructure in Pulumi using TypeScript. Pulumi creates the EKS cluster, database, buckets, permissions, Helm installations, routes, and DNS records. Running "Pulumi up" creates or updates the cloud resources, while commenting out a resource removes it on a later run. He says this makes infrastructure changes reviewable in source control and gives the team a history of how the platform changed. Terraform is presented as a comparable alternative.
MLflow connects training runs to a repeatable serving path
The model template uses Poetry to define Python dependencies and expose commands such as training. MLflow autologging records the LightGBM run, while custom metrics such as log loss and accuracy are also logged. The trained model is stored in the S3 artifact bucket and can be inspected from the MLflow dashboard. The serving package loads a selected MLflow model, accepts flower measurements through a FastAPI POST endpoint, creates a dataframe, and returns a predicted class. The Docker image then packages the server for Kubernetes.
The demo uses CI/CD and data versioning, while leaving production work open
The model repository includes a Makefile and GitHub Actions workflow. A push to the main branch installs dependencies, trains the model, obtains its MLflow run ID, builds the container, and deploys the model with Pulumi. DVC is configured with an S3 remote so the Iris dataset is stored outside Git while a small DVC file is committed to the repository. Gubkin ends by listing unfinished production work: HTTPS, authentication, staging and production environments, shared preprocessing code, model and input validation, and a stronger pull-request workflow.
