# How to Leverage ML Tooling Ecosystem

Mariya Davydova, Neu.ro | MLOps Meetup | Episode 26 | 55:57
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=kZ4F48Er__k
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/how-to-leverage-ml-tooling-ecosystem
Published: 2020-07-20
Tags: model-serving, monitoring, orchestration, platform-teams

## TL;DR
- Neu.ro combines Kubernetes-based resource orchestration with integrations for tools that cover development, pipelines, deployment, monitoring, and version management.
- The production workflow uses Airflow, Seldon, Grafana, DVC, TensorBoard, and platform storage to retrain and deploy an object-recognition model as new retail data arrives.
- Mariya says the system is still incomplete, with manual steps, weak code and model registry workflows, and planned improvements around Git-based CI/CD and reproducible image builds.

## Summary
Mariya Davydova walks through Neu.ro's workflow for developing and operating an object-recognition system for retail. New images arrive from shops, so the model needs frequent retraining and redeployment. Neu.ro provides a Kubernetes-based abstraction for clusters, storage, Docker images, and jobs, while open-source tools handle specific parts of the workflow. Cookiecutter creates project scaffolding, Jupyter and TensorBoard support experimentation, Git manages code, Airflow runs pipelines, Seldon handles inference deployment, Grafana supports monitoring, and DVC versions data. The production workflow has separate pipelines for preprocessing data, training and registering models, and deploying models. Mariya is candid that the setup is not ideal. Several operations remain manual, code is copied to storage, and TensorBoard is being used as a model registry. She describes plans for Git-based CI/CD, automated retraining, platform-side image builds, and a better registry. She also explains how Docker and pinned dependency versions reduce environment problems.

## Key ideas
### The example is a retail object-recognition system with frequent retraining
[07:38](https://www.youtube.com/watch?v=kZ4F48Er__k&t=458s)
Neu.ro's example involves recognizing products in photographs taken across different shops and countries. New images arrive every day, and new products appear over time. The team therefore needs to combine incoming data with the existing dataset, retrain the model, and deploy a new version soon after. Mariya says the difficult part is mainly production work rather than the machine learning model itself. The system must keep operating while data and models change. This makes the workflow different from a one-off training project, where experimentation can remain separate from deployment.

### Neu.ro hides infrastructure behind four platform concepts
[09:34](https://www.youtube.com/watch?v=kZ4F48Er__k&t=574s)
Neu.ro is built on Kubernetes and can run in public cloud, private infrastructure, or hybrid setups. Its platform abstracts the underlying machines through clusters, storage, Docker images, and jobs. A cluster groups compute resources such as CPUs and GPUs. Storage provides a common interface without requiring users to track a particular machine or disk. Jobs run inside containers against a selected resource preset, mounted volumes, and an image. Mariya gives an example of running a training job on a small GPU preset, without choosing the specific machine underneath.

### The platform integrates specialist tools instead of trying to rebuild them all
[14:32](https://www.youtube.com/watch?v=kZ4F48Er__k&t=872s)
Mariya says customers often want one place for data, models, and pipelines, but building every feature internally would require too much engineering and would leave the product behind as new tools appear. Neu.ro chose integrations with existing tools. The development setup uses Cookiecutter for project scaffolding, Jupyter Notebooks for experiments, TensorBoard for tracking, PyCharm Professional for editing and debugging, Git for code, and platform storage for data and models. A command-line flow and Makefile hide the longer platform commands, volume settings, and other setup details.

### Production adds versioning, deployment, load handling, and monitoring
[21:36](https://www.youtube.com/watch?v=kZ4F48Er__k&t=1296s)
Mariya describes production as the point where the workflow becomes difficult. Teams must track data versions, deploy the model, handle load, and monitor whether model behavior worsens after new data arrives. Neu.ro runs Airflow alongside the platform for pipelines. A custom integration lets Airflow launch jobs on Neu.ro and access its storage. Seldon provides production inference functions such as API serving, load balancing, and metrics collection. Grafana is used for monitoring because it integrates with Seldon and Prometheus. The production setup also uses Helm to store Airflow DAGs and update them when changes are available.

### Three pipelines connect data preparation, training, and inference
[25:00](https://www.youtube.com/watch?v=kZ4F48Er__k&t=1500s)
The example has three main Airflow pipelines. The first accepts a new portion of data and a base dataset, then produces a new dataset split into training, validation, and test data. The second takes a dataset, model code, and base model, retrains the model, and puts the result into a registry. The third takes a model and deploys it for inference. Many pipeline steps run as jobs on the Neu.ro platform. The workflow checks required arguments and invalid inputs, then exposes links to the related tool interfaces. Mariya says the model version, data version, and code version are retained so the result can be reproduced.

### The current version uses simple storage workflows with clear limitations
[31:30](https://www.youtube.com/watch?v=kZ4F48Er__k&t=1890s)
DVC files, caches, and dataset versions are stored on platform storage. Each dataset version has its own directory, and preprocessing jobs use the DVC files to retrieve the matching data. Code is also copied to storage because it changes infrequently in this workflow. Model metadata records which model, dataset, and other inputs produced each result. TensorBoard is used on top of stored logs to compare model metrics, epochs, and loss. Mariya admits that this is not a finished registry design. The team is considering MLflow or another model registry.

### The planned workflow is triggered by changes instead of manual runs
[38:05](https://www.youtube.com/watch?v=kZ4F48Er__k&t=2285s)
The current pipelines are run manually. Neu.ro wants Git and CI/CD to trigger the appropriate work when new data or model code appears. A new dataset would cause retraining, and a new model in the registry would lead to deployment. Production deployment may remain a deliberate manual step, because someone should understand what is being released. Mariya also wants to build Docker images on the platform from Dockerfiles rather than only copying images into a private registry. That would make it easier to recreate the environment used by an older pipeline.

### Docker and pinned dependencies reduce Python environment failures
[51:24](https://www.youtube.com/watch?v=kZ4F48Er__k&t=3084s)
Mariya recommends Docker for separating pipeline steps that need different CUDA or library versions. Each model or step can run in its own container rather than forcing every part of the workflow into one environment. For development dependencies, the team pins exact package versions in requirements files. Dependabot creates pull requests when packages or their dependencies change, allowing tests to run before the update is merged. This gives the team control over when dependency changes enter the project and reduces the risk that a new package release breaks an existing environment.

## Notable quotes
- Mariya Davydova: "The problem here is mostly in the production rather than in the machine learning part." (08:32)
- Mariya Davydova: "When you come to production you have to make it profitable for your business." (34:34)
- Mariya Davydova: "We realized that we clearly are not capable of doing everything by ourselves." (15:34)
- Mariya Davydova: "You need to train a model, now you need to keep your data somewhere and somehow, you need to update your data sets, you need to train your model, you need to wrap this model into API, you need to host this API, you need to balance the load and monitor the health of the model." (48:04)
- Mariya Davydova: "Docker just saves your life." (51:24)

## Tools & references mentioned
- Neu.ro
- Kubernetes
- Docker
- Cookiecutter
- Jupyter Notebooks
- TensorBoard
- PyCharm Professional
- Git
- DVC
- Airflow
- Seldon
- Grafana
- Prometheus
- Helm
- MLflow
- Dependabot
- Google Cloud
- AWS

## Who should watch
- You are assembling an ML production workflow from separate open-source tools and want a concrete example of how the pieces can connect.
- Your team has a working training project, but manual data versioning, deployment, and monitoring are making production difficult.
- You need practical guidance on container isolation and dependency updates for Python-based ML pipelines.

## Editor's note

Mariya Davydova says the difficult part of this retail workflow is production work, with manual steps and weak model registry workflows making retraining and deployment harder to reproduce. ZenML records each run's steps, inputs, outputs, and code version, so a model can be traced back to the data and code that produced it. 

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Building an ML Platform: Insights, Community, and Advocacy](https://mlopstalks.com/talks/building-an-ml-platform-insights-community-and-advocacy) (Stephen Batifol, Wolt, 45:49)
- [DevOps, Security, and Observability in ML](https://mlopstalks.com/talks/devops-security-and-observability-in-ml) (Luke Marsden, MLOps Consulting, 32:46)
- [Building an ML Platform from Scratch: Live Coding Session](https://mlopstalks.com/talks/building-an-ml-platform-from-scratch-live-coding-session) (Alon Gubkin, Aporia, 1:57:24)
- [Building an ML Platform from Scratch: Live Coding Session - Part 2](https://mlopstalks.com/talks/building-an-ml-platform-from-scratch-live-coding-session-part-2) (Alon Gubkin, Aporia, 1:12:53)
- [The Shipyard: Lessons Learned While Building an ML Platform](https://mlopstalks.com/talks/the-shipyard-lessons-learned-while-building-an-ml-platform) (Joseph Haaga, Interos, 40:00)