Neu.ro combines Kubernetes-based resource orchestration with integrations for tools that cover development, pipelines, deployment, monitoring, and version management.
2
The production workflow uses Airflow, Seldon, Grafana, DVC, TensorBoard, and platform storage to retrain and deploy an object-recognition model as new retail data arrives.
3
Mariya says the system is still incomplete, with manual steps, weak code and model registry workflows, and planned improvements around Git-based CI/CD and reproducible image builds.
Summary
Mariya Davydova walks through Neu.ro's workflow for developing and operating an object-recognition system for retail. New images arrive from shops, so the model needs frequent retraining and redeployment. Neu.ro provides a Kubernetes-based abstraction for clusters, storage, Docker images, and jobs, while open-source tools handle specific parts of the workflow. Cookiecutter creates project scaffolding, Jupyter and TensorBoard support experimentation, Git manages code, Airflow runs pipelines, Seldon handles inference deployment, Grafana supports monitoring, and DVC versions data. The production workflow has separate pipelines for preprocessing data, training and registering models, and deploying models. Mariya is candid that the setup is not ideal. Several operations remain manual, code is copied to storage, and TensorBoard is being used as a model registry. She describes plans for Git-based CI/CD, automated retraining, platform-side image builds, and a better registry. She also explains how Docker and pinned dependency versions reduce environment problems.
The example is a retail object-recognition system with frequent retraining
Neu.ro's example involves recognizing products in photographs taken across different shops and countries. New images arrive every day, and new products appear over time. The team therefore needs to combine incoming data with the existing dataset, retrain the model, and deploy a new version soon after. Mariya says the difficult part is mainly production work rather than the machine learning model itself. The system must keep operating while data and models change. This makes the workflow different from a one-off training project, where experimentation can remain separate from deployment.
Neu.ro hides infrastructure behind four platform concepts
Neu.ro is built on Kubernetes and can run in public cloud, private infrastructure, or hybrid setups. Its platform abstracts the underlying machines through clusters, storage, Docker images, and jobs. A cluster groups compute resources such as CPUs and GPUs. Storage provides a common interface without requiring users to track a particular machine or disk. Jobs run inside containers against a selected resource preset, mounted volumes, and an image. Mariya gives an example of running a training job on a small GPU preset, without choosing the specific machine underneath.
The platform integrates specialist tools instead of trying to rebuild them all
Mariya says customers often want one place for data, models, and pipelines, but building every feature internally would require too much engineering and would leave the product behind as new tools appear. Neu.ro chose integrations with existing tools. The development setup uses Cookiecutter for project scaffolding, Jupyter Notebooks for experiments, TensorBoard for tracking, PyCharm Professional for editing and debugging, Git for code, and platform storage for data and models. A command-line flow and Makefile hide the longer platform commands, volume settings, and other setup details.
Production adds versioning, deployment, load handling, and monitoring
Mariya describes production as the point where the workflow becomes difficult. Teams must track data versions, deploy the model, handle load, and monitor whether model behavior worsens after new data arrives. Neu.ro runs Airflow alongside the platform for pipelines. A custom integration lets Airflow launch jobs on Neu.ro and access its storage. Seldon provides production inference functions such as API serving, load balancing, and metrics collection. Grafana is used for monitoring because it integrates with Seldon and Prometheus. The production setup also uses Helm to store Airflow DAGs and update them when changes are available.
Three pipelines connect data preparation, training, and inference
The example has three main Airflow pipelines. The first accepts a new portion of data and a base dataset, then produces a new dataset split into training, validation, and test data. The second takes a dataset, model code, and base model, retrains the model, and puts the result into a registry. The third takes a model and deploys it for inference. Many pipeline steps run as jobs on the Neu.ro platform. The workflow checks required arguments and invalid inputs, then exposes links to the related tool interfaces. Mariya says the model version, data version, and code version are retained so the result can be reproduced.
The current version uses simple storage workflows with clear limitations
DVC files, caches, and dataset versions are stored on platform storage. Each dataset version has its own directory, and preprocessing jobs use the DVC files to retrieve the matching data. Code is also copied to storage because it changes infrequently in this workflow. Model metadata records which model, dataset, and other inputs produced each result. TensorBoard is used on top of stored logs to compare model metrics, epochs, and loss. Mariya admits that this is not a finished registry design. The team is considering MLflow or another model registry.
The planned workflow is triggered by changes instead of manual runs
The current pipelines are run manually. Neu.ro wants Git and CI/CD to trigger the appropriate work when new data or model code appears. A new dataset would cause retraining, and a new model in the registry would lead to deployment. Production deployment may remain a deliberate manual step, because someone should understand what is being released. Mariya also wants to build Docker images on the platform from Dockerfiles rather than only copying images into a private registry. That would make it easier to recreate the environment used by an older pipeline.
Docker and pinned dependencies reduce Python environment failures
Mariya recommends Docker for separating pipeline steps that need different CUDA or library versions. Each model or step can run in its own container rather than forcing every part of the workflow into one environment. For development dependencies, the team pins exact package versions in requirements files. Dependabot creates pull requests when packages or their dependencies change, allowing tests to run before the update is merged. This gives the team control over when dependency changes enter the project and reduces the risk that a new package release breaks an existing environment.
"You need to train a model, now you need to keep your data somewhere and somehow, you need to update your data sets, you need to train your model, you need to wrap this model into API, you need to host this API, you need to balance the load and monitor the health of the model."Mariya Davydova48:04
Who should watch
You are assembling an ML production workflow from separate open-source tools and want a concrete example of how the pieces can connect.
Your team has a working training project, but manual data versioning, deployment, and monitoring are making production difficult.
You need practical guidance on container isolation and dependency updates for Python-based ML pipelines.
