# DataOps and Data Versioning in ML

Dmitry Petrov, Iterative.AI | MLOps Meetup | Episode 19 | 1:01:55
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=ojV1tK9jXH8
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/dataops-and-data-versioning-in-ml
Published: 2020-06-22
Tags: data-engineering, experiment-tracking, model-registry

## TL;DR
- Dmitry Petrov argues that data operations are a core part of any ML platform and need a common protocol across training, deployment, and experiment tools.
- DVC uses Git to version data, models, metrics, plots, and pipeline definitions while storing large files in existing storage such as S3 or Google Cloud Storage.
- Teams should delay heavy tooling when they are small, then introduce automation and shared workflows as multiple people and teams create coordination problems.

## Summary
Dmitry Petrov explains why machine learning needs a versioning system for data alongside code versioning. Datasets can be too large for traditional systems, and teams usually want to keep them in existing cloud or file storage rather than create a separate storage service. DVC addresses this by storing metadata and pointers in Git while the actual data remains in the team's chosen storage. Petrov describes uses ranging from keeping dataset versions with a model, to tracking metrics and plots, to defining lightweight pipelines and supporting CI/CD, deployment, and model registry workflows. He argues that DVC should fit into an open ecosystem where teams choose separate tools for training, visualization, and deployment. The talk also covers adoption and team size. Small teams should keep flexibility and avoid imposing tools too early. As teams grow, shared processes, automation, and lineage become more useful. Petrov is honest that DVC's purpose can be hard to understand until people try it.

## Key ideas
### An open ecosystem needs a shared data layer
[06:58](https://www.youtube.com/watch?v=ojV1tK9jXH8&t=418s)
Petrov says a machine learning platform should be built from tools that can be selected for different jobs. One tool might handle training, another experiment tracking or metric visualization, and another deployment. He does not expect one open-source project to cover every need. Data operations connect these pieces because models, metrics, datasets, and deployments all depend on knowing which artifacts were used. In his view, a common protocol for data management is needed so teams can combine tools instead of adopting a closed end-to-end platform.

### Data management blocks broader ML platform work
[08:42](https://www.youtube.com/watch?v=ojV1tK9jXH8&t=522s)
While experimenting with resource orchestration, Petrov found that data management appeared before the harder platform problems could be solved. Data had to move from his machine to a cloud instance and then reach teammates working on the same project. Until that movement and versioning problem was handled, it was difficult to build a useful AI platform. He describes data operations as a fundamental part of MLOps, whether a platform makes that dependency visible or hides it. A unified approach would help teams understand what is happening across their ML platform.

### DVC keeps metadata in Git and large files in existing storage
[15:48](https://www.youtube.com/watch?v=ojV1tK9jXH8&t=948s)
ML datasets often range from megabytes and gigabytes to hundreds of gigabytes or terabytes, which makes ordinary version control a poor fit. Teams also want to use storage they already have, such as S3, Google Cloud Storage, Azure file services, or a file server. DVC therefore does not create a separate backend storage system. It creates metadata files and pointers that describe the data, pipelines, models, metrics, and plots, while Git versions those files and the actual artifacts remain in the chosen storage.

### DVC covers dataset history, experiments, and shared caches
[18:56](https://www.youtube.com/watch?v=ojV1tK9jXH8&t=1136s)
Petrov describes several ways people use DVC. A team can record changing datasets and recover the version used to build a particular model. Users can attach summary metrics to source data and metrics or plots to models, then connect the pieces through lightweight pipeline definitions. Larger teams can also use a shared cache when several data scientists work on the same machine or GPU instance. That avoids copying very large datasets for every user while allowing each account to work with the shared data.

### DVC connects versioned artifacts to automation and deployment
[23:34](https://www.youtube.com/watch?v=ojV1tK9jXH8&t=1414s)
In automated workflows, DVC can bring a specific dataset into a CI/CD training job, let the job train a model, and push the resulting model back to storage. It can also support deployment by allowing a system to request the latest model, a model from a branch, or a particular commit without directly handling the underlying S3 bucket or file storage. Petrov presents DVC as a protocol for moving versioned artifacts through a workflow. The same approach can support model registry use and reproducible training.

### Simple interfaces help separate tools work together
[41:13](https://www.youtube.com/watch?v=ojV1tK9jXH8&t=2473s)
Petrov sees tool dependencies as one of the largest problems in ML operations. Teams may combine different languages, data-processing systems, databases, frameworks, and deployment tools, each with its own API. His proposed way to reduce the burden is to agree on a simple workflow and clean workspace. If a tool can read files and write files, it can fit into a DVC pipeline. He compares DVC's approach to infrastructure codification with Terraform, where the system is described in files rather than hidden inside a service.

### Tooling should follow team growth and pain
[46:30](https://www.youtube.com/watch?v=ojV1tK9jXH8&t=2790s)
Petrov advises teams not to impose tools when they are building their first model or data product. A team of two or three can often work with flexibility, and strict processes may slow experimentation. Once the team grows beyond that point, coordination becomes harder. Multiple teams create a stronger need for tools, automation, and shared workflows. He also warns against waiting until a failure or lost revenue forces the decision. Teams need to balance flexibility with enough planning to avoid predictable operational problems.

### Lineage connects a model to its data and pipeline
[55:00](https://www.youtube.com/watch?v=ojV1tK9jXH8&t=3300s)
DVC can act as a lightweight model registry by letting users request a particular model version through a Git repository while the model artifact stays in external storage. For more mature processes, the registry should answer how the model was produced, which source data was used, and which version of that data was involved. DVC pipelines can preserve the commands, files, and versions that produced an artifact. Petrov says this lineage becomes especially important in settings where teams must explain how a model was built.

## Notable quotes
- Dmitry Petrov: "My fundamental belief is you need this part to be more unified, you need kind of a common protocol which works with one AI platform, with one deployment system, with one experiment orchestration or visualization system." (10:56)
- Dmitry Petrov: "What DVC does, it creates metafiles, it codifies your data, it saves the pointers to your data, it saves the information about the pipeline." (34:46)
- Dmitry Petrov: "When you build your first model, when you build your first data product, just don't worry about the tools at all." (46:30)
- Dmitry Petrov: "The most important part is just to think about data management as a special activity and don't undervalue those activities." (49:32)
- Dmitry Petrov: "We are not replacing your system, we just augment the system by data, pipelines, and model registries." (1:00:07)

## Tools & references mentioned
- DVC
- Git
- S3
- Google Cloud Storage
- Azure
- Terraform
- Python
- R
- Spark
- Scala
- Jupyter notebooks
- CI/CD
- Iterative.AI

## Who should watch
- You are building ML workflows and need to track which dataset produced a model.
- Your team stores large datasets in cloud or file storage and wants Git-based collaboration without moving those files into Git.
- You are deciding when to introduce ML tooling and want guidance on the trade-off between fast experimentation and shared automation.

## Editor's note

Dmitry Petrov says mature ML workflows need to show how a model was produced and which version of the data it used. ZenML records each run's steps, inputs, outputs, and code version, so a model or artifact can be traced back to the data and code that produced it. That gives teams the lineage Petrov describes.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- ['Git for Data' - Who, What, How and Why?](https://mlopstalks.com/talks/git-for-data-who-what-how-and-why) (Luke Feeney & Gavin Mendel-Gleason, TerminusDB, 57:44)
- [Applying DevOps Practices in Data and ML Engineering](https://mlopstalks.com/talks/applying-devops-practices-in-data-and-ml-engineering) (Antoni Ivanov, VMWare, 1:04:43)
- [Databricks Model Serving V2](https://mlopstalks.com/talks/databricks-model-serving-v2) (Rafael Pierre, Databricks, 43:17)
- [The Future of ML and Data Platforms](https://mlopstalks.com/talks/the-future-of-ml-and-data-platforms) (Michael Del Balso, Tecton, 55:17)
- [MLOps and DevOps, Parallels and Deviations](https://mlopstalks.com/talks/mlops-and-devops-parallels-and-deviations) (Damian Brady, Microsoft, 55:32)