# Reproducible experiments

A pack of 6 sessions from the MLOps Community YouTube channel, in the order to watch them. 4h 58m of video.
Page: https://mlopstalks.com/packs/reproducible-experiments

The score is in a spreadsheet, the model is in a bucket, and nobody can identify the data or environment that produced either. Rerunning the latest code does not recover the experiment when its inputs have changed. Start with the records people lose, then build a checklist for preserving code, data and execution conditions together. Data versioning gives a run an identifiable input; experiment tracking connects that input to parameters, outputs and comparisons. Notebook execution state exposes another source of missing context. Finish by asking what kind of reproduction you actually need, including the dependencies, runtime and randomness required to investigate a failure. These historical implementations show different parts of the record. No single dashboard supplies all of them merely by being installed.

## This pack is for you if

- You have an experiment's score but cannot recover its exact dataset or parameters.
- A colleague reran your code and got a different result after dependencies or source data changed.
- A promising model is lost among personal notes, copied datasets and poorly named runs.

## The talks, in order

### 1. Common Mistakes in the ML Development Lifecycle

Kseniia Melnikova, SoftwareOne | 54:32 | MLOps Meetup
Video: https://www.youtube.com/watch?v=OSqb4pmzaWI
Summary: https://mlopstalks.com/talks/common-mistakes-in-the-ml-development-lifecycle.md

Why first: Paper results and private dataset copies turn useful experiments into work other people must repeat. Melnikova's Samsung account gives the loss a concrete shape, including transcription errors and inaccessible records. Establish what needs to survive before deciding how much tracking infrastructure to add.

### 2. How to Make Your Data Science Reproducible (and Why You Should Care)

Ciro Greco, Bauplan | 11:59 | MLOps Community
Video: https://www.youtube.com/watch?v=geTJ7AAImmE
Summary: https://mlopstalks.com/talks/how-to-make-your-data-science-reproducible-and-why-you-should-care.md

Why here: A shared record needs to describe the conditions of the run, not just its score. Greco lays out code, data, environment and sometimes hardware, then separates their versioning responsibilities. That checklist guides the next two talks on preserving large inputs and organizing the resulting experiment history.

### 3. DataOps and Data Versioning in ML

Dmitry Petrov, Iterative.AI | 1:01:55 | MLOps Meetup
Video: https://www.youtube.com/watch?v=ojV1tK9jXH8
Summary: https://mlopstalks.com/talks/dataops-and-data-versioning-in-ml.md

Why here: Large datasets can stay in existing storage while versioned pointers identify what a run used. Petrov's DVC account connects those artifacts to the commands and versions that produced a model. This fills in the data part of the checklist without pretending that a code commit also freezes a bucket's contents.

### 4. Making MLflow

Corey Zumar, Databricks | 59:11 | MLOps Coffee Sessions
Video: https://www.youtube.com/watch?v=odEWCeYPZkU
Summary: https://mlopstalks.com/talks/making-mlflow.md

Why here: Recoverable files still need an intelligible experiment history. Zumar describes tracking parameters, artifacts, source and environments, then admits that thousands of logged models can remain difficult to navigate. His 2022 account makes naming and retrieval conventions part of reproducibility, rather than assuming that logging every run finishes the job.

### 5. The SAME Project: A Cloud Native Approach to Reproducible ML

David Aronchick, Microsoft | 53:47 | MLOps Meetup
Video: https://www.youtube.com/watch?v=rIgvT7ohDR0
Summary: https://mlopstalks.com/talks/the-same-project-a-cloud-native-approach-to-reproducible-ml.md

Why here: Notebook state can be missing even when metrics and artifacts were saved. Aronchick's 2021 SAME proof of concept records run parameters and the variables entering and leaving tagged steps. Its limited scope matters: use the demonstration to ask what execution context you need, without assuming every notebook can be captured or translated automatically.

### 6. Human-centric ML Infrastructure: A Netflix Original

Savin Goyal, Netflix | 56:07 | MLOps Meetup
Video: https://www.youtube.com/watch?v=TzRNZO2E-eM
Summary: https://mlopstalks.com/talks/human-centric-ml-infrastructure-a-netflix-original.md

Why last: Reproduction has a purpose and a cost. Goyal asks what it takes to recreate a production failure, including dependencies, runtime and randomness, then connects that record to a usable debugging experience. Finish by choosing the guarantees your investigation needs and checking that the captured evidence can actually support them.

## Editor's note

Greco's checklist connects code, data and execution conditions; Melnikova shows what is lost when experiment records stay private. ZenML records pipeline steps, inputs, outputs and code versions so a model or artifact can be traced to its producing run. That provides a shared history for investigation while the team decides which execution conditions must also be preserved.

Written by the MLOps Talks editors (the ZenML team), not by any of the speakers.