# The SAME Project: A Cloud Native Approach to Reproducible ML

David Aronchick, Microsoft | MLOps Meetup | Episode 73 | 53:47
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=rIgvT7ohDR0
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/the-same-project-a-cloud-native-approach-to-reproducible-ml
Published: 2021-07-30
Tags: debugging, notebooks, orchestration

## TL;DR
- The SAME project lets data scientists move notebooks toward production without rewriting them into a separate application.
- SAME uses a manifest, CLI, notebook tags, and adapters to split notebook work across cloud resources and workflow platforms.
- The project focuses on straightforward notebooks that need reproducible execution, while David is honest that it will not cover every workload.

## Summary
David Aronchick presents SAME, the Self-Assembling Machine Learning Environment, as a way to move exploratory notebooks into reliable production workflows. He argues that forcing data scientists to rewrite notebooks creates a gap between the original work and the deployed system. It can lose intent, repeat translation work, and introduce drift. Running a notebook unchanged in one large container also misses cloud workflow features such as parallel execution and separate resources for validation, training, and deployment. SAME adds an abstraction layer around existing tools. A short manifest points to a notebook and its run parameters, while a CLI and platform adapters convert it for execution on systems such as Kubeflow and Azure Machine Learning. Notebook tags can divide work into steps, with variables and context passed between them. SAME also captures execution state for debugging. The project was an early proof of concept, with plans for more environments, parameterization, logging, mocking, and dataset handling.

## Key ideas
### Rewriting notebooks for production creates drift and loses the author's intent
[04:09](https://www.youtube.com/watch?v=rIgvT7ohDR0&t=249s)
David describes the common process of taking an exploratory notebook and translating it into another production form. Teams may rewrite it as a Python file, container, or other application, but the data scientist may not be available to maintain that translation. A later model change requires repeating the work. Small choices can also disappear. He gives the example of a data scientist multiplying a value by negative 0.99 for a reason that is not obvious to someone downstream. If that person removes the step, the deployed result changes and nobody knows why. David says this translation gap is a standard process, but he does not support it as the default.

### Notebooks are where much data science happens, but they need production execution
[09:33](https://www.youtube.com/watch?v=rIgvT7ohDR0&t=573s)
David calls Jupyter notebooks the center of gravity for data science. A notebook includes a file format, an IDE, a kernel, and supporting tools, so it is more than a source file. He points to public GitHub and LinkedIn data as evidence of broad use, while acknowledging that the figures are imperfect. Notebooks still need to run outside a laptop when teams need production data, security controls, or more compute. He gives the example of using a cloud machine with eight GPUs instead of a laptop with one GPU. Production systems often expect a Python file or container, which forces rewriting and can omit local settings.

### SAME wraps existing cloud and workflow tools instead of replacing them
[14:35](https://www.youtube.com/watch?v=rIgvT7ohDR0&t=875s)
SAME is intended to help notebook developers build a reliable workflow from the environment they already use. David says he is reusing existing tools for serialization and fan-out rather than asking data scientists to learn every underlying framework. The project adds an SDK, CLI, and adapters between authoring environments and execution platforms. The intended goals are portable notebooks, easier environment capture, platform portability, less boilerplate, and declarative execution. It is designed to work with managed services such as Azure Machine Learning, SageMaker, Google Cloud, and hosted Databricks. The proof of concept demonstrated Kubeflow and Azure Machine Learning because those were the platforms David could access.

### A manifest can turn a notebook run into a reproducible workflow
[23:36](https://www.youtube.com/watch?v=rIgvT7ohDR0&t=1416s)
In the demo, David starts with a messy notebook containing hard-coded URLs, scattered imports, functions in different locations, plots, tables, and other output. He creates a SAME file of about 11 lines. It contains metadata with a name and version, a pipeline reference to the notebook, and a run section containing the parameters needed for execution. The file can be checked into code. The CLI reads it and converts the notebook into a format compatible with Kubeflow. SAME adds run information, gathers imports and packages, and executes the notebook with the declared parameters. The data scientist does not have to manually strip notebook output or rewrite the notebook first.

### Notebook tags split work into steps and preserve execution context
[28:59](https://www.youtube.com/watch?v=rIgvT7ohDR0&t=1739s)
David adds SAME step tags to the notebook and reruns the same manifest without changing it. The content between the tags becomes separate execution steps. SAME collects the imports and packages for each section, creates multiple steps, and passes context between them. The demo changes a single notebook execution into three serial steps. David says the proof of concept does not yet build complex dependency graphs, but variable awareness and function handling work automatically. This lets a notebook use separate resources for different parts of the workflow. Data validation might need machines with lots of memory, model building might need GPUs, and rollout might need no dedicated machine.

### Captured variable state can help diagnose failures in cloud runs
[30:22](https://www.youtube.com/watch?v=rIgvT7ohDR0&t=1822s)
SAME captures the notebook context at the start and end of each tagged step. David serializes Python's local and global variable dictionaries, then compares them to show what changed. His demo displays each step's inputs and outputs, including when a variable was undefined, when it became defined, and how its value changed. This works without relying on print statements. He distinguishes this low-level execution context from tools that record model metrics, such as Weights & Biases and MLMD. In his view, metrics belong in those systems, while variable-level state helps explain why a CI/CD run failed or why a later step received an unexpected value. He imagines sending this data to a standard observability store.

### SAME is aimed first at straightforward notebooks and remains an early proof of concept
[44:55](https://www.youtube.com/watch?v=rIgvT7ohDR0&t=2695s)
David says the initial release should address a limited portion of notebook workloads. He describes a strong fit as a notebook that needs to move off a laptop, has relatively straightforward execution, and has limited external file imports. He estimates that about 75 percent of the public notebooks he examined were direct hits because they were monolithic and used no control flow. He also says the project is unlikely to run the next major language model without substantial hand coding. Planned work includes environment-specific datasets, output logging, mocking external systems, parameterization, external file downloads, and more execution environments. The binary was available, while the project was still working toward opening the code and accepting broader contributions.

### Important models should not depend on a single developer's laptop
[46:50](https://www.youtube.com/watch?v=rIgvT7ohDR0&t=2810s)
David is firm that a model which matters to a business should not remain on a local machine. A laptop can die, its owner can leave, training can stop without anyone noticing, and the team may lack automated tests or drift analysis. He does not prescribe a large Kubernetes cluster for every case. His recommendation is to check the work into Git and push it to some automated execution location through MLOps or GitOps. SAME is intended to give notebook developers a path to that kind of execution without requiring them to become specialists in distributed computing. He also says teams should ignore the problem when a model genuinely does not matter to their use case.

## Notable quotes
- David Aronchick: "The core of the issue is first we need to recognize that these notebooks do need to be run in some production environment." (11:23)
- David Aronchick: "Our goal here is let's let you break this up in intelligent ways and reuse just the resources you require." (19:58)
- David Aronchick: "All they did was just write their notebooks and add these tags and we did the rest for them." (32:21)
- David Aronchick: "If you are betting your business on it, if it is sitting on your laptop you are not if but when you will have a nightmare scenario." (46:42)

## Tools & references mentioned
- SAME
- Self-Assembling Machine Learning Environment
- Microsoft
- MLOps Community
- Jupyter notebooks
- GitHub
- LinkedIn
- Dask
- Kubeflow
- Azure Machine Learning
- SageMaker
- Google Cloud
- Databricks
- Weights & Biases
- MLMD
- TensorBoard
- PaperMill
- JupyterX
- nbconvert
- fast.ai
- MLflow
- Honeycomb
- GitOps
- MLOps

## Who should watch
- Your data scientists work in notebooks, and another team rewrites those notebooks before deployment.
- You need to run notebook workflows with different resources for data preparation, training, and rollout.
- Your team wants more execution context when a cloud or CI/CD run fails, without adding extensive debugging code to every notebook.

## Editor's note

David Aronchick argues that rewriting exploratory notebooks for production can lose the author's intent and create drift when the original data scientist is no longer maintaining the translation. ZenML lets teams write workflows as Python steps and records each run's inputs, outputs, code version, and lineage, so the resulting model or artifact can be traced back to the work that produced it.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Doing MLOps](https://mlopstalks.com/talks/doing-mlops) (Noah Gift, Pragmatic AI Labs, 1:01:22)
- [From Notebooks to Production FASTER](https://mlopstalks.com/talks/from-notebooks-to-production-faster) (Shahd Alghrsi, Virgin Media, 13:43)
- [Operationalizing Machine Learning at a Large Financial Institution](https://mlopstalks.com/talks/operationalizing-machine-learning-at-a-large-financial-institution) (Daniel Stahl, Regions Bank, 1:04:58)
- [Human-centric ML Infrastructure: A Netflix Original](https://mlopstalks.com/talks/human-centric-ml-infrastructure-a-netflix-original) (Savin Goyal, Netflix, 56:07)
- [Challenges Operationalizing ML (And Some Solutions)](https://mlopstalks.com/talks/challenges-operationalizing-ml-and-some-solutions) (Nathan Ryan Frank, WW Grainger, 52:28)