Elle O'Brien says academic research often lacks code review, shared repositories, and frequent checks, while industry software teams may deploy several times a day.
2
Continuous integration for ML must account for data versioning, model compute needs, infrastructure, and useful evaluation reports alongside source code.
3
CML builds on GitHub Actions and GitLab CI to connect data versioning, cloud compute, and human-readable reports to an ML workflow.
Summary
Elle O'Brien describes her move from academic research in computational neuroscience to data science at Iterative. She found a large gap between research coding, which may never receive review, and software development, where teams check in, test, merge, and deploy work frequently. David Aponte and Elle then explain continuous integration and continuous delivery in practical terms. ML adds complications because models depend on data, training can require substantial compute, infrastructure affects reproducibility, and a simple pass or fail signal often does not explain model quality. Elle explains how DVC extends Git workflows to large datasets and how CML uses existing CI systems rather than replacing them. CML can provision cloud machines for training and place plots, tables, and other artifacts in merge request reports. Elle closes with a systems-focused lesson: failures should lead to blameless post-mortems that ask whether the process, automation, or team structure made the mistake likely.
Academic research and production software reward different working habits
Elle O'Brien says the transition from academic research to industry was much bigger than she expected. As a student and scientist, she wrote code, but nobody usually checked it, reviewed it, or tried to reproduce it. At Iterative, people may deploy software several times a day. The workflow includes checking code in, creating branches, discussing changes, merging them, and testing continuously. She misses the freedom to spend a month exploring a research tangent, while also appreciating the feedback and collaboration that come from an organized engineering process.
A statistics background helps Elle judge whether a model is plausible
Elle says her years in research gave her a strong statistical foundation. She describes one benefit as being able to look at a model and think, "this is probably junk," based on experience rather than only following a procedure. She sees research training and production engineering as complementary, although labs often do not reward production-quality code. Her current work focuses more on tools, so she sometimes misses making scientific inferences. David agrees that understanding both the model and how to put it into production is valuable.
Continuous integration means checking in and testing work frequently
David explains continuous integration as the practice of having several people make changes on branches and regularly integrate those changes into a main codebase. The aim is to avoid long-lived branches, large merge conflicts, and surprises after integration. Elle adds that each check-in should also test the code. The DevOps habit is to keep practicing what it would be like to have the change in production, instead of waiting until the end of a project to discover problems.
Continuous delivery makes production feedback available sooner
David defines continuous delivery as keeping the ability to release available at any time and increasing the speed and frequency of releases. Frequent releases can reduce the risk and cost of waiting while problems accumulate. Elle gives an example from a startup where she was the only data scientist. She could create a new model, but deployment depended on another person's queue of work, so the model eventually went stale without being tested in production. Continuous delivery gives data scientists faster feedback about dependencies, behavior, and business value.
Git workflows need extensions for ML data and infrastructure
Elle explains that GitHub, GitLab, and Bitbucket evolved around complex, changing software projects with branches, dependencies, and multiple contributors. ML projects add data and infrastructure dependencies that are harder to track. DVC extends Git versioning to large datasets while keeping the data in storage such as an S3 bucket and storing a link in the repository. With data under source control, dataset changes can trigger CI. Without that link, CI only sees source-code changes.
ML training can exceed the resources available in ordinary CI runners
ML models may require much more memory or compute than a normal GitHub Actions or GitLab CI runner provides. Elle says even scikit-learn work can run out of memory on a laptop, while many applications need GPUs. Teams can use self-hosted runners on premises or in the cloud. CML also includes a use case built with Docker Machine that can start an EC2 instance with the requested GPU, run the training job, and turn the instance off afterward.
Model evaluation needs more than a binary CI result
Elle says standard CI systems often report whether a job passed or failed. That signal is insufficient for many ML decisions. Teams may need detailed metrics, visualizations, and performance checks on important subsets of the data. They also need to inspect which features are being used and whether that is acceptable. CML reports can include PNG files, tables, and other artifacts in a pull or merge request. The approach does not depend on a particular visualization library or on whether the code uses Python or R.
Blameless post-mortems should examine the system around a failure
Elle's main DevOps lesson is to investigate failures without placing all responsibility on the person who made the final mistake. A missed step may have been better automated. A manual process may have imposed too much cognitive load. A handoff may have been delayed by a bottleneck or by competing team priorities. She recommends asking whether people were set up to move quickly and whether the system could have prevented the failure. David connects this to the idea that a person is not the same thing as their code.
"ML is really challenging to, you often need in addition to some really detailed metrics being reported, data visualization helps a lot."Elle O'Brien45:33
Who should watch
You are a data scientist whose models often wait on another team before anyone tests them in a production-like environment.
Your ML repository contains large datasets or expensive training jobs, and ordinary Git-based CI does not handle the data or compute well.
You want a practical introduction to CI/CD without adopting a separate ML platform, and you are considering DVC or CML.
