# MLOps: Isn't That Just DevOps?

Ryan Dawson, Seldon | MLOps Coffee Sessions | Episode 3 | 1:06:32
Hosted by David Aponte

Source: https://www.youtube.com/watch?v=_PV_y4BzQv4
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/mlops-isnt-that-just-devops
Published: 2020-07-20
Tags: build-vs-buy, drift, model-serving, monitoring, platform-teams

## TL;DR
- MLOps applies engineering practices to the full machine learning build, deployment, and monitoring lifecycle, which includes data, training, serving, and changing inputs.
- Machine learning differs from ordinary software because models are produced from data and code, training is often stochastic, and large datasets and serialized models create different build and packaging problems.
- Teams should define their use case and constraints before choosing tools, including data quality, serving mode, retraining, drift, explainability, team skills, and budget.

## Summary
Ryan Dawson explains why MLOps overlaps with DevOps but cannot be reduced to it. Traditional software starts with code and produces an executable or web service. A machine learning project starts with data and a question, then adds data preparation, training, experiment tracking, model packaging, serving, and monitoring. The model can change when data changes, and live data can drift away from the training distribution. Ryan uses salary benchmarking and fashion recommendations to show why a one-off prediction can become a recurring production service. He also describes Seldon's approach to serving models on Kubernetes, including inference graphs and integrations for monitoring and drift detection. The conversation ends with practical scoping questions. Teams should first understand their use case, then decide whether they need real-time or batch predictions, retraining, explainability, or a larger platform. Ryan is careful not to prescribe one stack because the right choice depends on the problem, existing skills, and budget.

## Key ideas
### MLOps covers the whole machine learning lifecycle
[02:48](https://www.youtube.com/watch?v=_PV_y4BzQv4&t=168s)
Ryan defines MLOps as what is needed to make the machine learning build, deploy, and monitor lifecycle smooth and safe. The aim is to let teams focus on machine learning and business value instead of repeatedly solving operational problems. He says these challenges are underestimated when people from traditional DevOps do not see the full machine learning lifecycle. The conversation also separates small projects from larger platforms. A company with a few simple models may stitch together a small set of tools, while a larger company with many models may need a broader platform.

### DevOps practices apply to ML, but treating the systems as identical hides the work
[06:18](https://www.youtube.com/watch?v=_PV_y4BzQv4&t=378s)
David and Ryan agree that machine learning should be treated as an engineering discipline and that many DevOps practices still apply. The problem begins when that overlap is taken to mean the systems are the same. Ryan points to the different starting point, the wider lifecycle, and the range of production use cases. David adds that companies can underestimate the work and spend time building an internal solution without understanding the requirements. Ryan says the industry is still experimenting with standards, and open source collaboration may help those standards form.

### The machine learning build starts with data and a question
[19:06](https://www.youtube.com/watch?v=_PV_y4BzQv4&t=1146s)
Ryan contrasts a conventional application with a salary-benchmarking model. A normal software project can start with a user story, code, packaging, and a CI pipeline. The ML example starts with employee data and a question about using experience and skills to benchmark salaries. The first model may answer a one-time question, but the requirements change when the business wants predictions for later reviews or another department. The team then needs a predictive function, a way to serve the model repeatedly, and a way to check whether the training data applies to the new department.

### Models create different build, packaging, and reproducibility problems
[25:37](https://www.youtube.com/watch?v=_PV_y4BzQv4&t=1537s)
Ryan says the fundamental difference comes from how the system is made. Traditional programs use explicit rules to turn inputs into outputs. Machine learning captures rules indirectly from data, so training is stochastic and may not produce exactly the same result each time. Large datasets may not fit naturally into Git, and a trained model is packaged by serializing it rather than compiling it into an ordinary executable. Training is also a long-running part of the build. The model can perform well on its training data and still fail when live data has a different distribution.

### Serving is only one part of a wider MLOps system
[29:34](https://www.youtube.com/watch?v=_PV_y4BzQv4&t=1774s)
Ryan describes Seldon as a serving solution for models on Kubernetes. A user can provide a model path and the toolkit used to build the model in a Kubernetes custom resource, after which the required lower-level resources are created. Seldon can also use a Docker-based approach when a serialized model is not the right fit. Ryan stresses that serving must be understood within the wider MLOps landscape, which also includes storage, processing, training, tracking, batch predictions, and real-time APIs. The hard part is often choosing what the particular use case actually needs.

### Model rollout requires both traffic engineering and business metrics
[34:37](https://www.youtube.com/watch?v=_PV_y4BzQv4&t=2077s)
Ryan uses an online store to explain model rollout. A control model remains live while two new versions receive smaller portions of traffic. The team can compare them after collecting a statistically significant sample, while limiting risk by sending most traffic to the control. Conversion rate may be enough in one case, but another model could recommend products that increase immediate sales while causing customers to leave. David points out that this combines engineering work, such as routing, traffic splitting, monitoring, and rollback, with scientific decisions about experiments and metrics.

### Inference graphs keep request transformations close to the model
[46:26](https://www.youtube.com/watch?v=_PV_y4BzQv4&t=2786s)
Ryan explains that raw requests often need preprocessing before a model can consume them, especially for text. The same transformation used during training must also happen at inference time. Seldon uses an inference graph to place these steps into a serving pipeline, so a request can pass through preprocessing, the model, and later transformations before the response returns to the consumer. The graph is directed and acyclic, but it can branch. Ryan gives multi-armed bandits as an example where a component chooses between model variations.

### Drift and outliers require monitoring beyond ordinary service health
[44:54](https://www.youtube.com/watch?v=_PV_y4BzQv4&t=2694s)
Ryan defines concept drift as a change in the live data distribution relative to the training data. His fashion example has a model trained during summer that continues recommending T-shirts during winter. Seldon can feed live requests asynchronously to a component that checks whether the data still matches the expected distribution, so the prediction path is not slowed down. A separate outlier detector can identify individual inputs outside the training distribution. Ryan also discusses adversarial examples and explains that a model can be highly confident while being wrong.

### Scoping questions should come before selecting a platform
[54:48](https://www.youtube.com/watch?v=_PV_y4BzQv4&t=3288s)
Ryan advises teams to identify their particular use case before choosing tools. His questions cover whether the data exists and is clean, how often new data arrives, whether training connects to continuous integration, how the model will be served, and whether predictions need low latency. Teams should also consider changing data, seasonal effects, online learning, retraining triggers, outliers, and explainability. Platform teams have additional concerns, including the range of internal use cases, deployment ownership, user skills, familiar tools, budget, and whether to buy a platform or assemble one from existing components.

## Notable quotes
- Ryan Dawson: "MLOps is whatever is necessary to make the whole ML build deploy monitor lifecycle as smooth and as safe as possible." (02:28)
- Ryan Dawson: "With machine learning you really capture rules indirectly from data." (25:58)
- Ryan Dawson: "The ML code is this one little tiny block and then you've got these huge blocks for data cleaning and prep and serving and monitoring stuff." (14:47)
- Ryan Dawson: "Tread carefully because there's a range of use cases and you need to figure out where your use case fits within that space." (55:35)
- David Aponte: "Good teachers give you a framework to work with and allow you to get there on your own." (54:59)

## Tools & references mentioned
- Seldon
- Seldon Core
- Kubernetes
- Jenkins
- Docker
- GitLab
- Bamboo
- Concourse
- scikit-learn
- S3
- MLflow
- LF AI Foundation
- KFServing
- Airflow
- Argo
- Alibi Detect
- MLOps Community
- awesome-mlops
- awesome production machine learning
- Stitch Fix

## Who should watch
- You are coming from a DevOps team and need to understand why deploying and operating machine learning systems adds work beyond a normal application.
- Your team is deciding whether a project needs a few connected tools, a serving system, or a broader internal platform.
- You need a practical set of questions for scoping data quality, serving, retraining, drift, explainability, team skills, and budget before choosing an MLOps approach.

## Editor's note

Ryan Dawson says platform teams must weigh their internal use cases, deployment ownership, user skills, budget, and whether to buy a platform or assemble one. ZenML lets teams write workflows as Python pipelines and choose the orchestrator, artifact store, and other infrastructure through configuration. The same pipeline code can then run on a laptop, Kubernetes, Airflow, Kubeflow, or a cloud service.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Doing MLOps](https://mlopstalks.com/talks/doing-mlops) (Noah Gift, Pragmatic AI Labs, 1:01:22)
- [Operationalize Machine Learning at Scale with MLOps](https://mlopstalks.com/talks/operationalize-machine-learning-at-scale-with-mlops) (Christopher Bergh, DataKitchen, 57:50)
- [The Motivation for MLOps](https://mlopstalks.com/talks/the-motivation-for-mlops) (Steven Fines, CoreLogic, 56:42)
- [MLOps and DevOps, Parallels and Deviations](https://mlopstalks.com/talks/mlops-and-devops-parallels-and-deviations) (Damian Brady, Microsoft, 55:32)
- [Machine Learning Operations: What Is It and Why Do We Need It?](https://mlopstalks.com/talks/machine-learning-operations-what-is-it-and-why-do-we-need-it) (Niklas Kühl, IBM and Karlsruhe Institute of Technology (KIT), 58:47)