Meetup

The Motivation for MLOps

Steven Fines, CoreLogicEpisode 118 · 56:42 · Jan 2023 · 554 viewsHosted by Ben Epstein
Thumbnail for The Motivation for MLOps Watch on YouTube
TL;DR
  1. 1

    MLOps gives teams a shared process for running machine learning in production, including data, model versions, compliance, delivery, and monitoring.

  2. 2

    The number of models in production drives the need for MLOps more than company size, so even a small company may need a framework when it operates many models.

  3. 3

    MLOps and DevOps overlap, but model training, model validation, model surveillance, and data lineage require machine learning-specific ownership and processes.

Summary

Steven Fines explains why machine learning becomes difficult to operate once several models reach production. Teams need to track model and pipeline versions, confirm that data can legally be used, monitor compliance and prediction quality, and decide whether unused models should remain deployed. Bespoke pipelines also create support problems when teams use different languages and operating practices. Fines defines MLOps as a conceptual framework for the processes and tools that move machine learning products into production. The framework can be lightweight for one model and more formal as the model count and dependencies grow. He separates MLOps from DevOps, while describing how the teams should work together. His suggested path starts with version control and CI/CD, then adds repeatable pipelines, delivery platforms, data catalogs, validation, surveillance, and workflow coordination. For adoption, he recommends standards, automation, training, and a low-risk migration approach that moves the surrounding pipeline before changing an existing model.

Key ideas
02:11

Production machine learning creates compliance work that grows across teams

Fines focuses on production rather than model training because operational and compliance problems appear quickly when several models are deployed. Teams must apply compliance monitoring uniformly, determine whether data may legally be used, and account for rules around data handling and automated decisions. He gives credit and healthcare as examples with different burdens. Credit has extensive regulation around automated decisions and discriminatory outcomes, while healthcare focuses heavily on private health data. Compliance work can consume senior staff time because someone must monitor both inputs and outcomes over time. The relevant rules also depend on industry, country, and state or province.

09:08

Version and usage tracking are needed before teams can manage model behavior

A production model is part of a larger chain that includes data acquisition, enrichment, preparation, training, and prediction. Teams need to know which version of each component is running and whether anyone is actually using the deployed model. An unused model may need to be retired, or its traffic may not have been routed correctly. Teams also need model surveillance to detect degrading performance. If accuracy falls and revenue depends on the model, someone will ask why the change was not detected. Without a shared framework, the alternative is assigning people to watch model outputs manually.

15:57

MLOps is a framework whose size should match the number of models

Fines defines MLOps as a conceptual framework for developing processes and tooling that support production machine learning. The implementation depends on requirements, organization, and budget. One model may only need a written, structured process, while an organization running 20, 50, or 200 models may need a supported framework rather than bespoke tooling. Company size is less important than the number of models in production. A two-person company with 50 production models can have a stronger need for MLOps than a much larger company with one model. The framework also helps leaders connect a model to revenue production or cost reduction.

20:50

MLOps overlaps with DevOps while retaining machine learning-specific responsibilities

MLOps and DevOps share parts of their domain, but Fines says they are not clones. DevOps or Cloud Ops should manage infrastructure such as networks and identity access. MLOps should work with model development on training pipelines, model surveillance, and questions about whether accuracy or population stability has changed. CI/CD infrastructure belongs more clearly to DevOps, while project-specific CI/CD can belong to the team building the model. Scattered machine learning teams can also create organizational problems because they may omit compliance or operations discussions. A unified function or clearly documented process makes those boundaries easier to manage.

26:02

Formalization becomes useful after more than a couple of production models

Any organization with machine learning in production is already performing some MLOps function, whether it uses that name or not. Fines recommends writing down how models move into production and having people follow that process. He recommends formalizing it when there are more than a couple of models. The process does not need to constrain every technical choice. It can begin with many options and still give teams a shared path. In his view, this helps organizations deliver products faster and get better results because staff are not spending all their time managing existing pipelines.

27:55

Version control and CI/CD are prerequisites for a workable MLOps process

Before adopting a broader MLOps framework, teams need a revision control system that integrates with their tools. They also need some CI/CD implementation that checks the repository and builds it regularly. Ideally, it runs training and validation tests so a team can tell whether a commit is usable. Fines presents these as basic foundations rather than a fully mature setup. Having CI/CD first reduces the risk of taking on two complicated projects at once, since many MLOps tools already connect to CI/CD systems.

33:13

Scaling requires workflow coordination, validation, surveillance, and lineage

At a larger scale, separate model pipelines may depend on one another and may receive data on different schedules. A workflow engine must coordinate those dependencies and wait for a coherent set of inputs instead of relying on delayed cron jobs or manual triggers. Teams also need automated model validation and a way to supervise models without watching every execution. A data catalog should record available data, lineage, inputs, and outputs. Model surveillance goes beyond checking a validation score. It can identify longer-term changes such as hidden discrimination or a downward trend before existing validation guards fail. A feature store becomes more useful when many teams produce features for another team to turn into products.

41:08

Migration works better when teams move the surrounding pipeline before the model

For existing models, Fines recommends a Strangler approach from the Microsoft Azure Cloud Patterns Library. The team first containerizes the model and moves the supporting data and execution framework into the new MLOps environment. It leaves the model itself unchanged at first. This lowers business risk and makes adoption easier because modelers are not being asked to alter the model immediately. The migration should then connect the model's inputs and outputs to a data catalog, add systematic data validation and surveillance, and build reusable components. Fines also recommends developing standards with team buy-in, training people to follow them, and reviewing the standards regularly as tools and practices change.

"The size of your organization doesn't matter. It's really more the number of models you have in production is what drives your need for an MLOps framework."Steven Fines18:44
Who should watch
  • You are a senior engineer, software architect, or manager deciding whether your organization needs a formal MLOps process.
  • Your team has several models in production and struggles with versions, compliance, data dependencies, or model monitoring.
  • You are moving existing models into a shared platform and need a lower-risk migration plan.