# The State of Production Machine Learning in 2024

Alejandro Saucedo, Zalando SE | AI in Production 2024 | 33:13

Source: https://www.youtube.com/watch?v=6s9Y5fgP3dg
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/the-state-of-production-machine-learning-in-2024
Published: 2024-04-04
Tags: governance, monitoring, observability, security

## TL;DR
- A machine learning model's lifecycle begins when it reaches production, because its data, performance, and operating context can change after training.
- Production machine learning combines data, software, infrastructure, compliance, security, and domain expertise, so teams need systems that cover the full lifecycle.
- Organizations should adapt governance, team structure, and operational controls to the risk and scale of each use case instead of copying software processes unchanged.

## Summary
Alejandro Saucedo describes production machine learning as a system that begins, rather than ends, when a model is trained and deployed. Production adds specialized hardware, complex data flows, compliance, reproducibility, bias, personal data, and security concerns. He maps the system from training data and experimentation through model artifacts, deployment, inference, metadata, monitoring, and feedback. The talk covers the move from model-centric work to data-centric machine learning systems, along with the need for metadata interoperability across experiments, labels, datasets, deployments, and data products. Saucedo also discusses monitoring through performance metrics, slice-level analysis, explainability, alerts, SLOs, progressive rollouts, drift, and outlier detection. At the organizational level, he argues that machine learning needs its own development lifecycle, closer links between MLOps and DataOps, product-oriented teams, and staffing that grows with system complexity. He ends by arguing for proportionate risk and remembering that AI is not always the right solution.

## Key ideas
### Production is where the model lifecycle begins
[02:08](https://www.youtube.com/watch?v=6s9Y5fgP3dg&t=128s)
Saucedo argues that training is not the end of a model's lifecycle. Once a model is consumed in production, it can begin to decay and needs new operational controls. Production systems must account for specialized hardware such as GPUs, complex training and inference data flows, compliance requirements, and reproducibility. They also carry risks from bias, outages caused by changing contexts, personal data, and cybersecurity. A bad solution can have a worse impact than having no solution. Large language models make these issues easier to see because they combine interactive data flows with many system components.

### Responsible AI needs infrastructure and organizational accountability
[04:51](https://www.youtube.com/watch?v=6s9Y5fgP3dg&t=291s)
Saucedo connects AI ethics, regulation, standards, and engineering practice. He says that principles and roundtables are not enough if the underlying software infrastructure cannot enforce them. Responsibility cannot sit with one software engineer or data scientist. Teams need cross-functional skills and domain experts at relevant decision points, while organizations need governance structures and aligned objectives. He also describes responsible AI frameworks and open-source tools as part of the practical infrastructure for developing, deploying, and operating machine learning systems.

### A production ML system connects data, artifacts, serving, and feedback
[08:45](https://www.youtube.com/watch?v=6s9Y5fgP3dg&t=525s)
The anatomy of production machine learning starts with training data, model artifacts, and inference data. Teams experiment through notebooks or distributed systems, turn training data into model artifacts, and then deploy those artifacts as real-time or batch models. Production adds safeguards such as drift detection and explainability. Inference data should feed back into the system for later use. Metadata flows across all these stages, linking experimentation, deployment, models, data, and operational behavior.

### Organizations are choosing between composable stacks and end-to-end platforms
[10:02](https://www.youtube.com/watch?v=6s9Y5fgP3dg&t=602s)
Saucedo presents the MyMLOps project as a way to reason about the relationship between architecture and the many available MLOps tools. Its architectural components include experimentation, runtime engines, and code versioning, with trade-offs between different tool choices. Organizations are deciding whether to assemble heterogeneous, best-of-breed open-source components or adopt an end-to-end provider. The point is to make those choices explicit instead of treating the tooling ecosystem as one undifferentiated collection.

### Production systems require relationships beyond model artifacts
[11:00](https://www.youtube.com/watch?v=6s9Y5fgP3dg&t=660s)
A model-centric view tracks datasets that produce model artifacts. A production ML view also tracks where models are deployed, which environments host them, and how pipelines combine models from different contexts. This creates a different metadata problem. Organizations need interoperability across training and inference data, model metadata, experiments, deployment artifacts, labels, datasets, and data products. Saucedo presents this as part of the move from managing models to managing complete machine learning systems.

### Monitoring must turn system behavior into action
[14:07](https://www.youtube.com/watch?v=6s9Y5fgP3dg&t=847s)
Monitoring includes accuracy, precision, recall, statistical performance, aggregate analysis, slicing production behavior, and explainability. Saucedo says teams should build observability into the system instead of relying only on people opening dashboards. Useful controls include alerts, automated SLOs, progressive rollouts, GPU usage checks, drift detection, and outlier detection. SLOs can specify request rate, throughput, and other operating limits. ML systems also need metrics about data distributions and whether those distributions still fit the use case.

### Machine learning security spans the entire system
[15:15](https://www.youtube.com/watch?v=6s9Y5fgP3dg&t=915s)
Saucedo places cybersecurity concerns across the full machine learning lifecycle. Risks can occur during data processing, model training, model serving, and work with data and metadata. He describes machine learning security as an area that is growing across the industry and mentions his work chairing a Linux Foundation working group on ML and MLOps security. Security therefore needs to be considered as part of the system architecture rather than added only at the serving stage.

### ML development processes should depend on use-case risk
[16:31](https://www.youtube.com/watch?v=6s9Y5fgP3dg&t=991s)
A machine learning development lifecycle cannot simply copy a software development lifecycle. Some use cases need risk assessments or ethics board approval, while a short experiment may need much less process. Saucedo says governance and operations should depend on the use case. The same principle applies to deployment decisions: high-impact systems need closer alignment with domain experts, appropriate KPIs and SLOs, and proportionate controls. Sandboxing and human review can reduce risk in some cases. Some settings may be better served by simpler, more explainable models.

### Teams move from projects to products as ML systems grow
[18:41](https://www.youtube.com/watch?v=6s9Y5fgP3dg&t=1121s)
Saucedo describes a shift from delivering a one-off machine learning project to maintaining an AI product that gains new model versions, features, and system capabilities over time. Teams increasingly use cross-functional structures such as squads, bringing together machine learning practitioners, UX researchers, full-stack engineers, and domain experts. As the number of models and pipelines grows, organizations add machine learning engineers and MLOps engineers and increase automation, standardization, security, and observability gradually rather than starting with every control at once.

## Notable quotes
- "We no longer believe that the life cycle of the model ends once it's trained. If anything, it actually begins once it's trained and once it starts being consumed." (02:08)
- "The impact of a bad solution can be worse than no solution at all." (03:30)
- "It is not possible to just copy paste what we've already done for software when it comes to the governance and operations of our production machine learning methodologies at scale." (17:28)
- "Not everything has to be solved with AI. When you run with a hammer, everything looks like a nail." (21:21)
- "It doesn't matter how many abstractions and how many LLMs, the impact will always be human." (21:42)

## Tools & references mentioned
- Zalando SE
- Institute for Ethical AI
- ACM
- Linux Foundation
- MyMLOps
- Hugging Face
- AI Act
- GPUs
- LLMs
- DataOps
- Spotify's Squad

## Who should watch
- You are designing an ML platform and need a practical map of the data, model, deployment, metadata, monitoring, and security concerns it must cover.
- Your organization is moving from experiments to multiple production models and needs guidance on team composition, governance, and operational controls.
- You are deciding how much process to apply to an AI use case and want to connect deployment decisions to proportionate risk, human review, and domain expertise.

## Editor's note

Alejandro Saucedo says production ML needs metadata interoperability across experiments, labels, datasets, deployments, and data products, rather than tracking model artifacts alone. ZenML records each pipeline run's steps, inputs, outputs, and code version, so artifacts and models can be traced back to the data and code that produced them.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Maturing Machine Learning in Enterprise](https://mlopstalks.com/talks/maturing-machine-learning-in-enterprise) (Kyle Gallatin, Etsy, 47:09)
- [Hierarchy of Machine Learning Needs](https://mlopstalks.com/talks/hierarchy-of-machine-learning-needs) (Phil Winder, Winder Research, 58:26)
- [Luigi in Production](https://mlopstalks.com/talks/luigi-in-production) (Luigi Patruno, ML in Production, 47:23)
- [Scaling AI in Production](https://mlopstalks.com/talks/scaling-ai-in-production) (Srivatsan Srinivasan, AIEngineering, 51:56)
- [From Idea to Production ML](https://mlopstalks.com/talks/from-idea-to-production-ml) (Lex Beattie, Spotify, 53:18)