# MLOps Critiques

Matthijs Brouns, Xccelerated.io | MLOps Coffee Sessions | Episode 100 | 49:44
Hosted by David Aponte

Source: https://www.youtube.com/watch?v=SS2_jQN3sG0
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/mlops-critiques
Published: 2022-05-27
Tags: build-vs-buy, deployment, human-in-the-loop, monitoring

## TL;DR
- MLOps tooling still has gaps because deployment, monitoring, routing, and batching needs do not fit neatly into existing products.
- Matthijs Brouns chooses tools through regret minimization, favoring clear interfaces and easy migration over the highest possible payoff.
- Safe releases need automation around the process, while human stakeholders still decide whether a model is acceptable.

## Summary
Matthijs Brouns describes MLOps as a field where standard answers are still hard to find. His experience with a client that grew from about 3,000 to 100,000 individually trained sensor models exposed problems with retraining, deployment, concurrency, and release management. He wants deployment systems to support consistent routing, shadow deployments, flexible canary rules, and automatic batching, but he has not found tools that cover these needs well. He also argues that MLOps products often grow into bloated suites, which makes migration difficult. When choosing technology, he favors libraries, clear interfaces, and tools that a team can replace later. For model releases, he automates repeatable work and produces reports for human review instead of forcing every decision into an automated pipeline. His advice is practical and cautious: reduce release risk, define interfaces between teams, and automate from observed human checks rather than assumptions.

## Key ideas
### MLOps decisions depend on the surrounding system
[00:29](https://www.youtube.com/watch?v=SS2_jQN3sG0&t=29s)
David Aponte says the conversation repeatedly showed how rarely a blanket answer applies in MLOps. Matthijs Brouns agrees through his examples from training and consulting work. Teams have different clouds, infrastructure, levels of ML experience, and operating constraints. A data scientist may know little about production infrastructure, while an infrastructure specialist may miss important model concerns. Xccelerated.io trains people before and during client work, then aims to transfer them into the client's team. Matthijs presents this practical experience as a reason to resist universal tool recommendations.

### Scaling individual models creates release and infrastructure problems
[17:10](https://www.youtube.com/watch?v=SS2_jQN3sG0&t=1030s)
One client collected data from a large sensor network and trained a separate time-series model for each sensor, using nearby sensors when they provided useful correlations. The models followed the same blueprint, but each still had to be trained and deployed individually. The system first ran on Azure Functions and encountered concurrency limits and rising costs. At around 3,000 models, deploying new code already required synchronous retraining and manual work. At 100,000 models, that approach no longer worked. The harder problem became how to release changes safely and with low risk across the whole fleet.

### Deployment tools need more flexible routing and batching
[23:20](https://www.youtube.com/watch?v=SS2_jQN3sG0&t=1400s)
Matthijs wants deployment systems to support shadow deployments, partial rollouts, and routing based on a useful unit such as a user or sensor. He criticizes default canary implementations that randomly assign each request, since one entity can move back and forth between model versions. A consistent assignment is often the minimum requirement. He also wants automatic request batching, preferably handled at the load-balancer level. Application-level batching can leave several servers active and waiting for queues to fill. A smarter ML load balancer could combine routing, shadow traffic, batching, and scaling decisions.

### MLOps should reuse ordinary infrastructure where it fits
[19:34](https://www.youtube.com/watch?v=SS2_jQN3sG0&t=1174s)
The conversation distinguishes ML-specific concerns from general infrastructure work. Kubernetes scheduling, ingress, load balancing, and other systems problems do not always need to be rebuilt as ML products. Matthijs says the field repeatedly reinvents infrastructure components for ML. David Aponte connects this to high-performance computing and existing scheduler work. Matthijs also describes cycles in workflow tools, from XML-based systems such as Oozie and Azkaban, to code-based tools such as Airflow and Prefect, and then back toward YAML in Argo. He questions what the next interface will be without claiming that drag-and-drop tools solve the underlying design problem.

### Tool choice should include an exit plan
[24:18](https://www.youtube.com/watch?v=SS2_jQN3sG0&t=1458s)
Matthijs does not want teams to build everything themselves, but he evaluates how easily a tool can be removed if it becomes unsuitable. He prefers libraries to frameworks because libraries usually create fewer ties to the rest of an application. This matters in MLOps because the right interfaces are still unsettled. He is wary of products that combine experiment tracking, deployment, monitoring, fairness, and other functions into one package. A tool may become difficult to maintain or replace as its scope grows. His preferred choice is often the option a team will regret least several years later, with good APIs and limited surprises.

### Monitoring needs to include the whole software stack
[29:21](https://www.youtube.com/watch?v=SS2_jQN3sG0&t=1761s)
Matthijs says ML monitoring products often focus too narrowly on the model or data while ignoring the software around it. In many of his projects, the ML component is small compared with the surrounding services and pipelines. He does not want users to monitor different parts of one system in disconnected products. He also asks whether monitoring data can be exported to existing systems such as Prometheus, Splunk, or Datadog. The lack of a default interface makes this difficult. He sees useful tools emerging, but says teams need to make conscious decisions about what they actually want to adopt.

### Automation should support human approval rather than replace it
[40:56](https://www.youtube.com/watch?v=SS2_jQN3sG0&t=2456s)
Matthijs identifies manual validation, unavailable stakeholders, unclear data interfaces, and missing rollback paths as common barriers to production. He does not recommend automating every model decision. Instead, a retraining process can run in CI/CD, open a pull request, and attach a report comparing the new model with the previous one. A stakeholder can inspect the report and approve deployment. Teams can discover further automation opportunities by asking stakeholders what they check repeatedly. Matthijs says he starts cautiously because he often cannot predict which release decisions will be safe to automate. The process becomes more automated as the team's real checks become clear.

## Notable quotes
- Matthijs Brouns: "For me that is probably the most important thing about MLOps as a movement or as a culture, how do we get rid of barriers to do that." (14:00)
- Matthijs Brouns: "I sort of err on the side of caution and think, okay, what is the thing that we're gonna regret the least three years down the line?" (32:20)
- Matthijs Brouns: "I don't like frameworks, I like libraries for the same reason that David just mentioned, frameworks tie you into a lot of things." (24:30)
- Matthijs Brouns: "I learned to err on the side of not automating those kinds of decisions, but automating the process and just giving them a report." (43:23)

## Tools & references mentioned
- Xccelerated.io
- Go Data Driven
- Azure Functions
- Kubernetes
- Oozie
- Azkaban
- Hadoop
- Airflow
- Luigi
- Prefect
- Argo
- Prometheus
- Grafana
- Splunk
- Datadog
- Pi Script
- Rhythm of War
- Brandon Sanderson
- Database Internals
- Alex Petrov
- scikit-learn
- PyData Amsterdam

## Who should watch
- You are choosing MLOps tools and want a way to think about migration risk before committing to a large platform.
- Your team runs many model instances and needs safer rollouts, consistent traffic routing, or request batching.
- You are trying to move model work into production while manual validation and unclear team interfaces keep slowing releases.

## Editor's note

Matthijs Brouns says a retraining process can run in CI/CD, open a pull request, and attach a report comparing the new model with the previous one. ZenML records each run's steps, inputs, outputs, and code version, so the comparison has a traceable record of how each model was produced. Its pipeline code can run on configured infrastructure without rewriting the workflow.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Practical MLOps Part 2](https://mlopstalks.com/talks/practical-mlops-part-2) (Alfredo Deza, Author and Speaker, 1:01:38)
- [Most Underrated MLOps Topics](https://mlopstalks.com/talks/most-underrated-mlops-topics) (Marian Ignev, CloudStrap.io & SashiDo.io, 53:56)
- [MLOps vs LLMOps](https://mlopstalks.com/talks/mlops-vs-llmops) (Richa Sachdev, JPMorgan Chase & Willem Pienaar, Feast & Chris Van Pelt, Weights & Biases & Aparna Dhinakaran, Arize AI & Alex Ratner, Snorkel, 34:57)
- [MLOps in Practice: Common Challenges and Lessons Learned](https://mlopstalks.com/talks/mlops-in-practice-common-challenges-and-lessons-learned) (Marouen Hizaoui & Mo Basirati, Machine Learning Reply, 1:03:01)
- [Who's MLOps for Anyway?](https://mlopstalks.com/talks/whos-mlops-for-anyway) (Jonathan Rioux, EPAM Systems, 1:10:15)