MLOps tooling still has gaps because deployment, monitoring, routing, and batching needs do not fit neatly into existing products.
2
Matthijs Brouns chooses tools through regret minimization, favoring clear interfaces and easy migration over the highest possible payoff.
3
Safe releases need automation around the process, while human stakeholders still decide whether a model is acceptable.
Summary
Matthijs Brouns describes MLOps as a field where standard answers are still hard to find. His experience with a client that grew from about 3,000 to 100,000 individually trained sensor models exposed problems with retraining, deployment, concurrency, and release management. He wants deployment systems to support consistent routing, shadow deployments, flexible canary rules, and automatic batching, but he has not found tools that cover these needs well. He also argues that MLOps products often grow into bloated suites, which makes migration difficult. When choosing technology, he favors libraries, clear interfaces, and tools that a team can replace later. For model releases, he automates repeatable work and produces reports for human review instead of forcing every decision into an automated pipeline. His advice is practical and cautious: reduce release risk, define interfaces between teams, and automate from observed human checks rather than assumptions.
David Aponte says the conversation repeatedly showed how rarely a blanket answer applies in MLOps. Matthijs Brouns agrees through his examples from training and consulting work. Teams have different clouds, infrastructure, levels of ML experience, and operating constraints. A data scientist may know little about production infrastructure, while an infrastructure specialist may miss important model concerns. Xccelerated.io trains people before and during client work, then aims to transfer them into the client's team. Matthijs presents this practical experience as a reason to resist universal tool recommendations.
Scaling individual models creates release and infrastructure problems
One client collected data from a large sensor network and trained a separate time-series model for each sensor, using nearby sensors when they provided useful correlations. The models followed the same blueprint, but each still had to be trained and deployed individually. The system first ran on Azure Functions and encountered concurrency limits and rising costs. At around 3,000 models, deploying new code already required synchronous retraining and manual work. At 100,000 models, that approach no longer worked. The harder problem became how to release changes safely and with low risk across the whole fleet.
Deployment tools need more flexible routing and batching
Matthijs wants deployment systems to support shadow deployments, partial rollouts, and routing based on a useful unit such as a user or sensor. He criticizes default canary implementations that randomly assign each request, since one entity can move back and forth between model versions. A consistent assignment is often the minimum requirement. He also wants automatic request batching, preferably handled at the load-balancer level. Application-level batching can leave several servers active and waiting for queues to fill. A smarter ML load balancer could combine routing, shadow traffic, batching, and scaling decisions.
MLOps should reuse ordinary infrastructure where it fits
The conversation distinguishes ML-specific concerns from general infrastructure work. Kubernetes scheduling, ingress, load balancing, and other systems problems do not always need to be rebuilt as ML products. Matthijs says the field repeatedly reinvents infrastructure components for ML. David Aponte connects this to high-performance computing and existing scheduler work. Matthijs also describes cycles in workflow tools, from XML-based systems such as Oozie and Azkaban, to code-based tools such as Airflow and Prefect, and then back toward YAML in Argo. He questions what the next interface will be without claiming that drag-and-drop tools solve the underlying design problem.
Matthijs does not want teams to build everything themselves, but he evaluates how easily a tool can be removed if it becomes unsuitable. He prefers libraries to frameworks because libraries usually create fewer ties to the rest of an application. This matters in MLOps because the right interfaces are still unsettled. He is wary of products that combine experiment tracking, deployment, monitoring, fairness, and other functions into one package. A tool may become difficult to maintain or replace as its scope grows. His preferred choice is often the option a team will regret least several years later, with good APIs and limited surprises.
Monitoring needs to include the whole software stack
Matthijs says ML monitoring products often focus too narrowly on the model or data while ignoring the software around it. In many of his projects, the ML component is small compared with the surrounding services and pipelines. He does not want users to monitor different parts of one system in disconnected products. He also asks whether monitoring data can be exported to existing systems such as Prometheus, Splunk, or Datadog. The lack of a default interface makes this difficult. He sees useful tools emerging, but says teams need to make conscious decisions about what they actually want to adopt.
Automation should support human approval rather than replace it
Matthijs identifies manual validation, unavailable stakeholders, unclear data interfaces, and missing rollback paths as common barriers to production. He does not recommend automating every model decision. Instead, a retraining process can run in CI/CD, open a pull request, and attach a report comparing the new model with the previous one. A stakeholder can inspect the report and approve deployment. Teams can discover further automation opportunities by asking stakeholders what they check repeatedly. Matthijs says he starts cautiously because he often cannot predict which release decisions will be safe to automate. The process becomes more automated as the team's real checks become clear.
"For me that is probably the most important thing about MLOps as a movement or as a culture, how do we get rid of barriers to do that."Matthijs Brouns14:00
Who should watch
You are choosing MLOps tools and want a way to think about migration risk before committing to a large platform.
Your team runs many model instances and needs safer rollouts, consistent traffic routing, or request batching.
You are trying to move model work into production while manual validation and unclear team interfaces keep slowing releases.
