John Crousse argues that machine learning teams should move from one-off projects to products that can be improved over time.
2
A useful ML system starts with a simple baseline and builds the factory needed to test and replace models quickly.
3
The main gaps he sees across teams are reusable data and features, connected evaluation and monitoring, and enough front-end skill to make model outputs useful.
Summary
John Crousse describes what changes when companies build machine learning products instead of isolated projects. A model is usually one part of a larger system, so teams need ownership of the surrounding product as well as the model. He recommends putting a simple baseline into a usable system early, then improving the system and replacing the model as feedback arrives. This approach is harder when teams lack data engineers or other specialists, so companies need ways to share libraries, data transformations, monitoring helpers, and other internal tools without removing team autonomy. John also discusses an internal ML maturity survey. Its recurring gaps were reusable pipelines and features, plus evaluation and monitoring that are split across disconnected processes. He would like a consistent evaluation API across the model life cycle. He also points to front-end and dashboard work as an overlooked way to make ML products useful to their users.
Companies are moving from one-off ML projects toward products that keep improving
John Crousse says machine learning began with a model-centred view shaped by academic papers, libraries, tutorials, courses, and data science job descriptions. That pattern encouraged a zero-to-one project: build something in six months, then move on to something else. He sees companies moving toward longer-lived products that are meant to be iterated on and improved. In this model, a recommender, search system, or document classifier has a team that specialises in the product. Recruitment and maintenance belong to that product rather than to a central pool of interchangeable data scientists.
John says he has rarely worked on a machine learning problem where ML was the only part of the solution. A working product also contains other algorithms, tools, and system components. This is why specialised teams often form around products such as recommendations, search, or document classification. Adam Sroka describes a related failure mode: technical teams can build complicated systems that customers do not need, while a simple feature becomes unexpectedly valuable. John accepts that a long-lived product is not always justified. Some businesses need a small feature that can be put in place quickly.
Decentralised teams need shared solutions to avoid rebuilding the same work
John says decentralised product teams create a staffing challenge because each team needs specialised skills. They can also reinvent the same solutions. He recommends a central team, or some other incentive, that helps data scientists build reusable and shareable solutions. He mentions Hamilton from Stitch Fix as an example of an internal product that is taken to different teams and adapted to their needs. Central monitoring helpers are another example. The goal is to reduce duplication without returning to a model where one central team does all machine learning work.
The first production system should be simple and ready for iteration
John dislikes treating production as a final ceremony after the first model is complete. Once a company decides to build an ML product, it can start with a baseline or even a model that predicts zero. Work on the surrounding system does not need to wait for the first sophisticated model. The important question is how quickly the team can try new models. He describes the surrounding system as a factory for iteration, with the model as an interchangeable part. A proof of concept often means exploratory analysis or an estimate of the business case, but it does not show whether the system meets speed, runtime, or service-level requirements.
ML maturity is reached when user feedback becomes the bottleneck
John describes a useful maturity test: the team has reached a good level of MLOps maturity when it can iterate fast enough that the limiting factor is the speed of feedback from users. He clarifies that he initially meant feedback that provides new labels for data scientists, such as information from customer interactions. Adam asks about the role of product owners in turning that feedback into priorities. John agrees that product owners must consider the wider business goal. Model metrics are only proxies for outcomes such as customer retention or customer happiness, and a possible ML product owner role could help connect those measures.
A single end-to-end ML platform can make integration harder
John warns that vendor products promising everything needed for ML can slow teams down. Machine learning is usually only one component in a production system, and an all-in-one platform may not interoperate well with the system's other algorithms and services. Teams can end up hacking integrations so that a platform works with a custom component in their pipeline. For that reason, he is sceptical that one generic ML platform will fit every enterprise. When choosing tools, he personally prefers open source with an SDK that is easy to use, and he is wary of tools that make a generic problem look easier than the team's actual problem.
Evaluation and monitoring should be connected across the model life cycle
John explains that an internal survey assessed models with mostly yes-or-no questions. It checked whether teams could identify the exact Git hash in production and link it to the relevant code and artifacts. It also covered A/B testing, reusable feature engineering, feature stores, evaluation, and monitoring. The survey found that offline training evaluation and production monitoring were often handled by separate processes and tools, even though both assess model performance. John would like an evaluation store or consistent API that can be called throughout the model life cycle, so evaluation is integrated from training through production rather than repeated in disconnected systems.
Reusable data work and fast diagnosis were the largest repeated gaps
The survey found recurring problems with reusing data pipelines and features. Decentralised teams often rebuilt the same features and transformations because they could not immediately select work from another team and test it in their own model. John suggests a central feature store or library of transformations as one possible answer. Monitoring was the other major gap. Teams need fast alerts, a way to trigger retraining when appropriate, and a quick path to understanding what caused a model problem. His preferred organisational approach is an initial central-team version that becomes an inner-source library, where product teams can add features and submit pull requests.
Front-end work is an overlooked part of making ML useful
John says data science teams often lack web and front-end development skills. A quick, unattractive dashboard can still prove that a model works, but a more useful and better-tailored interface can bring more attention and value to the product. He finds it difficult to find people with both the time and skills to build good visualisations, dashboards, or interactive user interfaces. Adam Sroka connects this problem to the large number of dashboard tools used across teams. John adds that front-end work is often pushed aside because teams have model bugs, new features, and releases to handle first.
"I would be excited to see what solutions will exist in the near future for evaluation, and just like really an API you call at many different steps of your model life cycle to have a consistent way of evaluating and monitoring your model."John Crousse33:21
Who should watch
You are moving from notebooks and proofs of concept toward ML products that need regular updates and operational ownership.
Your teams work independently but keep rebuilding the same data transformations, features, or monitoring utilities.
You need to decide whether an ML feature deserves a long-lived product investment or only a simple implementation.
