MLOps level 2 automates the build, testing, deployment, triggering, retraining, and serving of an entire ML pipeline.
2
Continuous training can respond to new data, concept drift, or performance thresholds, but the hosts argue that people still need to monitor the system and interpret what happens in production.
3
Teams can move toward full automation gradually, starting with small improvements to source control, testing, reproducibility, deployment, and monitoring.
Summary
Demetrios Brinkmann and David Hershey discuss the highest automation level in Google's MLOps maturity model. At this level, teams deploy an entire pipeline rather than only a prediction API. Source code is built, tested, packaged, and delivered to a target environment. Automated triggers can rerun the pipeline on a schedule or when monitoring detects problems such as degraded performance, outliers, or concept drift. The resulting model is stored in a model registry and deployed as a prediction service. The conversation covers source control, test and build services, deployment services, feature stores, metadata stores, reproducibility, and model serving. The hosts are cautious about the idea of leaving a fully automated system alone. They argue that production data creates new information about model behavior, so data scientists still need to inspect results and design new experiments. Their practical advice is to improve maturity in small steps instead of trying to build the complete system at once.
Demetrios Brinkmann and David Hershey explain that Google's highest maturity level goes beyond deploying one prediction service. The production system includes the steps that extract and validate data, prepare features, train and evaluate the model, and launch the serving layer. Automation lets data scientists try new features, model architectures, and hyperparameters, then build, test, and deploy those changes more quickly. The model is treated as one output of a larger pipeline. This creates a link between experimentation and production because the code that produces the model can be rerun in the target environment.
The system needs shared components that preserve what happened
Source control provides a single place to track changes and lets several people work on the pipeline. Test and build services can build images or other packages whenever code changes, then run checks before deployment. A model registry records which models exist, which are in production, and the information needed to understand them, such as parameters, timestamps, training data references, and metrics. The metadata store records runs, jobs, logs, execution environments, artifacts, packages, models, and performance measurements. These records make it possible to reproduce experiments, rerun a pipeline, or roll back to an earlier version.
The hosts describe a feature store as an interface between data and models. It can support offline experimentation and online production use, and it can provide batch or real-time features. They caution that real-time serving has substantial cost and operational complexity, so teams should use it only when the business needs it. Feature freshness depends on the data sources and on how derived features change over time. The model itself can also become stale. For problems involving changing prices or other time-sensitive data, the team may need to decide whether daily updates are enough or whether continuous updates are required.
Automation can retrain models through monitored triggers
A fully automated setup can watch model performance and trigger retraining when a metric falls below a threshold. Other triggers can come from outlier detection, concept drift, new data, or a new experiment. The pipeline then produces a trained model from the latest available data, records it in the model registry, and deploys a prediction service. The hosts reject the idea that this means a model can simply be forgotten after deployment. Automated systems can repeat a bad process, and production behavior still needs human review. Monitoring creates a feedback loop from live performance back to data analysis, model analysis, and new experiments.
The pipeline has distinct stages with different outputs
The development and experimentation stage outputs source code for the ML pipeline, rather than only a trained model. Continuous integration builds that source code, runs tests, and produces pipeline components such as packages, executables, and artifacts. Continuous delivery deploys those artifacts as a pipeline in a development, staging, or production environment. An automated trigger runs the deployed pipeline and produces a trained model for the registry. Model continuous delivery turns that model into a prediction service. Monitoring collects statistics from live data and produces the next trigger or starts another experiment cycle.
CI testing should cover both software and ML behavior
The hosts describe unit tests for feature engineering and feature generation logic, along with integration and contract tests for the systems that provide data. Other checks can verify feature types and value ranges, schema validity, model loading, model convergence, missing values, expected artifacts, and successful pipeline compilation. Continuous integration runs these checks whenever code is pushed, so a change with a bug does not move forward. Continuous delivery also needs checks for infrastructure compatibility, prediction responses, load, latency, data quality, and model quality. Some domain-specific validation is difficult to automate, but the hosts argue that difficulty is not a reason to omit all checks.
The hosts say that operating at maturity level zero is not automatically wrong. A team can choose one improvement that reduces manual work, then add more pieces as its needs grow. They recommend starting small instead of provisioning a large system around an unrealistic migration plan. Full automation requires substantial engineering and domain expertise, especially when teams must deploy orchestrators, databases, clusters, or serving infrastructure. A gradual path can make the work easier to reproduce and reduce the cost of building infrastructure that is not yet needed. The goal is to improve the system without assuming that every component must exist on the first day.
