A useful MLOps platform is built with its users, through feedback and direct work with the data science teams who will use it.
2
Virgin Media's platform runs training and deployment pipelines across separate experiment, UAT, and production environments, with 48 projects in production.
3
The team uses modular agent templates, observability, and a Slack assistant called Pina to reduce support work and test new AI capabilities quickly.
Summary
Shahd Alghrsi describes how Virgin Media built an MLOps platform that helps data scientists move from experiments to production. The platform uses customizable container images, a Python package, a command-line tool, Copier templates, Vertex Workbench, Vertex AI pipelines, separate experiment, UAT, and production environments, and a central model registry. Shahd says the platform has run around 34,000 pipelines with a 97 percent success rate and supports 48 production projects. The team also added federated GCP projects for sensitive data, Evidently AI for model and data monitoring, and Pina, a Slack assistant connected to Gemini, GitLab documentation, and GCP logs. Her broader advice is to build with users, respond to requests such as federation, and create reusable templates for agents. The talk is practical about platform maturity: automation alone is insufficient when people cannot understand, trust, or use the platform.
Production delays often start with a gap between development and infrastructure
Shahd describes teams that experiment with a machine learning project but hesitate to release it because production infrastructure is missing or unreliable. That gap can create a six- to twelve-month delay between development and a production-ready model or project. The result is that fewer projects reach production, even when the initial experimentation was successful. She frames the MLOps platform as a way to reduce this friction, provided it is designed around the people who will use it.
A platform should be built with its users through a feedback loop
Virgin Media supports about 200 data scientists, so the platform team cannot assume that a new tool will be adopted simply because it has been built. Shahd says the team asks users what they need and develops tools with them. She recommends that teams first understand their current level of automation. The stages she describes range from fully manual work, to automated training with manual deployment, to fully automated pipelines with CI/CD support.
Maturity lets the platform run many production pipelines at scale
Shahd gives Virgin Media's recent operating figures as an example of what platform maturity enables. Over the previous year, the team ran around 34,000 pipelines successfully, with a 97 percent success rate, and had 48 projects in production. The platform supports data scientists with customizable base images, a Python package, and a command-line tool for pipelines, models, and other changes. Copier templates help new projects start from a standard structure.
Separate environments reduce the risk of moving work into production
The platform supports batch scoring, training, score analysis, and online models through model-as-a-service. Data scientists experiment in Vertex Workbench, while pipelines run in Vertex AI. The team uses separate experiment, UAT, and production environments. UAT is designed to be almost identical to production, so teams can test their work in a close replica before deployment. Shahd says this arrangement reduces the friction and risk associated with failing in production.
Federated projects give sensitive-data teams more control
The credit risk analysis team needed to work with sensitive data and wanted more control over its pipelines and projects. Virgin Media responded by allowing teams to create their own federated GCP projects. These teams retain access to the platform's tools while isolating their environments and data. Shahd presents federation as an example of a mature platform adapting to a specific use case instead of forcing every team into exactly the same setup.
Observability covers model quality, data quality, and responsible use
Virgin Media uses Evidently AI, an open-source observability tool, to monitor models and data quality over time. It produces HTML reports and dashboards where data scientists can inspect datasets and models for drift and performance degradation. Shahd connects this work to the FAT framework, meaning fairness, accountability, and transparency. Monitoring should help teams detect bias, understand who changed what, and investigate where a system behaved well or poorly.
Pina turns platform documentation and logs into support help
The platform team receives around 5,000 support messages each year and previously had to assign an engineer to the support rotation each week. They connected Slack with Gemini, GitLab documentation, and logs from GCP Log Explorer. When a data scientist tags Pina with a failed pipeline and its GitLab repository, the assistant can often resolve the problem without an engineer's intervention. Shahd says Pina was built in two weeks and moved into production after that test period.
Reusable agent templates make experimentation easier to control
Several teams wanted to copy Pina for their own support use cases. Instead of deploying unrelated agents across the organization, the platform team created a template. A team can use the same structure while changing the source of its documentation, such as using Confluence instead of GitLab Pages. Shahd recommends regular scanning of new tools and short experiments, including a dedicated two-week trial, so teams can test whether an idea is ready for production without allowing pilots to run indefinitely.
