# Uber's Michelangelo: Strategic AI Overhaul and Impact

 | MLOps Podcast | Episode 239 | 35:36
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=GSrPRxSwcvg
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/ubers-michelangelo-strategic-ai-overhaul-and-impact
Published: 2024-06-07
Tags: evals, governance, model-serving, platform-teams

## TL;DR
- Michelangelo moved through three phases, from predictive machine learning and tabular data to deep learning and generative AI.
- Michelangelo 2.0 added project tiering, deep learning support, model iteration as code, Ray, Nvidia Triton, and Kubernetes.
- Michelangelo 3.0 added a GenAI Gateway with logging, cost controls, policy guardrails, PII redaction, and model evaluation.

## Summary
Demetrios Brinkmann breaks down Uber's account of Michelangelo, its internal machine learning platform. The platform began in 2016 because teams were building models and production systems in disconnected ways. Michelangelo 1.0 created a shared path for predictive ML and introduced a feature store, but it treated projects alike, lacked deep learning support, and left teams with fragmented development tools. Michelangelo 2.0 added project tiering, self-service workflows, PyTorch support, Ray for training, Nvidia Triton for serving, and Kubernetes for GPU resource management. Demetrios then describes Michelangelo 3.0's support for generative AI through a GenAI Gateway that can route requests to internal or external language models. Uber added auditing, spend attribution, cost alerts, safety policies, PII redaction, and evaluation datasets. His broader conclusion is that early platform choices create long-lived maintenance decisions, so teams must weigh the value of upgrades against the work of replacing systems that still function.

## Key ideas
### Michelangelo began as a response to disconnected machine learning work
[04:56](https://www.youtube.com/watch?v=GSrPRxSwcvg&t=296s)
Uber started building Michelangelo in 2016 because data scientists used varied tools while engineering teams built separate systems for production. Demetrios describes the result as ad hoc work with no reliable, uniform, reproducible pipelines for training and prediction data at scale. Michelangelo created an established path to production and gave teams templates that were easy to use by default. Advanced users could still go deeper and customize components. The first phase, from 2016 to 2019, focused on predictive machine learning and tabular data. Its technology included TensorFlow, Spark, XGBoost, Cassandra, and MLlib.

### The first platform version treated high-value and experimental projects alike
[10:27](https://www.youtube.com/watch?v=GSrPRxSwcvg&t=627s)
Michelangelo 1.0 had no quality definitions or project tiering. Every project received the same treatment, even when one model drove millions of dollars in daily revenue and another had no clear return on investment. Demetrios presents tiering as a way to decide which projects deserve priority, support, and service-level agreements. The first version also lacked deep learning support and collaborative model development. Teams had the data needed for deep learning, but they did not have a supported developer experience, so some built their own tools around the platform.

### Michelangelo 2.0 made project impact and flexible architecture explicit design goals
[14:39](https://www.youtube.com/watch?v=GSrPRxSwcvg&t=879s)
From 2019 to 2023, Uber changed Michelangelo as teams found more revenue from machine learning and began using advanced methods. Its design principles included defining project tiers, focusing platform support on high-impact use cases, and providing self-service tools for the long tail. The architecture became plug-in based. Managed workflows supported a selected set of components for a simpler experience, while advanced users could bring their own components. The platform also combined an API-first approach with a user interface for visualization and fast iteration. Model changes made through the interface were intended to remain available for version control and code review.

### Uber embedded quality checks into the machine learning lifecycle
[19:36](https://www.youtube.com/watch?v=GSrPRxSwcvg&t=1176s)
Michelangelo 2.0 addressed quality at different points in the lifecycle. During development, the team considered code review and test coverage. During training, it considered reliability and hardware cost. For models, it tracked accuracy, freshness, and reproducibility. For serving, it considered latency, availability, and cost. Uber created a framework called the Model Excellence Score, or MLES, to measure and monitor these metrics. It also introduced model iteration as code, with managed dependencies, immutable Docker builds, an ML repository, and a continuous integration and delivery pipeline.

### Deep learning required changes to transformations, training, serving, and infrastructure
[21:41](https://www.youtube.com/watch?v=GSrPRxSwcvg&t=1301s)
Uber made deep learning a first-class part of Michelangelo by supporting deep learning in feature transformations and upgrading model training. The platform moved from Spark to Ray because deep learning workloads could fail in many ways on Spark. For serving, latency mattered for use cases such as driver ETAs and Uber Eats ranking, so Uber replaced Neuropod with Nvidia Triton. Triton could serve TensorFlow and PyTorch models directly. Uber also moved from Mesos to Kubernetes to improve GPU resource management and allow teams to share CPU and GPU capacity when resources were not being fully used.

### Michelangelo 3.0 routes generative AI requests through a shared gateway
[24:55](https://www.youtube.com/watch?v=GSrPRxSwcvg&t=1495s)
The third phase, starting in 2023, brought generative AI into the platform. Uber created a GenAI Gateway that can use external language model APIs and internal models. The gateway chooses a model for a request, using more general external models for some tasks and internal models for work that can benefit from Uber's own data. Demetrios says the gateway provides one calling experience while hiding the model choice from the user. Uber also added logging and auditing so it can track model use, along with cost guardrails, spending attribution, and alerts when usage exceeds expectations.

### Generative AI controls include policy checks, privacy protection, and model comparisons
[27:48](https://www.youtube.com/watch?v=GSrPRxSwcvg&t=1668s)
Michelangelo 3.0 includes safety and policy guardrails for generative AI use. When data goes to an external language model, personally identifiable information is redacted. Uber also built an evaluation framework for comparing models against use-case-specific datasets and evaluation frameworks. When a new model becomes available, teams can test it against their current setup and estimate whether it performs better or worse. These controls address the operational concerns that come with routing many internal requests to different language models.

### Early platform decisions create years of upgrade and maintenance choices
[30:07](https://www.youtube.com/watch?v=GSrPRxSwcvg&t=1807s)
Demetrios argues that Uber's history is a reminder that companies building ML platforms early had fewer tools available and often had to create their own systems. Those systems can become legacy infrastructure. Platform teams must decide whether to upgrade a component, replace it, or leave it alone when it still works for most users. He compares this with a discussion about Pinterest's ads platform, where engineers weighed maintaining an open-source tool with little community support against moving away from a deeply embedded dependency. His final point is that project tiering should guide where platform resources and support go: not all ML projects are created equal.

## Notable quotes
- Demetrios Brinkmann: "There is no established path to production nor once it's in production what are we doing with it and you'll see that this idea of trying to unify the system so that there's not so much ad hoc action happening that comes up again and again in the blog post." (12:26)
- Demetrios Brinkmann: "The platform team set out to create an architecture that allows for a bit of a Lego plug-in plate." (15:32)
- Demetrios Brinkmann: "They made sure that when people are using gen capabilities PII or the personal identifiable information is redacted." (28:11)
- Demetrios Brinkmann: "Not all ML projects are created equal having a clear ML tiering system can effectively guide the allocation of resources and support." (35:00)

## Tools & references mentioned
- Michelangelo
- Uber
- MLOps Community
- AI Quality Conference
- Weights & Biases
- TensorFlow
- Spark
- XGBoost
- Cassandra
- MLlib
- Tecton
- PyTorch
- PyTorch Lightning
- Ray
- Nvidia Triton
- Mesos
- Kubernetes
- Model Excellence Score
- MLES
- Neuropod
- GenAI Gateway
- Llama 3
- Mistral
- OpenAI
- Pinterest
- DoorDash
- Instacart
- Airbnb
- LinkedIn
- Spotify
- Nubank

## Who should watch
- You are designing an internal ML platform and need examples of how project tiering can change platform support.
- Your platform grew around older infrastructure, and you are deciding whether to upgrade components that still work for some teams.
- You are adding language models to production and want concrete examples of routing, auditing, spend controls, privacy checks, and evaluations.

## Editor's note

Demetrios Brinkmann describes Michelangelo as a response to ad hoc work with no reliable, uniform, reproducible pipelines for training and prediction data at scale. ZenML records each pipeline run's steps, inputs, outputs, and code version, so teams can trace an artifact or model to the data and code that produced it. Its configurable stack lets the same pipeline run on different infrastructure.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Inside Uber's AI Revolution: Everything About How They Use AI/ML](https://mlopstalks.com/talks/inside-ubers-ai-revolution-everything-about-how-they-use-ai-ml) (Kai Wang, Uber, 45:24)
- [Machine Learning Education at Uber](https://mlopstalks.com/talks/machine-learning-education-at-uber) (Melissa Barr & Michael Mui, Uber, 57:43)
- [GenAI in Production - Challenges and Trends](https://mlopstalks.com/talks/genai-in-production-challenges-and-trends) (Verena Weber, Verena Weber, 48:43)
- [How to Leverage ML Tooling Ecosystem](https://mlopstalks.com/talks/how-to-leverage-ml-tooling-ecosystem) (Mariya Davydova, Neu.ro, 55:57)
- [MLOps at the Age of Generative AI](https://mlopstalks.com/talks/mlops-at-the-age-of-generative-ai) (Barak Turovsky, Scale Venture Partners, 56:56)