# Build vs Buy an ML Platform

Diego Oppenheimer, Algorithmia | MLOps Meetup | Episode 20 | 57:20
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=1bHQE11Qq0k
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/build-vs-buy-an-ml-platform
Published: 2020-06-29
Tags: build-vs-buy, governance, monitoring, platform-teams

## TL;DR
- Building an ML platform includes deployment, versioning, monitoring, security, governance, upgrades, and support across many tools and environments.
- The build-versus-buy decision should start with the business result, the value of the internal team's time, and the number of use cases the platform must support.
- Teams should prove business value with a small number of use cases before building infrastructure for every possible ML workload.

## Summary
Diego Oppenheimer explains why an ML platform is much more than a model-serving API. A team must support serialized models, model and source-code versioning, reusable model libraries, pipelines, data connections, APIs, CI/CD, scaling, monitoring, alerting, security, governance, and audit trails. The work grows when the platform has to support different frameworks, languages, dependencies, clouds, and on-premises systems. Diego argues that building is a continuing commitment, since upgrades, zero-day fixes, framework changes, and user support remain after the first version. His advice is to start with the business improvement, define the smallest improvement that can be justified, and then work backward into the technology. Buying can make sense when it lets a team use its specialists on new business use cases. Building can make sense when platform development is where the organization creates value, but it requires a long-term staffing and maintenance plan.

## Key ideas
### The business outcome should decide where a team spends its time
[09:47](https://www.youtube.com/watch?v=1bHQE11Qq0k&t=587s)
Diego says technologists naturally enjoy assembling tools, but the goal is to move a business process forward. A data scientist should spend time on data, models, and parameters. A machine learning engineer should focus on repeatability and scale. An application engineer should focus on consuming the model in a business process. Building a platform is worthwhile only when it is where the team can add the most value. Otherwise, the team should use an existing platform and spend its time on the business result.

### A simple model API hides the hard operational work
[12:25](https://www.youtube.com/watch?v=1bHQE11Qq0k&t=745s)
Training a model and exposing it through a Flask API is relatively easy, Diego says. The difficult work starts when the system has to run in an organization. Teams must decide who handles security patches, updates, dependencies, and the rest of the pipeline behind the API. The platform also has to manage many models, track what is running, support knowledge sharing, and provide a catalog and versioning. The first working pipeline is only the beginning of the operational commitment.

### Building an ML platform grows into a changing, reusable system
[14:05](https://www.youtube.com/watch?v=1bHQE11Qq0k&t=845s)
Diego describes build as a journey rather than a one-time decision. He uses Uber's Michelangelo as an example of an internal platform that initially supported TensorFlow, then had to expand as requirements changed. A recipe that works for one model may fail when an organization has hundreds of models. At that point, teams need catalogs, versioning, utilities, observability, and ways to reuse knowledge. The platform must evolve as use cases, tools, and frameworks change.

### The production lifecycle starts with a serialized model
[23:25](https://www.youtube.com/watch?v=1bHQE11Qq0k&t=1405s)
Algorithmia's view of MLOps begins after model development produces a serialized model. The model needs to connect to inference data, enter a searchable catalog, work in pipelines, run on infrastructure, and be monitored and governed. Diego expects multiple development environments, such as DataRobot, OpenCV, Spark workflows, and Jupyter notebooks, to produce models that need to reach production. The platform therefore has to integrate with different development tools instead of assuming one environment.

### Every supported tool and environment expands the build matrix
[28:00](https://www.youtube.com/watch?v=1bHQE11Qq0k&t=1680s)
A production platform must handle different frameworks, libraries, languages, processing dependencies, data connections, and infrastructure providers. Diego describes this as a multiplication of possible combinations. CI/CD should trigger tasks such as containerization, registration, deployment, and testing without requiring manual clicks. The system also needs monitoring and alerts for failures, capacity problems, model degradation, and application errors. Supporting these combinations takes ongoing engineering work.

### Cloud portability does not remove infrastructure differences
[31:06](https://www.youtube.com/watch?v=1bHQE11Qq0k&t=1866s)
Organizations may need to run models on AWS, Azure, on-premises systems, or several of them because of data residency and other requirements. Managed services do not automatically solve portability. Diego points out that Kubernetes offerings such as EKS, AKS, and GKE use different networking profiles and behave differently in production. A team building its own platform must decide whether it can keep deployment, containerization, and operations consistent across those environments.

### Security and governance belong inside the ML lifecycle
[33:07](https://www.youtube.com/watch?v=1bHQE11Qq0k&t=1987s)
Diego treats ML workloads as software that needs the same security and compliance discipline as other production systems. Dependencies downloaded from package repositories can create attack vectors if teams do not check them. Organizations also need permissions, data security, network security, model security, audit trails, and explanations where regulations require them. Financial services, life sciences, and insurance may need to show regulators individual inferences and why they were made. Building this architecture is part of the platform decision.

### Maintenance is a large part of the build commitment
[36:25](https://www.youtube.com/watch?v=1bHQE11Qq0k&t=2185s)
Diego says the word build can hide the work that follows. A zero-day vulnerability may require an urgent upgrade, while new versions of Python, libraries, frameworks, and managed services require regular testing and migration. In his experience, successful long-term internal platform efforts have had at least ten engineers dedicated to the project. Those people also need to support the teams using the platform. A build decision therefore needs a budget for maintenance and user support, not only the first implementation.

### Start with a small, justifiable business improvement
[44:06](https://www.youtube.com/watch?v=1bHQE11Qq0k&t=2646s)
Diego recommends starting from the business result and working backward. He cites Ian Chow's idea of a 'minimal justifiable improvement', an ML version of an MVP. The improvement must justify the cost of finding data, training, deploying, and operating the model. He warns against building infrastructure for every possible ML use case before proving one. Teams should start with a small number of use cases, show that ML creates value, and then expand.

## Notable quotes
- Diego Oppenheimer: "My mission is to move the needle for my business." (10:27)
- Diego Oppenheimer: "The build is maybe 25% of the cost and maintain is the other 75% of the cost." (37:48)
- Diego Oppenheimer: "Data science and ML is the most iterative process in software we've ever seen." (51:18)
- Diego Oppenheimer: "Chewing off too much or starting without a use case are the two biggest fuck-ups that we've seen." (50:19)

## Tools & references mentioned
- Algorithmia
- Microsoft
- Carnegie Mellon
- USC
- Amazon
- Uber
- Michelangelo
- TensorFlow
- DataRobot
- OpenCV
- Spark
- Jupyter notebooks
- H2O
- Domino
- Kubernetes
- AWS
- Azure
- EKS
- AKS
- GKE
- Python
- Ian Chow
- ML is boring
- Flask
- GitHub
- Jenkins
- Travis CI
- CircleCI

## Who should watch
- You are deciding whether your team should build an internal ML platform or buy a product for model operations.
- Your models work in notebooks or prototypes, and you need to understand what deployment, security, governance, and maintenance will add.
- You manage an ML team and need a way to assess platform scope, staffing, and business value before committing to a large build.

## Editor's note

Diego Oppenheimer says building an ML platform is a continuing commitment, since upgrades, zero-day fixes, framework changes, and user support remain after the first version. ZenML lets teams write workflows as Python steps, record each run's inputs, outputs, and code version, and move the same pipeline across configured infrastructure.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Building ML/Data Platform on Top of Kubernetes](https://mlopstalks.com/talks/building-ml-data-platform-on-top-of-kubernetes) (Julien Bisconti, 48:13)
- [Scaling ML Capabilities in Large Organizations](https://mlopstalks.com/talks/scaling-ml-capabilities-in-large-organizations) (Bertjan Broeksema & Axel Goblet, BigData Republic, 1:02:47)
- [How to Choose the Right ML Tool](https://mlopstalks.com/talks/how-to-choose-the-right-ml-tool) (Jose Navarro, Cookpad & Mariya Davydova, Neu.ro, 1:00:39)
- [Building an ML Platform at SurveyMonkey](https://mlopstalks.com/talks/building-an-ml-platform-at-surveymonkey) (Shubhi Jain, SurveyMonkey, 55:42)
- [ML Platform Tradeoffs and Wondering Why to Use Them](https://mlopstalks.com/talks/ml-platform-tradeoffs-and-wondering-why-to-use-them) (Javier Mansilla, Mercado Libre, 53:57)