# Project/Product Management for MLOps

Korri Jones-Simarpal Khaira-Veselina Staneva | MLOps Meetup | Episode 68 | 57:22
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=G3gOz_7RBfw
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/project-product-management-for-mlops
Published: 2021-06-21
Tags: build-vs-buy, platform-teams, product-strategy, team-adoption

## TL;DR
- MLOps products need to solve today's problem while leaving room for the harder problems a growing business will face.
- Internal platform teams should choose build, buy, or a hybrid approach based on business value, ownership cost, available skills, maturity of the market, and the need for customization.
- Teams avoid stalled projects by agreeing on acceptance criteria, keeping stakeholders involved, assigning a driver, and defining what good enough means.

## Summary
The panel discusses how product and project managers can build MLOps capabilities that fit both present needs and future growth. Korri Jones argues that teams should plan around the difficult problems ahead, while building the skills needed to reach that future. Simarpal Khaira describes product discovery across technical user groups, with developer velocity as an internal platform's main success measure. The panel weighs build, buy, and hybrid choices against ROI, total cost of ownership, compliance, security, market maturity, and customization needs. Veselina Staneva warns external products against becoming a collection of unrelated features and recommends choosing a clear niche. The speakers also discuss shadow platforms, adoption problems, and the difference between treating machine learning as a product or a one-off project. They finish with practical ways to prevent projects from getting trapped in proof-of-concept work, including stakeholder check-ins, clear acceptance criteria, documentation, and explicit definitions of done.

## Key ideas
### Future planning should shape today's product and team decisions
[02:12](https://www.youtube.com/watch?v=G3gOz_7RBfw&t=132s)
Korri Jones says teams should solve today's problem while preparing for the harder version that may arrive later. His example moves from handling a gigabyte of data today to potentially needing a terabyte in a year. He recommends thinking about business and customer growth, then checking whether the team has the skills for the future challenge. A team that knows Excel may need time to learn Spark before the business requires it. Simarpal Khaira adds that product managers should watch what data scientists and other groups are trying, validate patterns across teams, and decide when a local need should become a shared platform capability.

### A product vision only works when the team can execute it
[06:22](https://www.youtube.com/watch?v=G3gOz_7RBfw&t=382s)
Veselina Staneva says a large vision cannot be delivered by one person. Product managers need to understand their team's skills, motivation, and readiness, then help people build the skills and mindset required. Team members may reach that point at different speeds. She also recommends looking outside machine learning for useful practices, since methods from other fields can improve a project. Korri adds that product and project managers need enough open time to study where the future is going and check whether it still fits their vision.

### Product discovery depends on asking users about their real work
[09:19](https://www.youtube.com/watch?v=G3gOz_7RBfw&t=559s)
Staneva recommends The Mom Test because leading or closed questions can produce answers that sound useful but do not reveal a real need. Khaira starts by defining who the product is for, since MLOps platforms may serve analysts, data scientists, machine learning engineers, software engineers, or citizen data scientists. He filters ideas against the intended customer group, builds mock-ups, and walks users through them before implementation. This gives his team early feedback from its internal data science community and helps decide which ideas deserve more work.

### Build, buy, or hybrid decisions need a business case and an ownership plan
[14:49](https://www.youtube.com/watch?v=G3gOz_7RBfw&t=889s)
For internal platforms, Korri Jones recommends using the five whys to identify the underlying purpose, business value, expected ROI, and total cost of ownership. A team should compare the cost of building and maintaining a capability with buying it from a provider, while also considering a hybrid option for the hardest parts. The decision depends on available skills and support capacity. Khaira adds that teams should consider how quickly users need the capability, whether it is core to the platform, how mature the market is, and whether an outside product can meet compliance, security, and privacy needs.

### Internal platform success is measured by developer velocity and adoption
[21:11](https://www.youtube.com/watch?v=G3gOz_7RBfw&t=1271s)
Khaira says an internal platform may increase company costs without being directly tied to revenue, so its value needs another measure. His team's success metric is developer velocity, meaning how quickly users can build models and productionalize pipelines. That improvement might come from self-service or automation. Jones warns that building a platform does not guarantee adoption. If the platform is hard to use or supports too few people, teams may create shadow machine learning or shadow IT systems. Those systems can bypass security, compliance, monitoring, and model registration.

### External products need a defined niche instead of every possible feature
[24:11](https://www.youtube.com/watch?v=G3gOz_7RBfw&t=1451s)
Staneva says an internal platform has one company's business context, but it still needs to match both current team needs and the company's longer-term machine learning direction. It should also stay aware of common market practices, since an overly custom platform can make hiring and onboarding harder. For an external product, she recommends choosing a specific niche and solving that group's problems well. Job titles alone are not enough to define users because people with the same title can have very different responsibilities. Team structure, the role of machine learning in the business, and how people work together are better signals.

### Machine learning needs product ownership after the first model reaches production
[39:03](https://www.youtube.com/watch?v=G3gOz_7RBfw&t=2343s)
Staneva explains that companies often treat machine learning as a project when it is not part of their core business. That can be acceptable if they explicitly arrange for maintenance, whether internally or through outsourcing. Jones says his brick-and-mortar organization treats models as products because data changes and models can drift. A project is not finished when training and deployment end. Khaira says machine learning culture grows when leadership connects machine learning work to business value, rather than mentioning AI only in external publicity.

### Clear acceptance criteria and regular stakeholder contact prevent POC stagnation
[46:48](https://www.youtube.com/watch?v=G3gOz_7RBfw&t=2808s)
Jones says every project needs a driver who keeps the work moving after the team's initial energy fades. His team uses agile data science, with stakeholder reviews every two or three weeks, even when there is no finished interface to show. The team also agrees on what good enough means, so it can deliver useful value before pursuing perfection. Staneva adds that teams need clear goals, decision rules, and definitions of done. Khaira recommends documenting acceptance criteria at the start, involving only the right people, and being transparent when a new request belongs in a later sprint.

## Notable quotes
- Korri Jones: "Solve today's problem but make sure that you're flexible enough to solve the really, really hard stuff." (02:54)
- Veselina Staneva: "If you're solving everybody's problems you're not solving anyone's problems." (25:33)
- Korri Jones: "If you build it, it doesn't mean they're going to come." (26:26)
- Simarpal Khaira: "Our success metric is the developer velocity." (21:47)
- Korri Jones: "Scope creep eats product teams for breakfast." (56:11)

## Tools & references mentioned
- Spark
- The Mom Test
- Five Whys
- Field of Dreams

## Who should watch
- You are deciding whether your company should build an internal MLOps platform, buy one, or combine both approaches.
- Your team has several technical user groups with different workflows and you need a way to choose which needs to support.
- Projects keep stopping at the proof-of-concept stage, or deployed models lack ownership, monitoring, and a maintenance plan.

## Editor's note

Simarpal Khaira says an internal platform should measure success by how quickly users can build models and productionalize pipelines. ZenML lets teams write workflows as Python steps and run the same pipeline on a laptop, Kubernetes, Airflow, Kubeflow, or a cloud service by changing the configured stack. Each run records its steps, inputs, outputs, and code version.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Practitioners Guide to MLOps](https://mlopstalks.com/talks/practitioners-guide-to-mlops) (Donna Schut & Christos Aniftos, Google Cloud, 46:34)
- [MLOps - The Blind Men and the Elephant](https://mlopstalks.com/talks/mlops-the-blind-men-and-the-elephant) (Saurav Chakravorty, Brillo, 55:02)
- [The Current MLOps Landscape](https://mlopstalks.com/talks/the-current-mlops-landscape) (Nathan Benaich, Air Street Capital & Timothy Chen, Essence VC, 58:31)
- [Engineering MLOps](https://mlopstalks.com/talks/engineering-mlops) (Emmanuel Raj, TietoEvry, 51:55)
- [Investing in MLOps](https://mlopstalks.com/talks/investing-in-mlops) (Leigh Marie Braswell, Founders Fund & Davis Treybig, Innovation Endeavors, 48:52)