# Challenges Operationalizing ML (And Some Solutions)

Nathan Ryan Frank, WW Grainger | MLOps Podcast | Episode 199 | 52:28
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=ezWdXzhwH8A
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/challenges-operationalizing-ml-and-some-solutions
Published: 2023-12-29
Tags: build-vs-buy, deployment, platform-teams, testing

## TL;DR
- Nathan Ryan Frank chooses tools after identifying repeated pain, then makes adoption easier with templates, internal SDKs, and familiar project structures.
- He recommends moving exploratory notebook work into tested, shareable project code with clear release and deployment processes.
- Integrated teams reduce translation work between data scientists, machine learning engineers, software engineers, data engineers, and product owners.

## Summary
Nathan Ryan Frank describes operationalizing machine learning through practical engineering habits and team design. He starts with tool selection: teams should understand the pain they are solving, decide whether it is recurring, and compare an external tool with a small internal solution. Templates and lightweight Python SDKs can help people adopt standards without forcing them to jump straight from familiar research workflows to an idealized process. Frank treats notebooks as useful for exploration, then recommends extracting the work into structured projects with tests, Git workflows, experiment tracking, and deployment pipelines. He also explains why tests help data scientists understand unfamiliar code and preserve context while ideas are still fresh. On team structure, he describes a successful sports prediction product built by an integrated group with shared ownership across technical and product roles. That setup reduced the amount of translation required before a model could become a working service.

## Key ideas
### Tool choices should follow recurring pain
[15:11](https://www.youtube.com/watch?v=ezWdXzhwH8A&t=911s)
Nathan Ryan Frank says teams should first identify a real problem and measure the pain or churn around it. The presence of a product that can solve a problem is not enough. He asks whether the work is a one-off or whether it will happen repeatedly. Repeated work deserves a maintainable and repeatable approach with standards, especially when people will move in and out of the system. Frank weighs bringing in an outside tool against addressing the problem internally. His starting point is the team's current skill level, the users of the platform, and the effort required to change habits. A new tool needs a clear outcome and an easy path to adoption.

### Platform standards should meet teams where they are
[17:31](https://www.youtube.com/watch?v=ezWdXzhwH8A&t=1051s)
Frank argues that platform teams should understand what users already know before imposing an ideal workflow. A research-heavy team may know basic Git practices but struggle to move directly to a fully mature process. If the change creates too much friction, people may hide work or return to familiar habits. He suggests templates that give every project a consistent structure and small internal SDKs that connect a new system to tools people already use. He also says platform adoption works better when people want the platform because its benefits are visible. Education and developer advocacy can show users how experiments, comparisons, and other workflows become easier.

### Notebook exploration needs a bridge into tested project code
[21:07](https://www.youtube.com/watch?v=ezWdXzhwH8A&t=1267s)
Frank considers notebooks useful for exploring ideas and iterating quickly. He wants the next step to be simple: extract the work into a package, module, container, or other shareable asset. A long notebook can be difficult for another person to understand because its context is tied to the original author's stream of thought. Frank recommends project structure, tests, Git practices, a branching policy, and a release process. He also wants code to move easily between workspaces and environments, with experiments and outcomes tracked along the way. The goal is to make operationalization a natural continuation of exploration rather than a separate handoff.

### Tests preserve context and teach unfamiliar systems
[29:41](https://www.youtube.com/watch?v=ezWdXzhwH8A&t=1781s)
Frank became a strong advocate for testing after working with software engineers on an ETL pipeline at Stats Perform. He had not written Java before, yet pairing and the existing tests helped him understand the code's structure and safely make changes. Tests showed examples of how enrichment classes and methods were expected to behave. He recommends adding tests while the idea is still fresh, rather than waiting until production several months later. At that point, the author may have lost the reasoning behind feature transformations or other logic. Frank also describes the immediate feedback of changing a failing test to a passing one as satisfying, which gives developers a simple reason to keep testing part of their workflow.

### An integrated product team can remove translation work
[36:13](https://www.youtube.com/watch?v=ezWdXzhwH8A&t=2173s)
Frank describes a sports prediction API built by a team that included data scientists, machine learning scientists, machine learning engineers, full-stack software engineers, a technical lead, an engineering manager, and a product owner. The roles did not work on identical tasks all the time, but they shared context and paired when appropriate. The product owner brought customer needs, while technical leads could explain the effect of a requested change on the system. Engineers learned about machine learning frameworks and model triggers, while machine learning practitioners gained software engineering experience. Frank says this arrangement reduced the gap between a model that looks promising and a service that engineering can operate. It also helped the team move faster.

### Shared vocabulary grows through embedded collaboration
[43:17](https://www.youtube.com/watch?v=ezWdXzhwH8A&t=2597s)
Frank connects machine learning team design to the way DevOps practices have moved into software engineering teams. He says engineers increasingly own their deployment pipelines and infrastructure definitions instead of handing those requests to separate teams. He sees a similar benefit when machine learning and data roles work closely with engineering and product roles. Terms such as serving and inference became easier to use once the team had built shared understanding. Frank also mentions feature stores as a communication aid because data scientists can own feature definitions through the system, although he says a tool alone does not replace working together. In his view, reducing specialization and siloed handoffs helps people understand a system end to end.

## Notable quotes
- Nathan Ryan Frank: "You got to know that you have a problem to solve first." (15:53)
- Nathan Ryan Frank: "It should be very easy for you to take that idea and operationalize it." (23:46)
- Nathan Ryan Frank: "You need to integrate testing early in the process." (31:53)
- Nathan Ryan Frank: "There was very little translation that needed to happen." (42:24)

## Tools & references mentioned
- WW Grainger
- University of North Carolina Chapel Hill
- Skynet robotic telescope network
- Strong Analytics
- Stats Perform
- MLflow
- Neptune
- Weights & Biases
- Databricks
- Tecton
- Accelerate
- The Phoenix Project
- The DevOps Handbook
- GitHub Actions
- CircleCI

## Who should watch
- You are deciding whether an MLOps tool is worth introducing, and need a way to judge recurring pain, adoption cost, and internal alternatives.
- Your machine learning work starts in notebooks and becomes difficult to test, review, reproduce, or deploy.
- Your product team loses time translating between data science, engineering, platform, and business roles during model development.

## Editor's note

Nathan Ryan Frank says notebook exploration needs a simple bridge into tested, shareable project code. ZenML turns that work into pipelines of Python steps, with each run recording its inputs, outputs, and code version. That gives teams a traceable record as experiments move into repeatable workflows, without tying the pipeline to one infrastructure setup.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [ML Stepping Stones: Challenges & Opportunities for Companies](https://mlopstalks.com/talks/ml-stepping-stones-challenges-opportunities-for-companies) (John Crousse, 47:47)
- [The Shipyard: Lessons Learned While Building an ML Platform](https://mlopstalks.com/talks/the-shipyard-lessons-learned-while-building-an-ml-platform) (Joseph Haaga, Interos, 40:00)
- [Reinvent Yourself and Be Curious](https://mlopstalks.com/talks/reinvent-yourself-and-be-curious) (Stefano Bosisio, Synthesia, 57:16)
- [Just Build It! Tips for Making ML Engineering and MLOps Real](https://mlopstalks.com/talks/just-build-it-tips-for-making-ml-engineering-and-mlops-real) (Andy McMahon, NatWest Group, 48:17)
- [Practical MLOps](https://mlopstalks.com/talks/practical-mlops) (Noah Gift, Pragmatic AI Labs, 58:36)