# Maturing Machine Learning in Enterprise

Kyle Gallatin, Etsy | MLOps Coffee Sessions | Episode 43 | 47:09
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=kfm3Iozxj8I
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/maturing-machine-learning-in-enterprise
Published: 2021-06-15
Tags: governance, model-serving, observability, platform-teams

## TL;DR
- Machine learning infrastructure is software infrastructure that supports ML tools at the scale required by a specific use case.
- Organizations need governance, observability, and control because deploying a model is easier than managing many models and their endpoints.
- Platform teams should design around real user cases instead of assuming that data scientists have particular skills or needs.

## Summary
Kyle Gallatin describes his move from data analysis and data science into machine learning infrastructure, including the software engineering skills he had to learn along the way. He explains how ML teams moved from isolated proof-of-concept work toward MLOps, and why the next stage involves governance, observability, and visibility. The hard part is often applying existing software practices to ML-specific problems such as drift, model serving, and data access. Kyle argues that vendors should solve common requirements while allowing organizations to build the remaining specialized pieces through integrations. He also discusses how platform teams should support data scientists with different levels of interest in engineering. For model serving, the basic endpoint is relatively easy, while managing dozens of models, permissions, resources, and costs creates operational debt. Throughout the conversation, he returns to organizational maturity: companies need a clear use case, appropriate support, and a platform shaped by real users rather than by fashionable tools.

## Key ideas
### Moving from data science to infrastructure requires rebuilding computer science foundations
[00:43](https://www.youtube.com/watch?v=kfm3Iozxj8I&t=43s)
Kyle Gallatin says his path began with a molecular and cellular biology master's degree, followed by a data analyst job where he taught himself R and Python. After a boot camp and work as a data scientist and machine learning engineer at Pfizer, he moved into software engineering for ML infrastructure at Etsy. He had to study data structures, algorithms, and other computer science basics on his own because his earlier education had not covered them. He describes learning advanced data science techniques while going back to what felt like CS101, which gave him the base for writing stronger production code.

### Python is practical for ML work, while YAML shapes much of the surrounding system
[04:36](https://www.youtube.com/watch?v=kfm3Iozxj8I&t=276s)
For a new SaaS application with substantial ML services, Kyle would probably choose Python because it is widely supported, quick to write, and easy to test. He also uses Scala when performance matters, although he finds it more difficult. He jokes that he would write more YAML than anything else because so many tools depend on templated configuration. For people learning MLOps, he recommends starting with Docker Compose, then moving toward Kubernetes. He also advises using editor plugins, such as a VS Code indentation plugin, rather than struggling with formatting by hand.

### MLOps is moving toward governance and visibility
[10:08](https://www.youtube.com/watch?v=kfm3Iozxj8I&t=608s)
Kyle describes an early period when data science teams were isolated and produced proof-of-concept projects that were hard to move into production. MLOps grew in response to the difficult handoff between data scientists and software engineers, who often used different language and had different expectations. He thinks the field is moving beyond deployment tools toward governance, with control over ML operations and better observability. He says visibility is especially important because teams need to understand how their processes run and how they manage them.

### Existing software practices help, but ML still has unresolved monitoring problems
[14:02](https://www.youtube.com/watch?v=kfm3Iozxj8I&t=842s)
Kyle says much of the technology needed for ML operations already exists in software engineering, so the daily problem is often choosing how to apply it to ML-specific work. Data drift, model drift, and serving a stateless model at scale create additional considerations. He agrees that the field still lacks clear answers about the most effective ways to monitor drift and model behavior. Even when a team can implement several possible approaches, it may not know which one fits its use case best.

### Platforms should solve common needs while leaving room for specialized requirements
[17:00](https://www.youtube.com/watch?v=kfm3Iozxj8I&t=1020s)
Kyle does not expect one platform to fit every industry or organization. He points to integration-first products as a useful pattern: a vendor can solve most common requirements while allowing a company to build the remaining requirements around its own systems. He warns that trying to build for every possible use case can cause a product to fail. Flexibility and integrations matter because healthcare, consumer products, and other domains have different constraints.

### Platform teams need to meet data scientists where they are
[18:48](https://www.youtube.com/watch?v=kfm3Iozxj8I&t=1128s)
Kyle's interest in platform engineering grew after repeated experiences taking models to production and finding no common process for handing them to infrastructure teams. He now thinks of data scientists as the platform team's customers. The team needs to understand what skills and inputs it can expect, then provide the support required to get models into production quickly. Data scientists differ widely: some want to hand over a serialized model and stop, while others want to learn Kubernetes and work directly with platform engineers. A useful platform supports both groups.

### Model serving is easy at the endpoint and difficult at the organizational scale
[24:31](https://www.youtube.com/watch?v=kfm3Iozxj8I&t=1471s)
Kyle enjoys model serving because the result is immediate: an API receives a request and returns a result. He says the basic technical task can be straightforward, such as training a TensorFlow model, putting it in a Docker image, and exposing an endpoint with TensorFlow Serving. The difficult work begins when an organization has many models. Teams need to track models, manage endpoints, control access, and avoid wasting resources. Without that governance, technical debt can grow until the system becomes unmanageable.

### The right tool follows the problem, rather than the other way around
[42:04](https://www.youtube.com/watch?v=kfm3Iozxj8I&t=2524s)
Kyle says teams should question whether a tool fits their actual scale and use case. His team tried Kubeflow and found that it was more difficult to manage than the value it provided for a small data science group. A standalone MLflow deployment was enough for that situation. Other organizations may have daily workflows that require a more complex orchestrator. He also explains that estimates are difficult in ML because model performance is unknown before training, so teams should break work into smaller pieces and estimate the parts they understand.

## Notable quotes
- Kyle Gallatin: "Machine learning infrastructure is software infrastructure, you know, it's all of the support that you need to actually support your ML software tooling at scale for your use case." (42:54)
- Kyle Gallatin: "The technical aspect of serving is going to be easy. Managing that and actually getting to that governance level of how do you manage if you have 50 models, how do you keep track of those and how do you manage those endpoints, make sure you're not wasting resources, being cost effective, maybe role-based access controls, all of that stuff is the real stuff that you have to watch out for in that space." (26:58)
- Kyle Gallatin: "Building a platform for data scientists, maybe you assume that they have some skill they don't, or want some feature that they don't, or maybe it's not important at all, but you really need to get those use cases and understand them to actually correctly map out those requirements." (33:44)
- Kyle Gallatin: "I think the next big thing is going to be governance. I think it's going to be machine learning in the enterprise, actually, like people seeing that it's driving value for certain companies." (36:33)

## Tools & references mentioned
- Etsy
- Pfizer
- Python
- R
- Scala
- YAML
- Docker Compose
- Kubernetes
- VS Code
- Fiddler
- OpenAI
- GPT-3
- Seldon
- TensorFlow Serving
- Kubeflow
- MLflow

## Who should watch
- You are building an ML platform and need to decide what belongs in the platform team versus the data science team.
- Your organization is choosing model-serving or orchestration tools before it has defined the scale and use cases they must support.
- You work with regulated or sensitive data and need to balance controlled access with a usable workflow for data scientists.

## Editor's note

Kyle Gallatin says the hard part of model serving begins when an organization has many models and must manage endpoints, permissions, and resource use. ZenML provides a model registry with lineage, so each model can be traced to the data and code that produced it. That gives platform teams a record to use when managing models at organizational scale.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Scaling ML Capabilities in Large Organizations](https://mlopstalks.com/talks/scaling-ml-capabilities-in-large-organizations) (Bertjan Broeksema & Axel Goblet, BigData Republic, 1:02:47)
- [A Journey in Scaling ML](https://mlopstalks.com/talks/a-journey-in-scaling-ml) (Gabriel Straub, Ocado Technology, 52:41)
- [Designing ML Infra for ML & LLM Use Cases](https://mlopstalks.com/talks/designing-ml-infra-for-ml-llm-use-cases) (Amritha Arun Babu, Klaviyo & Abhik Choudhury, IBM, 1:00:18)
- [Reliable Machine Learning](https://mlopstalks.com/talks/reliable-machine-learning) (Niall Murphy, Stanza Systems & Todd Underwood, Google, 1:02:25)
- [The Future of ML and Data Platforms](https://mlopstalks.com/talks/the-future-of-ml-and-data-platforms) (Michael Del Balso, Tecton, 55:17)