# PyTorch: Bridging AI Research and Production

Dmytro Dzhulgakov, Facebook | MLOps Coffee Sessions | Episode 63 | 52:55

Source: https://www.youtube.com/watch?v=zxd-YUbDITY
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/pytorch-bridging-ai-research-and-production
Published: 2021-11-16
Tags: deployment, open-source, platform-teams, training-pipelines

## TL;DR
- Dmytro Dzhulgakov says PyTorch gained adoption by making model development usable, flexible, and close to ordinary Python programming.
- PyTorch is expanding beyond model code by improving connections to data loading, distributed training, experiment tracking, deployment, and cluster infrastructure.
- Dmytro argues that research and production form a continuous funnel, so teams should optimize promising parts of a workload incrementally instead of converting the whole model at once.

## Summary
Dmytro Dzhulgakov discusses how he moved from infrastructure work at Facebook into machine learning tooling and became a technical lead for PyTorch. He explains that PyTorch initially prioritized researchers and modelers because rapid experimentation drives progress in AI. Its Python-based, flexible design made it easier to try new ideas, debug code, extend the framework, and build community projects. As adoption grew, PyTorch had to support more users, larger models, distributed training, hardware vendors, and production systems. Dmytro describes work on the interfaces around model code, including data ingestion, job launching, experiment metrics, serving, and integration with schedulers and cloud platforms. He is candid about an early strategic mistake: treating research and production as separate frameworks and attempting wholesale model conversion. He now favors keeping flexibility across the workload while specializing smaller parts for performance. He also expects the boundary between software engineering and machine learning engineering to keep becoming less distinct.

## Key ideas
### Dmytro entered machine learning tooling through infrastructure work
[01:06](https://www.youtube.com/watch?v=zxd-YUbDITY&t=66s)
Dmytro Dzhulgakov began with a systems and infrastructure background, working on indexing and serving for Facebook's recommendation systems. Working alongside machine learning teams led him to learn about features, models, training data, and model services. He was drawn to developer tooling because improving tools can make many people more productive. His work expanded across training data generation, feature engineering, model training, and production applications of deep learning. Around 2017, he joined PyTorch while it was still developing from a research-oriented project into a production framework.

### A PyTorch technical lead keeps a large open-source project coherent
[04:48](https://www.youtube.com/watch?v=zxd-YUbDITY&t=288s)
Dmytro says his role changed as PyTorch grew and attracted contributions from Facebook, vendors, cloud providers, academia, and the wider community. He now spends less time writing code and more time on project direction, code reviews, GitHub discussions, and conversations with researchers and machine learning engineers. He looks after the core abstractions, extension mechanisms, and overall user experience. The challenge is allowing many groups to add features without making PyTorch feel like a collection of unrelated parts.

### Machine learning infrastructure has to scale across several dimensions
[11:29](https://www.youtube.com/watch?v=zxd-YUbDITY&t=689s)
Dmytro describes scaling as more than handling larger models and datasets. Hardware capabilities, model sizes, and training workloads keep changing, which creates a continuing search for the next bottleneck. Developer count creates another kind of scale. A system that works for five machine learning teams cannot rely on the same hands-on enablement when dozens or thousands of developers use it. Facebook moved from one-off efforts that brought individual deep learning applications into production toward self-service platforms and reusable infrastructure.

### PyTorch first optimized for researchers because experimentation drives AI progress
[17:23](https://www.youtube.com/watch?v=zxd-YUbDITY&t=1043s)
Dmytro attributes PyTorch's early success largely to usability. AI has frequent changes in model types and techniques, so researchers need to test ideas quickly. PyTorch prioritized modelers with a flexible Python programming experience, even though it was not initially the fastest framework. That flexibility also helped system optimizers and application developers because they could debug, extend, and customize their workloads. Community members could build projects on top of PyTorch for many domains and production environments.

### The framework uses a stable core with extension points around it
[22:44](https://www.youtube.com/watch?v=zxd-YUbDITY&t=1364s)
Dmytro explains that PyTorch has to keep its core stable, compatible, and polished while still allowing newer ideas to develop quickly. The project focuses on primitives and extension points so users can add operators, connect data sources, support vendors, or experiment with new techniques without changing the framework's center. When an approach becomes mature and broadly useful, it can move into the main PyTorch packages. He gives mixed precision training as an example of a technique that began in community and NVIDIA-related work before becoming available directly in PyTorch.

### PyTorch is widening its interfaces to the surrounding production system
[28:20](https://www.youtube.com/watch?v=zxd-YUbDITY&t=1700s)
Dmytro divides the research-to-production problem into model optimization and integration with the wider machine learning pipeline. PyTorch work includes quantization, sparsity, framework performance, distributed training, and newer parallelism approaches. It also needs interfaces for data ingestion, launching jobs on different clusters, saving intermediate results, reporting experiment metrics, and deploying models on servers or devices. He mentions integrations with MLflow, Weights & Biases, Kubernetes, Vertex AI, and SageMaker, along with community work on serving and data loading.

### Whole-model conversion failed because real workloads contain arbitrary code
[38:43](https://www.youtube.com/watch?v=zxd-YUbDITY&t=2323s)
Dmytro describes an early strategy of using separate research and production frameworks, with a conversion step between them. That approach worked for standard models, but became difficult as use cases multiplied. Real workloads include dynamic shapes, control flow, preprocessing, and components such as beam search. The lesson was to preserve general programming flexibility across the whole environment while applying specialized optimization where it matters. A large training script may spend most of its compute in a small number of modules, so those parts can be optimized without forcing the entire workload through a separate framework.

### Research and production should be treated as a continuous funnel
[44:02](https://www.youtube.com/watch?v=zxd-YUbDITY&t=2642s)
Dmytro rejects the idea that research and production are two isolated worlds. Ideas and models move through a funnel, from many experiments to a smaller set of promising workloads and eventually to a deployed system. The tooling should make each transition easier. Teams can keep a working model in the same environment while specializing selected components, rather than converting everything wholesale. This also gives hardware vendors room to optimize specific workloads while retaining PyTorch as the broader development environment.

### Machine learning work is becoming part of ordinary software engineering
[47:46](https://www.youtube.com/watch?v=zxd-YUbDITY&t=2866s)
Dmytro expects the boundary between machine learning engineers and software engineers to keep blurring. In production teams, people often already combine data work, model development, systems integration, and deployment. He sees more software engineers using data-driven methods where they previously used hand-written heuristics. PyTorch should support both advanced practitioners pushing research forward and engineers who need higher-level building blocks for applying machine learning in a domain. He does not predict that everyone will become a machine learning engineer, but expects substantially more engineers to work with machine learning.

## Notable quotes
- "I think what happened is that there are different roles playing in this process." (17:43)
- "The usability really drove the success." (20:20)
- "You don't want to take your workload converted to something else wholesale." (42:07)
- "You should think of it as how do I make this journey of starting with a thousand ideas and maybe trying hundreds of them, and maybe ten of them are promising, easier to optimize." (45:02)

## Tools & references mentioned
- Facebook
- Meta
- PyTorch
- ONNX
- Caffe2
- TensorFlow
- NVIDIA
- MLflow
- Weights & Biases
- Kubernetes
- Vertex AI
- SageMaker
- Airflow
- fast.ai
- Jeremy Howard

## Who should watch
- You are building tools around PyTorch and need to balance a stable core with community extensions.
- Your team is trying to move experimental models into production across different clusters, clouds, or hardware platforms.
- You want practical guidance on structuring the relationship between general-purpose model code and targeted performance optimization.

## Related talks

- [How to Optimize Large AI Models with PyTorch](https://mlopstalks.com/talks/how-to-optimize-large-ai-models-with-pytorch) (Michael Gschwind, Meta Platforms, 57:44)
- [Building a Python-Centric Feature Platform to Power Production AI Applications](https://mlopstalks.com/talks/building-a-python-centric-feature-platform-to-power-production-ai-applications) (Matt Bleifer, Tecton, 27:11)
- [Productionizing AI: How to Think From the End](https://mlopstalks.com/talks/productionizing-ai-how-to-think-from-the-end) (Annie Condon, 11:11)
- [Tecton Round-table // Get your ML Application Into Production](https://mlopstalks.com/talks/tecton-round-table-get-your-ml-application-into-production) (Kevin Stumpf, Derek Salama, Eddie Esquivel & Isaac Cameron, Tecton, 55:42)
- [Just Build It! Tips for Making ML Engineering and MLOps Real](https://mlopstalks.com/talks/just-build-it-tips-for-making-ml-engineering-and-mlops-real) (Andy McMahon, NatWest Group, 48:17)