# Fast.ai, AutoML, and Software Engineering for ML

Jeremy Howard, Fast.ai | MLOps Coffee Sessions | Episode 47 | 57:43

Source: https://www.youtube.com/watch?v=VosGiHNV0tY
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/fast-ai-automl-and-software-engineering-for-ml
Published: 2021-07-15
Tags: deployment, developer-experience, notebooks, platform-teams

## TL;DR
- Jeremy Howard designs tools around human limits such as forgetfulness, impatience, and the need to experiment.
- He argues that useful APIs should be layered, hackable, consistent, and built from the user's experience backward.
- He wants machine learning tools to remove the need to code for most people, while warning that AutoML and infrastructure can add wasteful complexity.

## Summary
Jeremy Howard explains how his experience with notebooks, software engineering, and machine learning shaped fast.ai and nbdev. He wants tools to match how people actually work: experimenting, revising, documenting decisions, and returning to code months later. Fast.ai grew from the need to make PyTorch easier to use without losing the ability to inspect and change its internals. Howard describes APIs as layers that should build on each other while still allowing users to drop down to lower levels. He is skeptical of AutoML systems that spend large amounts of compute on searches that simple heuristics can solve. He also questions the routine use of Kubernetes and other complex infrastructure for problems that do not need it. From his experience deploying insurance pricing systems, he recommends simulation, gradual rollout, and careful reporting. He says large legacy companies often struggle to adopt data-driven work, while startups have been more receptive.

## Key ideas
### Tools should account for how people actually work
[03:59](https://www.youtube.com/watch?v=VosGiHNV0tY&t=239s)
Howard says his interest in workflow comes from being extremely lazy. He wants to write as little code as possible, avoid repeating it, spend less time debugging, and make it easier to understand decisions made six months earlier. He designs around human traits such as forgetfulness, impatience, and a preference for visual experimentation. After decades of coding, he found notebooks made the biggest difference to his productivity because they support exploration while preserving the record of how an answer was reached. He compares this to scientific journals, where the process matters rather than only the final result.

### Fast.ai combines PyTorch flexibility with higher-level abstractions
[08:08](https://www.youtube.com/watch?v=VosGiHNV0tY&t=488s)
Howard rewrote a fast.ai course around PyTorch when its pre-release appeared because he found it much better for exploring and implementing research. PyTorch alone created too much boilerplate for the introductory course, so he built something Keras-like on top of it. He wanted more than Keras offered: fast.ai had to remain hackable enough for research and teaching, while applying software engineering principles such as decoupling and layering. The result was intended to let practitioners build, test, deploy, and maintain models without losing access to lower-level code.

### A layered API should let users go deeper without replacing its foundation
[16:08](https://www.youtube.com/watch?v=VosGiHNV0tY&t=968s)
Howard distinguishes layers from simple boxes drawn around steps. Each layer should build on the one below it, while users should be able to change a small part of a lower layer without replacing the whole foundation. He uses AWS interface boundaries as an example of components working through well-defined contracts. In fast.ai, he builds higher-level APIs on mid-level APIs, which are built on lower-level APIs and PyTorch. Because he uses the layers himself, an irritating mid-level interface exposes a design problem. He also says consistency matters across tasks, so vision, tabular, collaborative filtering, and NLP use similar high-level code.

### Good API design often requires repeated rewrites and working backward from documentation
[30:09](https://www.youtube.com/watch?v=VosGiHNV0tY&t=1809s)
Howard says the data blocks API went through 25 rewrites. He keeps revising until he cannot see how to improve it, even though this frustrates collaborators. The redesign began when fast.ai had separate classes for combinations such as image regression, image classification, and bounding boxes. He stepped back and identified recurring operations: obtaining a data source, converting it into inputs and targets, splitting training and validation data, and batching it for a model. He also practices documentation-driven development by writing the README or documentation notebook first, then filling in the API needed to make the intended user experience work.

### AutoML should reduce coding without replacing simple, efficient methods with compute
[30:00](https://www.youtube.com/watch?v=VosGiHNV0tY&t=1800s)
Fast.ai's stated goal is to make deep learning accessible, and Howard says that requires eventually removing the need to code for most people. He is critical of AutoML approaches that mainly use large compute budgets for neural architecture searches or hyperparameter sweeps. In his view, a learning-rate finder can solve some choices in seconds, and known heuristics can guide batch-size decisions. He distinguishes this from the work Abhishek is doing with Hugging Face AutoML, which he describes as useful for end users with limited expertise. Howard is also uncertain about Copilot because it writes code for users but can produce poor code.

### Complex infrastructure should match the problem rather than become a default
[41:08](https://www.youtube.com/watch?v=VosGiHNV0tY&t=2468s)
Howard argues that many teams use infrastructure designed for a small number of companies with unusually large-scale needs. He says he has sometimes replaced systems involving Kafka or Kubernetes with a 10-line Python script. Complex abstractions create costs when they fail, because users may struggle to understand the cause or detect a configuration that consumes far more resources than needed. His criticism is conditional: these tools can be useful when the problem genuinely requires them. His concern is that people often adopt them without needing their scale or complexity.

### Production ML needs simulation, gradual rollout, and careful reporting
[50:51](https://www.youtube.com/watch?v=VosGiHNV0tY&t=3051s)
Howard describes deploying systems at Optimal Decisions that set insurance prices for major insurers. His team used simulation, gradual rollout, and careful reporting because they could not predict every outcome in advance. He says a system can be improved iteratively when its rollout is controlled and its effects are tracked. He also reflects on years spent helping large companies adopt data-driven decision-making. Senior leaders often supported the effort, but legacy companies still struggled to shift their culture because management was shaped by backgrounds in areas such as marketing, accounting, legal work, and domain expertise. He found startups more receptive to this kind of change.

## Notable quotes
- "My interest in workflow comes from my extreme laziness." (03:59)
- "We should endeavor to do the same thing with our APIs." (16:46)
- "We're clearly failing at our mission." (30:58)
- "I just want the bloody problem solved." (35:58)
- "The other thing I'd say is, I did, you know, the biggest stuff I've deployed was that Optimal Decisions, which was my insurance pricing startup." (46:43)

## Tools & references mentioned
- fast.ai
- nbdev
- PyTorch
- TensorFlow
- Keras
- Mathematica
- AWS
- EC2
- Kaggle
- AutoML
- Hugging Face
- Copilot
- GPT
- ImageNet
- Kubernetes
- Kafka
- Jupyter notebooks
- pytest
- Sphinx
- PyPI
- Discord
- Optimal Decisions
- EfficientNet

## Who should watch
- You are building an ML library or internal platform and need a way to balance simple APIs with access to lower-level code.
- Your team is deciding whether to adopt AutoML or complex infrastructure, and you want a skeptical view of compute and abstraction costs.
- You deploy models into business processes where mistakes are expensive and need practical advice on simulation, staged rollout, and reporting.

## Editor's note

Jeremy Howard says notebooks matter because they preserve how an answer was reached, making it easier to understand decisions months later. ZenML records each pipeline run's steps, inputs, outputs, and code version, so a model or artifact can be traced back to the data and code that produced it. That gives experimental work a durable record beyond the notebook.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [A Playground for AI Engineers](https://mlopstalks.com/talks/a-playground-for-ai-engineers) (Paulo Vasconcellos, Hotmart, 54:42)
- [The Future of User Interfaces: AI is Changing Everything](https://mlopstalks.com/talks/the-future-of-user-interfaces-ai-is-changing-everything) (, 27:33)
- [Hard Learned Lessons from Over a Decade in AI](https://mlopstalks.com/talks/hard-learned-lessons-from-over-a-decade-in-ai) (Mike Del Balso, Tecton, 48:43)
- [Building an ML Platform from Scratch: Live Coding Session](https://mlopstalks.com/talks/building-an-ml-platform-from-scratch-live-coding-session) (Alon Gubkin, Aporia, 1:57:24)
- [The Future of ML and Data Platforms](https://mlopstalks.com/talks/the-future-of-ml-and-data-platforms) (Michael Del Balso, Tecton, 55:17)