Podcast

MLOps Memes

Ariel Biller, ClearMLEpisode 37 · 1:00:32 · Apr 2021 · 615 views
Thumbnail for MLOps Memes Watch on YouTube
TL;DR
  1. 1

    Ariel Biller uses memes to test his own understanding of MLOps, explain its problems, and make technical jargon easier to discuss.

  2. 2

    Ariel argues that MLOps is still in a stone age, with many subcultures developing separate practices before the field has agreed on useful abstractions.

  3. 3

    For research and development, Ariel reduces MLOps to reproducibility, tracking, automation, orchestration, and easy integration, while warning that extra tools should remain abstractions over existing systems.

Summary

Ariel Biller explains how a background in quantum chemistry, distributed computing, deep learning, and startup data science led him to evangelism at ClearML. He began making memes inside his company and continued because they helped him vent, check his understanding, teach presentations, and comment on the MLOps ecosystem without making every statement sound literal. The conversation then moves from humor to Ariel's view of the field. He compares MLOps to a stone age in which separate subcultures have their own practices and tools. He expects the ecosystem to change through better abstractions, though he does not think a single complete platform can cover every use case. For MLOps in research and development, he names reproducibility, tracking, automation, orchestration, and easy integration. He also describes provenance as a long-term goal, with models connected to data, code, training, metrics, and other identifiers. Ariel is direct about the field's jargon, marketing, and unresolved problems.

Key ideas
04:28

Memes let Ariel test ideas and teach without softening them

Ariel started making memes in an Israeli machine learning and data science meme group, then continued after leaving Allegro AI. He says memes helped him vent about a ridiculous field, check what he understood about MLOps, and teach ideas in presentations. Their ambiguity gives him room to make an extreme statement while leaving readers unsure whether he is joking. That lets a meme such as "MLOps is DevOps with extra steps" work differently for people coming from DevOps and people working in MLOps. He later learned to match a meme to a post or discussion almost immediately.

10:39

MLOps is still a collection of separate subcultures

Ariel compares MLOps with a stone age in which different subcultures have developed their own practices without a shared system connecting them. He says machine learning has not yet reached the point where one broadly copied practice provides the benefits of a common standard. That explains the steady stream of companies presenting a solution to one engineering problem and the many competing descriptions of how to operationalize machine learning. Ariel expects a Cambrian explosion of MLOps companies until a more general formula or abstraction appears. He also imagines that, once the work becomes fluid and automatic, some of MLOps could become part of DevOps.

14:25

The field needs a social language before its practices can settle

Ariel says he wants more people to make MLOps memes because shared jokes indicate that a community has consolidated enough practices to talk about them casually. He points to recurring jokes about MLOps being DevOps with extra steps and about feature stores appearing everywhere. Demetrios Brinkmann adds that this shared language can make a technical community more accessible to engineers who do not know its jargon or inside jokes. Ariel agrees that MLOps should keep asking whether a process has been over-engineered. For him, the community is useful when it attacks complicated processes and asks whether people built them because they were needed or because they could be built.

21:51

Marketing often turns ordinary capabilities into MLOps vocabulary

Ariel is skeptical of the way vendors use terms such as MLOps platform and feature store. He describes a product that began as an experiment tracking platform with orchestration and automation, then could be described as an MLOps solution after more capabilities were added. He says the word "store" can make an existing database, model registry, or evaluation callback sound like a new product category. His feature-store memes capture the pattern of mentioning a feature store and attracting a vendor response. Ariel's complaint is about jargon replacing a description of the problem an engineer actually needs to solve.

17:31

A complete end-to-end platform cannot cover every machine learning use case

Ariel says MLOps differs from DevOps because data creates fractal complexity and because production use cases have different requirements. Recommender systems, time-series analysis, and object detection can each lead to a different stack. He connects this variety to the failure of the first generation of end-to-end platforms, including the kind of platform his company previously built. His explanation is simple: a platform cannot be end to end when there are so many different ends. YAML can connect components, but writing configuration is not the same as composing a flexible system that works across many use cases.

44:26

Research and development needs a small set of lasting capabilities

For MLOps in research and development, Ariel identifies reproducibility, tracking everything, automation, orchestration, and easy integration. Reproducibility means being able to repeat work. Automation means that after doing something once or twice, a person should be able to do it again without touching every step. Orchestration means sending work to a remote machine and having it behave as it did locally. Integration matters because people will avoid a system that requires opening their code and adding decorators everywhere. Data versioning, model registries, production features, and feature stores can be added later, often as abstractions over databases already in use.

48:49

Provenance requires a shared identity structure

Ariel describes provenance as the ability to query a model and learn who created it, which data version it saw, how long it trained, and what its quality metrics were during testing and in live use. He calls this the MLOps "holy grail," while admitting that the field is not there yet. The difficulty is that each part of the system has its own identifier, including the model, dataset, pipeline job, hyperparameters, and Git commit. Until the community agrees how these identifiers form a connected tree, a clean provenance abstraction will remain difficult. He expects standardization eventually, perhaps through a domain-specific language that turns declarative research code into an MLOps process.

"The only thing you need to worry at your company is how you implement this one layer between that and the other stuff."57:40
Who should watch
  • You are trying to explain MLOps to engineers who dislike vendor language and need a more direct way to discuss engineering problems.
  • Your team is choosing between MLOps tools and wants to separate lasting capabilities from product categories built around terms such as "store" and "platform."
  • You work on research infrastructure and need reproducibility, tracking, automation, remote orchestration, and provenance without forcing researchers to become operations specialists.