Podcast

The Journey from Data Scientist to MLOps Engineer

Ale SolanoEpisode 80 · 41:30 · Feb 2022 · 1,489 views
Thumbnail for The Journey from Data Scientist to MLOps Engineer Watch on YouTube
TL;DR
  1. 1

    Ale Solano moved toward MLOps after seeing repeated proof-of-concepts fail to become products that people used.

  2. 2

    His team shortened experiments by defining the problem and its metric, deploying early, and iterating through continuous integration and continuous delivery.

  3. 3

    Ale argues that machine learning teams need software and product practices, while tools should reduce the need for every person to learn the entire stack.

Summary

Ale Solano describes how he moved from data science toward MLOps after years of building proof-of-concepts that produced no business value. He worked at a content creation company that wanted to automate images and other marketing assets, but his work initially happened separately from the product team. The turning point came when he saw software teams release value every two weeks and wanted the same feedback cycle for machine learning. He began using Docker, Amazon, data and model version control, Terraform, and CI/CD pipelines for training and inference. Later, a product manager connected the machine learning team to the wider product cycle, with support from a DevOps engineer and a teammate who translated product needs into testable tasks. Ale also explains why machine learning needs room for experiments, while still starting from a real problem, a metric, and a benchmark. He left his company temporarily to study software architecture, cloud computing, networking, microservices, and computer science.

Key ideas
05:57

Repeated proofs of concept pushed Ale toward MLOps

Ale had joined a content creation company that wanted to automate the production of images, videos, and text for marketing campaigns. He worked through one proof of concept after another while the company built a broader software product. The work stayed separate from the product process, and he felt that his models were being discarded rather than used. He describes the personal cost as wasting time and losing work. The change began gradually when he realised that he had worked for years without building something used by people. That experience led him to ask how machine learning could become part of a useful product.

10:06

Machine learning teams need to learn how to create product value

Ale says his advice to his younger self would be to learn computer science and software, then use those skills to help create something useful. He does not think a data scientist should commit only to machine learning. Companies may run machine learning projects as exploratory bets, but individuals need a path from their work to a product. In his experience, teams often shared an academic mindset. They focused on interesting papers and models while giving less attention to customers, products, and business value. For Ale, MLOps addresses this gap by helping an experimental field work inside a product cycle.

21:46

Fast delivery starts with a problem, a metric, and a benchmark

When asked where a new practitioner should start, Ale puts problem definition before model selection. The team should state what problem it is solving, decide how to measure it, and establish a benchmark. It should then check whether ordinary software could solve the problem before adding machine learning. Once the team knows what is valuable, machine learning becomes a way to improve an existing measure rather than an excuse to explore without direction. Ale says this removes a large amount of unnecessary exploration and makes it easier to judge whether a model is useful.

18:04

Deployment early changed the team's development cycle

Ale first worked alone on making his machine learning work iterable. He put the work into a Docker image, uploaded it to Amazon, and tried to version data and models. The work was difficult for one person, but it taught him where the technical problems were. The larger change came after a product manager reorganised the team and brought machine learning into the product cycle. The team decided to test ideas quickly and deploy early. Instead of waiting months for a proof of concept, they deployed, created a repository, and iterated from there.

23:45

CI/CD connected experiments to usable training and inference

A DevOps engineer helped the team make its work deployable by the following Friday. They created CI/CD for a development stage and split the work into a training pipeline and an inference pipeline. Changes were built and uploaded automatically, then the training pipeline ran through CI/CD. Terraform was part of the setup. Ale valued the immediate feedback: after an experiment, the team could see whether accuracy had gone up or down and whether the development had improved. He describes a one-click path from a change to a visible result.

25:11

Product roles and communication made the team more effective

The team grew to four people, including developers, a DevOps engineer, and a teammate who acted as a small product manager for machine learning. She connected the machine learning team with the product manager for the main software product and translated ideas into tasks. A product owner checked with customers to confirm that the team was pursuing the right goal. Ale says this communication made it possible to turn ideas into things the team could test and deploy within two weeks. The technical setup helped, but the product connection gave the work a direction.

27:33

Agile work can include machine learning exploration

Ale supports an iterative approach even though machine learning begins with experiments whose outcomes are unknown. He says teams should keep the problem they want to solve in mind and treat machine learning as one possible technology. That question changes the data work: the team asks whether the available data can help solve the problem instead of exploring data without a decision in mind. Ale acknowledges that machine learning is built around experiments as well as code commits. He believes the surrounding tools can account for this difference while still supporting short delivery cycles.

33:47

Ale chose software engineering to fill his own knowledge gaps

Rather than immediately learn another machine learning tool, Ale temporarily stepped away from machine learning to study system architecture, cloud computing, networking, microservices, and software design. He had identified gaps in his knowledge and wanted to understand software well before applying the same thinking to machine learning. He does not expect every data scientist to make the same choice. In his view, teams need different strengths, with data scientists focused on models and DevOps practitioners focused on automation. Tools can reduce the amount that each person must know, although Ale personally wants a wider technical range.

"The first thing is to know what you're trying to do, know what is valuable, and the second thing is to measure, and the third thing is to have a benchmark."21:46
Who should watch
  • You have built several machine learning proofs of concept, but none has become a product used by customers.
  • Your team needs a practical starting point for connecting model work to product goals, deployment, and feedback.
  • You are moving from data science into machine learning engineering and want a study path that includes software engineering.