# MLOps Engineering Labs Recap, Part 2

Laszlo Sranger & Artem Yushkovsky, Neuro & Paulo Maia, Nilgai | MLOps Coffee Sessions | Episode 31 | 1:04:16
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=iN8aC1BYl5A
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/mlops-engineering-labs-recap-part-2
Published: 2021-03-02
Tags: experiment-tracking, model-serving, orchestration, platform-teams

## TL;DR
- Team 3 built an NLP application that classifies Yelp restaurant reviews as positive or negative, with a Streamlit dashboard and an MLflow-backed deployment workflow.
- Artem Yushkovsky argues that an MLflow model registry should not be the central trigger for production pipelines because teams work with code and data pipelines, not models alone.
- The participants found that MLflow is useful for experiment tracking, while Kubernetes and end-to-end production workflows require more specialized engineering knowledge.

## Summary
Team 3 describes how it built and deployed an NLP application during the MLOps Engineering Labs. The project uses the Yelp polarity review dataset, a PyTorch model, an MLflow registry, a model-serving REST API, a proxy that records user feedback, and a Streamlit interface. Artem Yushkovsky explains a model operator that watches MLflow and redeploys a model in Kubernetes when its production tag changes. He also says this trigger belongs closer to code or pipeline changes in a real system. The group discusses difficulties with MLflow auto-logging, model pickling, network access, dependencies, and the limits of MLflow as an end-to-end platform. Laszlo Sranger describes the management challenge of a volunteer team choosing and changing its problem. The conversation ends with views on Kubernetes skills, engineering culture, cloud credits, better lab guidance, and the need to control infrastructure costs before adding monitoring or retraining.

## Key ideas
### Team 3 built a Yelp review classifier with a feedback dashboard
[08:51](https://www.youtube.com/watch?v=iN8aC1BYl5A&t=531s)
Paulo Maia says the team moved from its first dataset to the Yelp polarity review dataset after finding the original topic uncomfortable. The application classifies restaurant reviews as positive or negative, with positive defined as three stars or more. A user enters a review and also records whether they believe it is positive or negative. The system displays the model's prediction and saves statistics about the interaction. The model runs behind a REST API, while a Streamlit dashboard provides the user interface. This gave the group a concrete product around which to build its MLflow and PyTorch workflow.

### The backend used a model proxy and a model operator
[10:28](https://www.youtube.com/watch?v=iN8aC1BYl5A&t=628s)
Artem Yushkovsky describes Streamlit as the facade for a backend made from several small components. The model is exposed through a REST API. A model proxy communicates with the model inside the Kubernetes cluster while collecting the input, output, and feedback needed for prediction statistics. The team also wrote a model operator that synchronizes the MLflow state with the model deployed in Kubernetes. It watches the model registry, detects a change to the production tag, and redeploys the selected model. Artem compares this to a GitOps-style workflow for MLflow, although he questions whether the registry should drive production pipelines in a real use case.

### The model registry is convenient for a demo, but code is a better production trigger
[13:53](https://www.youtube.com/watch?v=iN8aC1BYl5A&t=833s)
Artem says the MLflow-centered workflow is visual and interactive, so it works well for the demonstration. He would not make a model registry the central component that triggers other pipelines in a real system. The other team placed a Git registry at the center, where changes could trigger data or model pipelines. Artem thinks that fits engineering work better because teams work with code. Dimi's separated frontend and backend also followed this practical division: Artem could deploy the model and provide an API while Dimi built the interface independently.

### The volunteer team changed its problem after reviewing the data
[16:57](https://www.youtube.com/watch?v=iN8aC1BYl5A&t=1017s)
Laszlo Sranger explains that the team had no manager with formal authority, so the group had to choose its own problem through Slack discussions. They initially considered a biased-language dataset. After Laszlo downloaded it and saw frequent offensive language, he raised the concern that the team would be uncomfortable working with it for weeks. They switched to the Yelp dataset. The replacement still gave them an NLP problem and enough data to require more than a simple scikit-learn solution. Laszlo used the lab to study how a team can move toward production-oriented machine learning and how quickly members can learn unfamiliar skills.

### The team wanted monitoring and retraining, but resisted adding components without a use
[22:22](https://www.youtube.com/watch?v=iN8aC1BYl5A&t=1342s)
The group considered monitoring incoming user data, using the saved interactions to detect changes in its distribution. Paulo Maia also suggests a feedback loop that could use user responses to retrain and redeploy the model when predictions become worse. He would add explainability so a more complex model could tell users why it classified a review as positive or negative. Laszlo Sranger pushes back on adding components just because they are available. Since the model has relatively low churn, user data may not accumulate quickly enough to justify retraining. Each extra component brings implementation, maintenance, and infrastructure costs.

### MLflow exposed practical limits around tracking and model packaging
[28:25](https://www.youtube.com/watch?v=iN8aC1BYl5A&t=1705s)
Artem says the training environment could not access the MLflow server over the internet, so the team trained the model first and uploaded the code version, metrics, and artifacts afterward. Paulo had started with an LSTM model using a custom data loader, but the model could not be pickled in MLflow. The team replaced it with a simpler model and used PyTorch's built-in dataset instead of a preprocessed JSON input. Dimi also found MLflow's PyTorch integration and auto-logging difficult to use because of version constraints. The group concludes that MLflow is useful for tracking, while its end-to-end production role was less convincing.

### Reproducible environments matter more than notebook convenience
[34:52](https://www.youtube.com/watch?v=iN8aC1BYl5A&t=2092s)
Laszlo recommends avoiding notebooks where possible because users can install packages into a shared notebook server and then send a model to production without knowing the full environment. His team runs code from Python scripts and recreates virtual environments from repository requirements. They pin dependency versions so another person can create the same environment on a new machine. Artem adds that training and inference should use the same base image. Even with the same Dockerfile, he has seen a system work in one cloud and fail with segmentation faults in another. The team mentions GitHub runners and Docker images as possible ways to make the lab workflow more repeatable.

### Kubernetes is powerful, but most data scientists need a smaller interface to it
[38:48](https://www.youtube.com/watch?v=iN8aC1BYl5A&t=2328s)
Laszlo says Kubernetes is too detailed for the average data scientist, who would need substantial domain knowledge and might spend much of the time searching Stack Overflow and GitHub. Artem enjoys Kubernetes because declarative configuration can create deployments and networking with relatively little effort, but he agrees that it is complicated for data scientists. He suggests platforms built on a restricted version of Kubernetes, with the same general tools and an extensible configuration model. The group agrees that data scientists should understand enough infrastructure to discuss where a model fits, while deployment remains an engineering responsibility.

## Notable quotes
- Artem Yushkovsky: "For the demo, it works perfectly, it's very visual and interactive. But for the real use case, I would not say that having model registry being the central component that triggers the other pipelines is the best idea." (13:53)
- Artem Yushkovsky: "We work with code, we don't work with models." (14:31)
- Laszlo Sranger: "Every component what you add in an MLOps system has a maintenance cost and an infrastructure cost and an implementation cost, and these add up." (26:04)
- Dimi: "I personally like it just for the tracking part, just because it kind of motivates data scientists to track whatever they're doing." (44:12)
- Paulo Maia: "We don't need to understand how to implement the Kubernetes server from scratch, but we need to be able to talk with them to see where our model would fit." (50:29)

## Tools & references mentioned
- MLflow
- PyTorch
- Kubernetes
- Streamlit
- Yelp polarity review dataset
- TorchServe
- Keras
- TensorFlow
- SageMaker
- TensorBoard
- Databricks
- Delta Lake
- Spark
- Prometheus
- Grafana
- GitHub
- Docker
- Git

## Who should watch
- You are building a small ML product and want to see how a team connected a model, API, dashboard, registry, and deployment system.
- Your team is deciding whether MLflow should handle experiment tracking, model deployment, or the full production workflow.
- You are joining a collaborative lab and want practical lessons about choosing a problem, managing dependencies, sharing environments, and dividing engineering work.

## Editor's note

Laszlo Sranger recommends avoiding notebooks because someone can change a shared environment and send a model to production without knowing its full dependencies. ZenML records each pipeline run's steps, inputs, outputs, and code version, so a model can be traced to the data and code that produced it. This gives teams a record beyond the notebook environment.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [MLOps Engineering Labs Recap, Part 1](https://mlopstalks.com/talks/mlops-engineering-labs-recap-part-1) (John Savage, Overstock & Alexey Naiden & Varuna Jayasiri & Michel Vasconcelos, Bank of Nordeste, 59:26)
- [Making MLflow](https://mlopstalks.com/talks/making-mlflow) (Corey Zumar, Databricks, 59:11)
- [Packaging MLOps Tech Neatly for Engineers and Non-engineers](https://mlopstalks.com/talks/packaging-mlops-tech-neatly-for-engineers-and-non-engineers) (Jukka Remes, Haaga-Helia University of Applied Sciences, 8wave AI, 55:31)
- [MLOps Insights](https://mlopstalks.com/talks/mlops-insights) (David Aponte-Demetrios Brinkmann-Vishnu Rachakonda, 37:47)
- [Practical MLOps Part 2](https://mlopstalks.com/talks/practical-mlops-part-2) (Alfredo Deza, Author and Speaker, 1:01:38)