# Introducing MLOps Engineering Labs - Join us to better your skills | MLOps Community | 14:39 Hosted by David Aponte Source: https://www.youtube.com/watch?v=vs2lagvTLWo Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks. Page: https://mlopstalks.com/talks/introducing-mlops-engineering-labs-join-us-to-better-your-skills Published: 2020-08-02 Tags: deployment, engineering-culture ## TL;DR - Engineering Labs will help practitioners improve their skills by solving difficult, real-world machine learning engineering problems together. - Each lab will combine case studies, milestone challenges, system architectures, production practices, and experiments with different technologies. - The initiative will use a rotating group of community members to design labs, with the first group initially focused on people who have production experience. ## Summary David Aponte and Ivan introduce Engineering Labs, an initiative from MLOps Community for practicing the engineering work around machine learning systems. The labs are intended to help people improve their skills, collaborate across locations, try tools they cannot use at work, and solve difficult problems together. The work will cover data ops, model ops, DevOps, system architecture, modularity, reproducibility, automation, and productionization. Each lab should be based on a real-world case study with milestones that lead toward an end-to-end system. The community will vote on challenges raised in community discussions, then a small group of developers will build and solve the selected challenge. Early contributors are expected to have around one or two years of experience putting systems into production. The organizers also describe the labs and their technical design as an evolving work in progress, with future participation opened to people who mainly want to learn. ## Key ideas ### Engineering Labs are for improving practical skills through shared work [01:15](https://www.youtube.com/watch?v=vs2lagvTLWo&t=75s) David Aponte says the initiative is aimed at people working in a fast-changing field who need regular practice with new technologies. The labs will give participants a place to improve their skills, collaborate with people in different time zones, share practices that work, and grow as team members. They will also provide a safe place to try tools that someone may not be able to use at work because of limited infrastructure, budget, or expertise. Participants are expected to learn by experimenting, including making mistakes and testing unfamiliar approaches. ### The labs will solve difficult problems and change with the community [02:46](https://www.youtube.com/watch?v=vs2lagvTLWo&t=166s) The organizers want the work to be enjoyable while dealing with hard engineering problems such as debugging production systems and making systems reliable. They describe the initiative as an ongoing work in progress. Early challenges may differ from later ones as new tools appear, the machine learning pipeline changes, or community members bring different ideas. The lab topics are therefore expected to evolve with the needs and interests of the community rather than follow a fixed curriculum. ### Each lab will cover the engineering around a machine learning system [04:36](https://www.youtube.com/watch?v=vs2lagvTLWo&t=276s) The scope is broader than model training or data preparation alone. David Aponte says participants can work on data ops, model ops, DevOps, or any other engineering area relevant to solving the problem. The labs are intentionally called Engineering Labs because they focus on the whole system. This can include the parts needed to move from a technical solution to something that addresses day-to-day engineering challenges. ### Case studies will use milestones to build toward a complete system [05:10](https://www.youtube.com/watch?v=vs2lagvTLWo&t=310s) The organizers want labs to be more involved than tutorials. A case study will ask participants to build parts of a solution and reach a series of milestones. The intended result is an end-to-end system, although the exact format may vary. Ivan gives the example of a community question about putting a model into production and monitoring its model currency. The selected problems should resemble situations that practitioners face in their jobs. ### Production use is the measure of whether a model creates value [06:02](https://www.youtube.com/watch?v=vs2lagvTLWo&t=362s) David Aponte says the work should focus on productionization, because machine learning has value only when it reaches end users. A model should not be built only to optimize a metric if nobody will use it in the real world. The labs should therefore connect technical work to a user-facing outcome. This emphasis keeps the initiative focused on building systems rather than treating model development as the final result. ### The community will choose challenges and form a rotating design group [06:46](https://www.youtube.com/watch?v=vs2lagvTLWo&t=406s) Engineering Labs will involve community members in choosing and creating the work. A rotating core squad of developers, engineers, and scientists will design the labs, with the workload distributed beyond the organizers. Ivan describes a process in which the community votes on three challenges drawn from community conversations. After the vote, a small group of developers will build the selected scenario and work through the challenge. The first contributors are expected to have experience putting systems into production, while later versions may include more people who are mainly looking to learn. ### The technical work will compare architectures, practices, environments, and technologies [10:21](https://www.youtube.com/watch?v=vs2lagvTLWo&t=621s) Ivan says the labs will explore batch, real-time, and streaming architectures. They will promote practices such as reproducibility, automation, and code modularity. The organizers also want participants to work in replicable environments, including containers and Kubernetes, with possible cloud platforms later if sponsorship becomes available. Since machine learning systems contain many components, the labs will also experiment with different technologies rather than prescribe one fixed stack. ## Notable quotes - David Aponte: "We want to provide an opportunity and a platform through this challenge to improve our skills." (01:15) - David Aponte: "We don't just model just to model, although it's fun, you know, data science is fun, but we have a real goal in mind. We want to build things for our users." (06:02) - Ivan: "The idea here is you build a four or five person core squad to build the scenario, to do the lab, and of course to solve the challenge." (09:33) - Ivan: "The goal is to have something that you can replicate, you can play with, you can practice." (11:10) ## Tools & references mentioned - MLOps Community - Kubernetes ## Who should watch - You work on machine learning systems and want a place to practice production engineering outside your day job. - Your team needs case studies that connect models and data work to end users and complete systems. - You have production experience and want to help a community group design hands-on challenges, or you want to follow the initiative as it opens to newer practitioners. ## Related talks - [MLOps Community 1 Year Anniversary!](https://mlopstalks.com/talks/mlops-community-1-year-anniversary) (Demetrios B., MLOps Community & David A. & Vishnu R., Tesseract Health, 59:24) - [I Am Once Again Asking "What is MLOps?"](https://mlopstalks.com/talks/i-am-once-again-asking-what-is-mlops) (Oleksandr Stasyk, Synthesia, 1:06:22) - [Our 1st MLOps Meetup](https://mlopstalks.com/talks/our-1st-mlops-meetup) (Luke Marsden, Dotscience, 56:11) - [Doing MLOps](https://mlopstalks.com/talks/doing-mlops) (Noah Gift, Pragmatic AI Labs, 1:01:22) - [Operationalize Machine Learning at Scale with MLOps](https://mlopstalks.com/talks/operationalize-machine-learning-at-scale-with-mlops) (Christopher Bergh, DataKitchen, 57:50)