MeetupReal-time Feature Pipelines, A Personal HistoryClaimThe first team trained models in R and manually reimplemented inference in F# so the models could run inside a C# backend.5:28
Threads / Serving and deployment / 2021
Serving and deployment in 2021
44 sessions
MeetupReal-time Feature Pipelines, A Personal History
PodcastLessons Learned From Hosting the ML Engineered PodcastPushed backCharlie You argues that the hardest or most differentiating part of production machine learning is often not the algorithm but data, deployment and maintenance.58:57
MeetupMachine Learning Design Patterns for MLOpsClaimTools can encode best practices at the library, service, and framework levels.16:41
PodcastMLOps Engineering Labs Recap, Part 1Pushed backMichel Vasconcelos argues that MLflow is a strong experimentation tool but is not sufficient by itself for large-scale, containerized deployments.34:54
PodcastMLOps Engineering Labs Recap, Part 2ClaimThe backend used a REST model-serving API, a model proxy that collected feedback, and a model operator that synchronized MLflow with the deployed model.10:28
MeetupOperationalizing Machine Learning at a Large Financial InstitutionPushed backDaniel Stahl disputes the assumption that models should be deployed as microservices by default, arguing that batch processing was simpler and sufficient for Regions' current needs.28:08
PodcastMachine Learning at AtlassianClaimGeoff Sims began with a physics degree, completed a four-year astrophysics PhD, and worked on instruments for evaluating astronomical observing sites in Antarctica.1:17
MeetupDeploying Machine Learning Models at Scale in CloudClaimVishnu Prathish brings software engineering practices to MLOps because his deployment patterns and pipelines combine data science workflows with traditional software engineering.0:29
PodcastWar Stories Productionising MLClaimNick Masca transitioned into data science after working in biomedical research, academia, industry and financial services.3:06
MeetupLaw of Diminishing Returns for Running AI Proof-of-ConceptsPushed backOguzhan Gencoglu says the most crucial role skill is problem translation, not model training or deployment, which may be controversial.45:06
SGT Model Deployment Working Group May 19, 2021 MeetingPushed backDeployment and monitoring should remain grouped for now rather than immediately becoming separate working groups.9:40
PodcastScaling AI in ProductionClaimSrivatsan Srinivasan says machine-learning algorithms make up only a small part of the total machine-learning work, which also includes data collection, deployment, monitoring, pipelines, feature engineering, and feature stores.1:45
MeetupOperationalize Machine Learning at Scale with MLOpsClaimChristopher Bergh says source code should be versioned, production systems should be monitored, development systems should be tested, and infrastructure should be scriptable, testable, and deployable.14:06
PodcastCI/CD in MLOpsClaimGitLab’s MLOps audience includes DevOps engineers who must collaborate with data scientists and deploy machine-learning models without necessarily being mathematicians.3:16
MeetupPractical MLOps Part 2ClaimAlfredo Deza learned Bash scripting after Carlos Cole encouraged him to write basic server-monitoring code and offered to answer questions for 15 minutes each day.6:42
MeetupBuilding an ML Platform from Scratch: Live Coding SessionPushed backAlon Gubkin chooses FastAPI for the workshop instead of Seldon Core or other model-serving systems because he considers it easier to customize.12:22
PodcastMaturing Machine Learning in EnterpriseClaimKyle Gallatin says machine learning has moved from isolated proof-of-concept data science projects toward MLOps, and that governance, observability, and visibility are the next stage.10:13
SGT Model Deployment Working Group June 22, 2021 MeetingClaimThe working group proposed two approaches for testing compliance with NVIDIA's v2 inference specification.3:34
How Pinterest Powers Image SimilarityClaimPinterest wanted to make the transition from its mature batch pipeline to near-real-time processing as seamless as possible for consumers.9:56
PodcastLearning from 150 Successful ML-enabled Products at Booking.comClaimPablo Estevez says Booking.com built shared machine learning infrastructure for model serving, feature engineering, feature serving, and monitoring as demand grew across teams.43:14
PodcastFast.ai, AutoML, and Software Engineering for MLClaimJeremy Howard created fast.ai to provide machine-learning practitioners with layers of abstraction for building, testing, deploying, and maintaining models.8:06
MeetupBuilding ML Blocks with Kubeflow Orchestration with Feature StorePushed backAniruddha Choudhury says Feast is not a workflow scheduler, data warehouse, pipeline orchestrator, feature engineering tool, or model-serving product.16:10
PodcastTour of Upcoming Features on the Hugging Face Model HubClaimJulien Chaumond says the Hugging Face platform lets users and organizations share and discover models and data sets, and increasingly train and deploy models.5:49
SGT Model Deployment Working GroupPushed backmlctl should provide neutral interconnection layers rather than become another horizontal platform that owns every service.40:05
PodcastAggressively Helpful Platform TeamsClaimModel Envelope packages a model and its required information so a data scientist can register it, provide configuration, and deploy it through an API or batch workflow.29:12
PodcastCreating MLOps StandardsClaimAlex Chung says MLOps is fragmented because its lifecycle includes data, features, model development, training, deployment, monitoring, metadata, experimentation, logging, and artifacts.10:10
mlctl and Hydrosphere Open Source MLOps Libraries DemoPushed backAlex Chung questioned whether Hydrosphere should continue serving models or focus on monitoring and integration with existing tools, and recommended the latter focus.31:41
MeetupBuilding Machine Learning Models into Docker ImagesPushed backLuke Marsden argued that putting model files on S3 and restarting containers is dangerous because it can change production software without preserving the exact version.26:13
PodcastMLOps InsightsClaimTesting in machine learning involves more than software unit tests; it can include data quality tests, training checks, serving infrastructure tests, and end-to-end tests.1:12
PodcastMachine Learning SREPushed backThe discussion challenges the idea that machine learning incident response can follow the same urgency and resolution pattern as ordinary web-service incidents.28:35
MeetupSystem Design for Recommendations and SearchClaimRecommendations and search systems are moving from batch processing toward real-time processing, which introduces additional constraints.0:55
PodcastLearning and Teaching MLOps ApplicationsClaimSalwa Muhammad says employer conversations showed that many candidates had data and modeling skills but lacked deployment skills.11:35
PodcastA Few Learnings from Building a Bootstrapped MLOps Services StartupClaimThe steep learning curve and the satisfaction of deploying a working model motivated Soumanta Das to continue building Yugen.ai during the pandemic.5:54
PodcastThe Future of ML and Data PlatformsClaimErik Bernhardsson recommends self-service access to data and says teams can automate recurring uses after observing how people use the initial tools.17:09
MeetupDoing MLOpsClaimOrganizations need DevOps capabilities, including automated testing and deployment, before they can build effective MLOps systems.9:23
MeetupEnd to End MLOps BasicsPushed backRaviraja Ganta disputes the idea that AWS Lambda is always the best deployment choice and says the choice depends on latency, GPU, scaling, and other use-case requirements.36:23
PodcastI Don't Like Jupyter NotebooksClaimJoel Grus says his team is constrained by AWS security rules and therefore mainly uses standard AWS platform services rather than choosing any tool it wants.18:52
PodcastPyTorch: Bridging AI Research and ProductionClaimDmytro Dzhulgakov said machine learning infrastructure had to shift from hands-on enablement to self-service platform building as the number of developers and use cases grew.14:05
PodcastThe Future of Data Science Platforms is AccessibilityClaimSkylar Payne says packaging models is an important first MLOps problem because it creates an interface between training and serving systems.14:29
PodcastWikimedia MLOpsClaimChris Albon says his team wants to deploy even a baseline model to production immediately so that the full workflow is tested before the model is improved.11:59
MeetupBuilding 12-Factor Data Apps with KedroClaimIvan Danov says the 12-Factor App methodology emerged from Heroku's observations of applications that were easy or difficult to scale.14:50
MeetupSetting up an ML Platform on GCP: Lessons LearnedClaimMefta Sadat's team builds the machine learning and data platform that data scientists use to run experiments, deploy models, and manage data ingestion and engineering.2:12