# MLOps at GetYourGuide

Jean Machado, Meghana Satish, Olivia Houghton & Theodore Meynard, GetYourGuide | MLOps Podcast | Episode 182 | 1:03:53
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=KNx8Gz0LGRk
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/mlops-at-getyourguide
Published: 2023-10-17
Tags: model-serving, observability, platform-teams, testing

## TL;DR
- GetYourGuide built its ML platform during the COVID downturn so the company would have a foundation for future machine learning products.
- The team uses templates for batch and online services, shared Kubernetes infrastructure, model observability, and data testing to help teams move models into production.
- The platform team prioritizes work through business needs, adoption, production productivity, operating costs, and emerging use cases such as LLM-based products.

## Summary
GetYourGuide's ML platform grew from an engineering standards initiative during the COVID downturn, when the company had time to invest in infrastructure for its future recovery. Theodore Meynard, Jean Machado, Meghana Satish, and Olivia Houghton describe how the platform supports growth, marketplace, supply, search, and activity-ranking products. The team started with batch templates, then added an online service template after launching a real-time ranker. It shares company infrastructure where possible, including Kubernetes and deployment practices, while adding ML-specific tools for serving, observability, and data testing. Model health meetings and dashboards have made monitoring a regular team practice. A sampling tool combined with Great Expectations lets teams test data and run end-to-end checks in CI without processing full production datasets. The team is also working out how to productionize LLM use cases, while keeping experimentation open to teams that do not use the platform's Python-based templates.

## Key ideas
### The platform began as a long-term investment during a business downturn
[03:36](https://www.youtube.com/watch?v=KNx8Gz0LGRk&t=216s)
Theodore Meynard says GetYourGuide's ML platform came from an engineering standards initiative during the COVID period. GetYourGuide is a marketplace for travel experiences, and its revenue was heavily affected when tourism stopped. That gave the company time to work on foundations for the future. The goal was to launch and operate machine learning systems, which the team often calls data products. Meynard and Jean Machado describe strong support from management, especially their manager, who sponsored the work and helped explain why ML needed its own infrastructure. The team first researched training and batch use cases, built proofs of concept, and used their results to make the case for further investment.

### The platform supports many products, with activity ranking as a demanding online use case
[06:50](https://www.youtube.com/watch?v=KNx8Gz0LGRk&t=410s)
GetYourGuide's data products cover growth, marketing optimization, retention, recommendations, marketplace ranking, supply forecasting, and search. Olivia Houghton explains that the activity ranking service orders tours and other experiences after a user searches for a destination. The ranking depends on the page, the user's platform, the user's location, and other signals. The platform team supports this work alongside other data product teams. Its first broad deliverable was a batch template that could transform and monitor data products over time. The team says at least 40 data products had adopted the template, although they do not give an exact count. A search project uses entity recognition on user queries and builds on these shared platform principles.

### Templates separate batch pipelines from online services
[20:18](https://www.youtube.com/watch?v=KNx8Gz0LGRk&t=1218s)
The platform is a group of configurable templates. A batch configuration creates a basic structure for an asynchronous job, such as a Databricks or Spark job that can run for a longer period and produce a file or model. An online configuration creates a service with an endpoint that can answer API requests. The team originally had only the batch template. When GetYourGuide launched its first online inference model, a real-time ranker, it used the lessons from that work to create the V2 service template. Olivia Houghton says this separation helps clarify responsibilities between data scientists and MLOps engineers. The templates continue to change as the team learns from new services and experiments.

### Latency is the main constraint in real-time experimentation
[24:41](https://www.youtube.com/watch?v=KNx8Gz0LGRk&t=1481s)
Olivia Houghton describes latency as the hardest consistent problem in the activity-ranking experimentation system. Earlier ranking logic used precomputed scores and simple arithmetic. Calling a model for each request added work, even though the model itself took about 20 milliseconds on average. At the time, the service timeout was 60 milliseconds, so the model used roughly one-third of the available time. The team has raised the threshold, but still needs room for future model improvements. Small engineering changes matter. Houghton gives the example of calculating experiment assignment locally rather than paying networking overhead. The team tries to reduce latency without over-optimizing, while keeping enough capacity for a model that may become more useful but slower.

### The team reuses company infrastructure and contributes across team boundaries
[27:54](https://www.youtube.com/watch?v=KNx8Gz0LGRk&t=1674s)
GetYourGuide's data products mostly use Python, while much of the wider organization uses Java and Spring Boot. That difference means some ML-specific tools are needed, especially for serving Python models. The team still tries to reuse existing deployment and development practices. Packaging services in Docker containers lets them use shared infrastructure, and production systems run in a Kubernetes cluster used by other teams. Jean Machado says the ML platform team does not have enough people to build a separate version of everything. Engineers and data scientists from other teams contribute to the platform, and those contributions are part of how performance is viewed. A hack day led to a faster notebook development workflow being added to the platform within about a week.

### Model observability became a recurring operational practice
[09:29](https://www.youtube.com/watch?v=KNx8Gz0LGRk&t=569s)
When Meghana Satish joined, teams wanted better ways to observe and debug models. Previously, teams recorded their own metrics, and there was no standard way to judge model performance. GetYourGuide adopted Arize and began treating model health more like service health. The team holds recurring meetings where people from different data product teams inspect model behavior and drift over time. Dashboards also track engineering practices, such as end-to-end tests, dependency management, tool versions, and the time needed to move a data scientist's work from zero to production. Satish says rituals help teams act on those measurements. The platform team describes this as a cultural change from reacting to model problems after they appear toward routinely checking model health.

### Data needs tests alongside code
[36:36](https://www.youtube.com/watch?v=KNx8Gz0LGRk&t=2196s)
Jean Machado argues that a machine learning product depends on both code and data, so data should be tested in the same way that software code is tested. A syntactically correct pipeline can still produce no rows after a bad join, leaving a model with nothing useful to train on. GetYourGuide uses Great Expectations and an open-source sampling tool the team calls D data flow to check pipelines with subsets of production data. The samples make local development faster and allow the checks to run in CI without processing the full dataset. The team values the abstraction because a pipeline may draw from many sources, making separate sample data for every source difficult to maintain. These end-to-end checks give data scientists confidence that a change can reach the service without pairing with an MLOps engineer every time.

### LLM work is moving from quick experiments toward production controls
[48:06](https://www.youtube.com/watch?v=KNx8Gz0LGRk&t=2886s)
GetYourGuide saw a surge of internal LLM ideas after ChatGPT and OpenAI's announcements. Meghana Satish describes work on unwanted-content detection for search queries, using ChatGPT to identify unethical terms so they can be excluded from paid search ads. Another project uses LLMs to check and correct supplier content so tours can go live faster. The platform team is trying to make these use cases more self-service, while recognizing that a quick API demo is different from a production service. Production concerns include LLM observability, cost estimates at scale, and rate limiting. Jean Machado says the team will strengthen its current foundations rather than immediately support every language used by internal prototypes. The roadmap is shaped by business needs, adoption, production throughput, operating costs, and new LLM requirements.

## Notable quotes
- Jean Machado: "The ML platform is not like, you have to use it. It's the ML platform I helped build." (15:59)
- Jean Machado: "You should also test your data the same way you test your code." (36:59)
- Olivia Houghton: "The biggest consistent challenge is definitely the latency portion." (24:41)
- Jean Machado: "To productionize it is more challenging." (50:54)

## Tools & references mentioned
- GetYourGuide
- MLOps Community
- Arize
- Great Expectations
- Databricks
- Spark
- Kubernetes
- Docker
- Python
- Java
- Spring Boot
- ChatGPT
- OpenAI
- PyData Berlin

## Who should watch
- You are building an ML platform and need examples of how to earn support from management before the platform is fully defined.
- Your teams struggle with model monitoring, data quality, or the handoff between data scientists and MLOps engineers.
- You are moving LLM prototypes toward production and want a candid account of the operational issues that appear after the demo works.

## Editor's note

Jean Machado says a syntactically correct pipeline can still produce no rows after a bad join, leaving a model with nothing useful to train on. ZenML records each pipeline run's steps, inputs, outputs, and code version, so teams can trace a model or artifact back to the data and code that produced it.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [MLOps Insights](https://mlopstalks.com/talks/mlops-insights) (David Aponte-Demetrios Brinkmann-Vishnu Rachakonda, 37:47)
- [Kubernetes, AI Gateways, and the Future of MLOps](https://mlopstalks.com/talks/kubernetes-ai-gateways-and-the-future-of-mlops) (Alexa Griffith, Bloomberg, 50:12)
- [From Notebooks to Production FASTER](https://mlopstalks.com/talks/from-notebooks-to-production-faster) (Shahd Alghrsi, Virgin Media, 13:43)
- [Engineering MLOps](https://mlopstalks.com/talks/engineering-mlops) (Emmanuel Raj, TietoEvry, 51:55)
- [Project/Product Management for MLOps](https://mlopstalks.com/talks/project-product-management-for-mlops) (Korri Jones-Simarpal Khaira-Veselina Staneva, 57:22)