Podcast

ML in Production: A DS from Ubisoft Perspective

Jean-Michel Daignan, UbisoftEpisode 151 · 49:26 · Mar 2023 · 547 viewsHosted by Abi Aryan
Thumbnail for ML in Production: A DS from Ubisoft Perspective Watch on YouTube
TL;DR
  1. 1

    Jean-Michel Daignan says data scientists joining an ML platform team need to learn developers' tools and language, then start with a small platform that answers specific needs.

  2. 2

    At Ubisoft, his team uses Merlin SDK modules for tasks such as messaging, data quality checks, A/B testing, fallback updates, and access to game-service data without forcing teams to adopt the SDK.

  3. 3

    Jean-Michel prioritizes production use cases with measurable business value, especially game recommendations, and prefers simple prototypes, batch processing, and cost controls before adding more complex systems.

Summary

Jean-Michel Daignan describes machine learning production work at Ubisoft from a data scientist's perspective. He explains how his earlier work in energy and proof-of-concept projects shaped his preference for fast prototypes and simple models. On Ubisoft's platform team, he works with developers, uses MLflow for model versioning, and tries to offer data scientists simple APIs, standard machine sizes, documentation, and low-maintenance tools. His team runs recommendation systems for in-game stores and other game features, with pipelines for telemetry processing, model training, deployment, monitoring, and A/B testing. Merlin SDK packages the team's experience into reusable functions, while adoption remains voluntary. Daignan is candid about data freshness, monitoring gaps, feature store adoption, and the limits of managed platforms. He supports generative AI experiments when they have a clear use case, but his platform work currently focuses on player-facing products whose value and cost can be measured.

Key ideas
07:37

A data science background can shape a practical platform approach

Jean-Michel Daignan moved from physics and engineering into data work through smart-home forecasting, IoT applications, energy projects, and machine learning proofs of concept. At Ubisoft, he joined a platform team and had to work closely with developers. He says this means learning their language and tools, communicating clearly about a specific need, and comparing the cost of integrating a technology from both the data science and developer perspectives. He recommends narrowing the tool set to actual needs and starting with basic capabilities rather than building around every possible data science request.

09:39

The platform should make ordinary production work simple

For Daignan, the basics include an easy API call for serving models, model versioning, metric logging, access to different machine sizes, and simple setup. Data scientists should be able to choose a small, medium, or large machine without knowing the infrastructure underneath. This abstraction lets the platform team change underlying technologies later. He also values low-maintenance tools, documentation, tutorials, and power users who can give direct feedback. The platform should expose enough of the system that users are not unnecessarily constrained.

10:59

Fast prototypes matter more than fashionable model choices

Daignan says he is not focused on building the best deep learning model. He spends more effort on data and prefers models that are good enough to test whether an application has value. His earlier jobs involved producing prototypes quickly so businesses could see what an application might do. At Ubisoft, he carries that habit into production work. He advises keeping the first version simple, putting it into production, and iterating later instead of starting with Transformers or another fashionable technology.

14:49

Production support requires more than moving a notebook into deployment

Daignan does not think exporting a notebook to a Python file is enough for production. His team shares example projects and practices, but it does not enforce every software engineering practice equally. Unit tests become more useful when functions are shared across projects because they document expected behavior and help catch breaking changes. For his own pipelines, he gives more attention to scalability testing. Since the team uses Spark, he wants to know whether an ETL or machine learning pipeline can handle much larger player volumes than the current expectation.

16:49

Ubisoft recommendation systems connect telemetry to measurable product work

Daignan describes an end-to-end recommendation workflow for in-game stores. The team collects telemetry about transactions and player actions, builds training data and targets such as the next item a player may buy, trains a model, and places the code in a GitHub repository. Airflow runs the production pipeline and refreshes recommendations served in the game. Separate pipelines monitor prediction quality and compare treatment and control groups in A/B tests. The data scientist also works with game developers and clients, proposes experiments, and handles data migrations as Ubisoft's storage systems change.

19:41

Data freshness and quality are the main operational problems

Daignan identifies data quality and freshness as his main sources of trouble. His team depends on other people to collect events and make them available, so problems can remain hidden until predictions or accuracy change. He does not trust data providers blindly and prefers to check the data throughout the pipeline. Ubisoft has a data office and tools for event collection and processing, but the platform team still has to connect the pieces. Their monitoring checks freshness, raw-data distributions, training sets after joining sources, prediction quality, and metric drift.

21:35

Merlin SDK spreads platform experience without forcing adoption

Merlin SDK is a collection of wrapper functions created by the platform team. It includes modules for sending messages through Slack, Teams, or email, splitting populations for A/B tests, checking data quality, updating fallback predictions, and reading information from Ubisoft's online services. The SDK also gives teams a way to share modules, such as collaborative filtering functions. Adoption is voluntary. Users can submit changes, discuss testing and maintenance with developers, and gradually turn useful workarounds into supported platform features.

31:13

Batch recommendations are useful before streaming is worth the cost

Daignan's main recommendation use case runs in batch, which fits in-game store recommendations. Ubisoft has also released live prediction, where the game client sends session information and the model builds features and computes a prediction as events arrive. The team is discussing streaming feature computation as part of its future data strategy, but Daignan says batch prediction already delivers value. A feature store could help share standard features across games, though he describes adoption as difficult and the work as a long-term effort.

40:13

Generative AI needs a real player-facing use case

Daignan is open to Ubisoft teams running generative AI models and using available GPUs, but he would not advise running an expensive setup continuously without a matching use case. His own platform work focuses first on what can be delivered to players, including gameplay, monetization, and recommendations. Other Ubisoft groups investigate tools for building games and code-related tasks. He says the platform can support new ideas, but production work should start when there is a clear use case rather than an interesting demonstration alone.

"What is important is that an ML platform will not bring also take the world. It's not scalable in terms of cost, it's terrible maintenance, and really it's only bringing what is really needed."Jean-Michel Daignan32:54
Who should watch
  • You are building an internal ML platform and need to decide which capabilities data scientists actually need first.
  • Your recommendation or personalization pipeline depends on telemetry, batch jobs, monitoring, and changing data systems.
  • You are weighing fast prototypes and measurable product value against feature stores, streaming, or generative AI experiments.