Pinterest moved ads ranking from a hybrid stack of GBDT, TensorFlow, and C++ toward more consistent neural network systems, while weighing migration effort against business value.
2
Conversion ranking required new data integrations, training pipelines, product work, sales enablement, monitoring, and low-latency serving because conversion events happen outside Pinterest.
3
Aayush Mudgal says sequence modeling and Transformers can model a user's interactions with Pins, but Pinterest has to control sequence length, serving cost, and latency through GPUs, quantization, offline computation, and custom serving systems.
Summary
Aayush Mudgal walks through Pinterest's ads ranking work from 2018 through 2024. He explains how the team moved from basic impression and click optimization toward conversion modeling, video ads ranking, neural networks, sequence modeling, and Transformers. The hardest work was often outside the model. Conversion data came from third-party providers, pipelines had to connect that data to training, and the product needed monitoring, sales support, debugging, and fast serving. Pinterest also replaced legacy Scala-based data jobs from Twitter with Spark and moved from TensorFlow to PyTorch after comparing implementation experience across teams. Aayush is candid about the cost of these migrations. Better infrastructure makes future experiments faster, but large serving changes can take multiple quarters while the business continues to operate. He also describes how Transformer-style models can learn feature interactions and predict a user's next likely interaction with Pinterest content, subject to strict latency and infrastructure limits.
Ads ranking grew from showing ads to predicting business outcomes
When Aayush joined Pinterest in 2018, impression and click products already existed, while conversion optimization was still being built. The progression was from delivering more ads, to driving more clicks, to getting users to buy something or add an item to a cart. Conversion ranking had to identify both users who were in a buying state and the ad they were most likely to act on. Aayush describes the model itself as a binary classification problem, but the product required much more: external conversion integrations, data transformation, training pipelines, decision logic, performance, debugging, and sales support.
Conversion modeling made the data pipeline much harder
Click data was generated inside Pinterest, while conversion data came through third-party providers. Pinterest had to integrate those providers, transform their data, identify the relevant events, and pass the results into model training. The predictions then had to influence ad decisions and work within a product that advertisers compared with click optimization. Aayush says conversion ranking also needed personalization and strong visibility into failures. Since the system affected advertiser budgets, it had to make predictions quickly and support decisions about which ads to show and how much value to assign them.
Pinterest replaced legacy tools when support, hiring, and scale made migration worthwhile
In 2018, Pinterest used Scala-based Cascading and Scalding jobs for basic ETL work. The technology was no longer being updated, it became harder to find engineers who knew it, and Spark offered more flexibility and scalability. Pinterest gradually moved those jobs to Spark. Aayush says migrations are difficult, so the team had to judge the return before starting. Pinterest also moved from a hybrid model stack that combined XGBoost-based GBDTs, TensorFlow-based components, and a C++ serving library toward more consistent systems.
Low latency limits how complex ads models can become
Ads ranking has to run in real time because predictions affect delivery and advertiser spending. Aayush gives a latency range of roughly 300 to 400 milliseconds for the system. Pinterest uses targeting and ranking stages rather than scoring every ad at once, which helps control latency. As models became more complex, the team began hitting its latency budget. It moved some serving workloads to GPUs, while accepting higher infrastructure cost when better personalization could recover that cost through improved business results.
Monitoring now covers the full path from data preparation to serving
Aayush describes a shift from systems where monitoring and visibility were often secondary to systems that monitor offline data pipelines, online serving, model predictions, features, and end-to-end pipeline health. Pinterest checks whether predictions stay within expected ranges and watches changes over time. It also compares how features are logged and used during training with how they are served online. Alerts and automated checks can stop an affected model or pipeline from being promoted. Incidents still happen, but the controls prevent many of the failures the team saw earlier.
Simple first versions reduce risk and expose the real bottleneck
For new work such as video ads ranking, Pinterest could initially treat a video as an image by using a representative frame. The team could then add video-specific understanding over time. Aayush applies the same approach to infrastructure and product design. Teams should start with the simplest useful version and add complexity when a specific pain makes it necessary. He says systems that lacked feature backfilling once forced Pinterest to wait three months before knowing whether a new feature worked. Improving the systems that support experimentation can produce more value than adding a sophisticated model too early.
Large migrations need parallel operation and replayable comparisons
Pinterest's C++ serving system sat on a critical path, so replacing it could not happen as a clean stop-and-start project. Aayush compares the work to changing the wings on a plane while it is flying. The migration took about two years, with the business continuing to run and improve during the change. He recommends commitment from leadership, tooling that compares old and new systems, and the ability to replay events to find where outputs diverge. Teams also need agreement on priorities, because continued improvements to the old system can otherwise keep the migration target moving.
Sequence models treat user interactions like a sentence
A ranking model can represent the user, the Pin or ad, and their interactions. A Transformer can learn how those features interact, even without the positional structure found in language tasks. It can also process a sequence of actions such as clicks, hides, and saves, then predict what the user may interact with next. Aayush connects this to language modeling, where a system predicts the next token in a sequence. To make the approach usable, Pinterest controls sequence length, removes repeated or similar content, computes longer histories offline, and references that work during online serving.
"You can think about it very similar to what is in the text processing domain and try to model what is the next action or what's the next content that you might interact with."Aayush Mudgal47:17
Who should watch
You are building a recommendation or ads system where model quality has to fit strict latency and spending limits.
Your production stack has accumulated multiple languages, legacy pipelines, or unsupported open source components, and you need a practical migration approach.
You want to understand the operational work behind conversion modeling and sequence-based ranking, including data integration, monitoring, backfilling, and serving.
