Recommendation systems often split into offline and online parts, then use fast candidate retrieval followed by slower ranking.
2
Batch recommendations are easier to operate, while real-time recommendations help when user intent changes quickly, such as news, social feeds, and some e-commerce sessions.
3
A practical MVP can use self-supervised item embeddings, an approximate nearest neighbor index, basic user and item features, and logistic regression for ranking.
Summary
Eugene Yan explains recommendation and search systems through two divisions: offline versus online computation, and candidate retrieval versus ranking. Batch systems precompute recommendations and store them for simple, reliable serving. Real-time systems react to current user behavior, but require low-latency, high-throughput services and more operational work. Yan describes a common architecture in which offline jobs train embedding models, build approximate nearest neighbor indexes, and prepare feature stores. An online request retrieves a few hundred or thousand candidates, adds user, item, and context features, then ranks the candidates for display. Examples from Alibaba, Facebook, and DoorDash show how graphs, word2vec-style training, two-tower models, query processing, and business rules fit into this pattern. For an MVP, Yan recommends starting with batch if possible, using interaction sequences to learn embeddings without labeled data, and beginning ranking with logistic regression. He also explains why filtering and final ordering may need separate stages for availability, safety, diversity, and business rules.
Batch and real-time recommendations make different operating trade-offs
Batch systems run a large job, store recommendations in a key-value data store, and refresh them periodically. Since computation is already finished, serving is simple and a failed Spark job can leave yesterday's recommendations available. Real-time systems take a customer ID, browsing history, time, and other context through a REST or gRPC API and generate results on demand. They can react to short-lived intent, such as a shopper suddenly looking for winter gear, but the API must stay available 24/7 with low latency and high throughput. Yan recommends starting with batch unless the use case clearly needs faster updates.
Real-time recommendations matter most when user intent changes quickly
Yan gives several cases where stale recommendations can lose a user's attention. A new e-commerce user may browse dresses or phones, giving the system useful information before much demographic data exists. Travel choices change with the next destination, and YouTube receives new videos constantly, which makes old history less useful. Real-time generation can also avoid computing recommendations for users who do not return. Yan contrasts these cases with groceries, household appliances, clothing, and some electronics, where a recommendation being a day late may not matter.
Offline artifacts feed a two-stage online pipeline
The offline environment trains models, indexes items, builds graphs, and loads user and item data into feature stores. Candidate retrieval is fast and coarse. Given an item, query, or user representation, it searches a catalog of millions for perhaps hundreds or a thousand similar candidates. Ranking is slower and more precise. It can use user features, item metadata, and context such as time of day, day of year, category, brand, seller, recent sales, and price preferences. The online service combines these artifacts, scores the candidates, and returns a small final set for the page.
Embeddings and approximate nearest neighbor indexes reduce retrieval work
Yan describes training an embedding model that converts every catalog item into a small dense vector, then placing those vectors in an approximate nearest neighbor index. At request time, an item such as The Matrix becomes a vector, and the index returns similar films before the ranker sees them. Alibaba's example builds an item graph from user behavior, performs random walks, trains word2vec-style embeddings, and creates an item similarity map. Facebook's search example uses separate document and query towers so both can be represented in the same vector space before retrieval.
Industry systems add domain-specific processing around the same pattern
Alibaba's ranking system combines item profiles, user profiles, and behavioral data in a graph-based ranker. Facebook adds query processing, retrieval, a forward index for features, ranking, and filtering. DoorDash uses spell checking, query understanding, and a knowledge graph to expand a query such as KFC into related restaurants and foods. The expanded candidate set is then ranked with location, price, rating, and food similarity. These examples differ in model details, yet they retain the offline versus online and retrieval versus ranking structure.
Simple interaction statistics can provide incremental candidate retrieval
Yan explains collaborative filtering with weighted user and item interactions. A user with many interactions receives less weight because that behavior is noisy, while a user with only a few interactions receives more weight because it may be more selective. Item-to-item similarity comes from shared users and their weights. Since the values are counts in key-value stores, the scores can be updated incrementally. This offers a path to real-time candidate retrieval without immediately building a complex neural model.
A recommendation MVP can start with sequence embeddings and logistic regression
For a first system, Yan recommends putting each user's interacted items into a sequence and training word2vec or skip-gram embeddings. This avoids designing labels and negative samples. The item vectors go into an approximate nearest neighbor index. Ranking can begin with user and item features passed to a single logistic regression layer, with a feature store supplying those values. Yan suggests serving the retriever and ranker on multiple instances behind a load balancer, or using SageMaker, and scaling the number of instances according to request throughput.
Filtering and ordering belong after retrieval and scoring
Yan describes a refinement in which invalid candidates are removed before expensive ranking. Filters can exclude content unsuitable for a user's age group, items unavailable in the user's location, or books the user already owns. The ranker then produces scores, while a final ordering stage applies business logic. A list of only Harry Potter films might be highly relevant but visually repetitive, so ordering can prevent adjacent duplicates and mix in related films such as The Lord of the Rings or The Hunger Games.
"Just start with batch first and then after that really think through from the customer's perspective if real time is going to be helpful for you."Eugene Yan24:42
Who should watch
You are building a recommendation or search MVP and need a practical architecture before choosing a more advanced model.
Your team is considering real-time recommendations and needs a way to judge whether the extra operational work matches the user need.
You work on retrieval, ranking, or search and want to see how industry examples fit into one reusable system pattern.
