Near-real-time features let recommender systems react to a user's current intent within seconds or minutes, while real-time inference usually refers to millisecond-level responses.
2
Teams should add real-time features after improving their models and batch features, and should usually add real-time versions of important batch features alongside the existing historical features.
3
Real-time feature systems are difficult because machine learning engineers must work with distributed streaming infrastructure, operational support, and consistency between batch training data and online inference data.
Summary
Rupesh Gupta explains how recommender systems use recent user actions to infer changing intent and preferences. Batch pipelines can leave a delay of hours or days, while near-real-time feature generation can update recommendations within seconds or minutes. The guests distinguish this from real-time inference, which concerns millisecond responses. They advise teams to optimize models and batch features first, then add real-time versions of features that already matter. Historical and recent features should work together so a recommender can balance long-term interests with short-term needs. Sasha Ovsankin describes LinkedIn's difficulties with streaming processors, including the operational burden on machine learning engineers and the need for managed infrastructure. They discuss Apache Pinot, precomputed feature pipelines, ease of use, portability between batch and streaming, and the need for training-serving consistency. Rupesh gives examples of gains in job, feed, and search recommendation systems, with newer and less active members benefiting most.
Recent actions let recommenders respond to changing intent
Rupesh Gupta says recommender systems infer intent and preference from actions because users rarely state them directly. Batch processing can create a delay of several hours or even days before an action becomes a feature. A person who previously searched for web developer jobs in San Francisco may begin searching in New York, while the recommender continues using the older preference. Near-real-time features reduce this delay, allowing the system to adapt recommendations to what the user is doing now.
Feature values summarize recent actions into numbers or vectors
The common approach is to turn recent actions into features that a model can use. Rupesh gives the example of collecting jobs a user applied to during the previous 24 hours, obtaining an embedding for each job, and averaging those embeddings. The resulting vector represents the kinds of jobs the user preferred recently. A simpler feature might count applications to jobs in San Francisco or New York during the same period. The summary makes the action history easier for a model to process.
Near-real-time features usually mean seconds or minutes
Sasha Ovsankin separates several meanings of real time. In this conversation, the focus is near-real-time generation of feature values from recent events. That usually means a delay of seconds to minutes and often involves streaming systems. He distinguishes this from real-time inference, which he describes as operating at millisecond scale, and from real-time systems such as robotics and self-driving cars.
Teams should add real-time features after fixing basic model problems
Rupesh argues that real-time features are useful for recommender systems that are already well optimized. Teams should first improve the model structure and the existing batch-computed features. Since adding streaming features takes real effort, they also need a way to decide which features deserve that effort. His practical rule is to start with real-time versions of batch features that are already important to the model.
Recent and historical features need to work together
Rupesh says real-time features should complement batch features rather than replace them. A model might use the average embedding of jobs applied to today from a real-time pipeline, while still using the average embedding of jobs applied to during the previous day or month from a batch pipeline. These separate time windows help the recommender balance short-term intent with long-term intent. Historical data is especially limited for new users, so their current-session activity becomes more useful.
Streaming infrastructure creates a large operational burden
Sasha describes the difficulty of making real-time features practical at LinkedIn. Existing streaming processors could support some use cases, but machine learning engineers found them hard to run. Apache Pinot helped with use cases where the system could aggregate a manageable amount of member activity. Larger aggregations over jobs or content required precomputed feature processors. Those processors introduce distributed-systems failures, hardware problems, and many separate services that need specialist support.
Managed services and simple configuration improve adoption
Sasha says LinkedIn's streaming team developed a managed service so machine learning engineers could run feature pipelines without operating the underlying infrastructure. Rupesh adds that machine learning engineers are more likely to adopt real-time features when access is exposed through a simple configuration rather than requiring them to learn infrastructure work. The division of responsibility between application, infrastructure, and machine learning engineers is part of the system design.
Batch and streaming definitions need portability and consistency
Sasha identifies portability between streaming and batch as an under-discussed problem. Ideally, a feature definition can work in both modes. The features used during training also need to match the features retrieved during inference. Without that consistency, the model is trained on data that does not match what it sees in production. He says progress exists, but the problem still needs more work.
"I don't care about your infra. Well, I do care about it, but I don't know that much about the infra and I don't want to spend my time responding to these on-call events in the middle of the night."Sasha Ovsankin35:50
Who should watch
You are deciding whether a recommender system needs near-real-time features and want a concrete rule for choosing which features to stream.
Your team has batch pipelines and is worried about the operational cost of adding streaming processors.
You need training-serving consistency across batch and streaming feature computation, especially for new users with little historical data.
