LLMs can reduce the hand-built feature engineering and clustering used in traditional recommender systems by interpreting user and content information directly.
2
Using an LLM during feed inference creates latency and infrastructure problems, so a smaller distilled model or an offline LLM plus traditional ranking may be more practical.
3
A traditional recommender is often the better choice when it already performs well enough, because an LLM adds cost, complexity, and harder debugging.
Summary
Arpita Vats compares traditional recommender systems with LLM-based approaches through examples from content feeds and sequential recommendations. Traditional systems depend on explicit features, user clusters, and separate models. LLMs can infer smaller signals from user and item information without being told every feature to inspect. That can reduce manual setup, but direct LLM inference is too slow and expensive for many feed-serving situations. Arpita describes lighter models, knowledge distillation, and offline LLM use followed by traditional ranking as possible designs. She says evaluation can still use user reactions such as likes, comments, and attention. The discussion also covers cold-start recommendations, privacy, agentic systems, and debugging. Her position is practical: LLMs may help where their broader understanding adds value, but teams should compare them with existing models and use them only when the improvement justifies the infrastructure cost.
A recommendation depends on many small signals about a person and a post
Arpita explains that a feed model considers when someone tends to browse, what formats they prefer, which people or feeds they recently interacted with, and which subject areas interest them. Someone may like AI content generally but prefer AI infrastructure posts specifically. Timing matters too. A person may scroll quickly in the morning and spend more time reading after work, so the same content may have different value at different times. She says these actions can look meaningless to the user, yet each one provides information for ranking the next post.
Traditional systems make teams specify features and clusters explicitly
The conversation describes a traditional recommender as a system that turns profile data, past interactions, and content properties into explicit features. It may group people into clusters, such as IT or finance, and use one person's reaction to predict what others in the same group might like. This approach can miss the reason behind an action. Demetrios gives the example of liking a LinkedIn post to support a friend, which does not necessarily mean wanting more posts on that subject. The system still treats the reaction as a positive signal unless the team finds a way to distinguish those intentions.
LLMs shift some recommendation work from feature engineering to prompting
Arpita says an LLM can receive a user and a candidate post, then infer which signals matter without being given a long list such as recent likes, time windows, and content preferences. The model has learned patterns from large amounts of data and can extract details that a traditional system would need engineers to define. She describes this as a shift from deciding which features to provide toward writing prompts that make the model use the information the team wants. She is also honest that finding the right prompt remains difficult and is still an active area of work.
A feed cannot wait for a large model to make a separate call for every candidate and every user. Arpita identifies latency as the main deployment problem. One option is a lightweight model designed for inference. Another is knowledge distillation, where a smaller student model imitates a larger teacher model. A third design keeps the LLM offline, using it to generate features or candidate information during training, while a conventional machine-learning model performs the final ranking online. This preserves some LLM capabilities without putting the full model in the serving path.
Evaluation can still focus on whether users value the recommended content
Arpita says the evaluation target does not change much when an LLM replaces another recommender. If a system selects five posts, the team can compare whether users liked, commented on, paid attention to, or ignored those posts. She gives the example of comparing five LLM-selected posts with five posts from a traditional model. The important question is whether the recommendation produced value and interaction. The model type changes how candidates are produced, but user response remains the practical test of whether the recommendation worked.
An LLM is not automatically worth its infrastructure cost
Arpita argues that teams should compare an LLM with the model they already have instead of adopting it because it is fashionable. If a traditional recommender achieves the needed result, an LLM may add infrastructure expense without improving the feed. LLMs can reduce the number of hand-built clustering and recommendation stages, but traditional systems may use several smaller models with lower serving costs. The trade-off is between operating more models that need retraining and maintaining one larger model that needs more hardware and has a harder inference problem.
LLMs can help with sequential and cold-start recommendations
Arpita describes sequential recommendation as predicting what a person is likely to do after an earlier action. After booking a flight, a system might suggest a car, a hotel, and later activities. A traditional design may need separate logic for each step, while an LLM can produce a sequence offline and let a normal model choose the specific item. She also says LLMs can handle cold-start cases more naturally because a small amount of information, such as a person's broad preferences, can produce a useful initial recommendation. More personal results still require the system to learn additional preferences over time.
Agentic systems add privacy and debugging problems
Arpita expects agents could change recommendation systems, but she does not consider them reliable enough yet. Companies would need to control what user information an agent receives and verify that it is used correctly. Debugging is also harder. With a traditional model, engineers can inspect the stages and tune the part producing a bad result. With an LLM, they may be only partly sure why an output was produced. With an agent coordinating several steps, that uncertainty increases, making it difficult to locate the cause of irrelevant recommendations.
