LLM applications produce better answers when they receive rich, current context about both the user and the items being considered.
2
Personalization can progress from historical batch data to recent activity, real-time signals, and conversational memory.
3
A context platform can hide much of the data engineering needed to ingest, transform, evaluate, serve, and join context at inference time.
Summary
Mike Del Balso explains why a general-purpose LLM gives generic recommendations when it knows a destination but does not know the user. In his travel example, a useful system combines destination data with user preferences, history, recent activity, location, weather, traffic, and other current signals. He describes four levels of context: no context, batch data, streaming data, and real-time data, followed by a bonus level for conversational memory. Each level improves the recommendation while adding data engineering work around pipelines, evaluation, serving, joins, cost, and reliability. Del Balso presents Tecton as a context platform that lets teams define signals in Python, create evaluation data, productionize them, and retrieve them at inference time. He is candid that compressing all this information into a prompt, weighting recent data, and avoiding contradictions remain open problems. The approach also depends on companies collecting useful user data in the first place.
Generic LLM knowledge does not provide personal context
Del Balso contrasts an ordinary LLM recommendation with the answer a personalized travel product should give. ChatGPT may know that Greece is a popular destination, but it does not know whether a particular person likes rock climbing, prefers desert places, or has a certain budget. His target experience combines the expertise of a travel agent with the personal knowledge of a best friend. In the example, that context leads to Utah and a specific national park instead of a generic destination. He argues that many products could improve their experience in the same way if they connect proprietary business data to the model.
Naive RAG retrieves candidates without enough context
The basic RAG design embeds a user query, searches a vector database, retrieves similar destinations, and places those candidates in a prompt for the LLM. Del Balso says this does not solve personalization because the candidates contain little information about the individual user or the current situation. He proposes enriching each candidate with data such as current weather, activities, cuisine, safety, and budget. This information may be structured or unstructured. It needs to be added at inference time because it changes after the model was trained. The model can then reason over richer destination candidates rather than relying mainly on memorized information.
Personalization needs user data alongside destination data
Destination context alone can still produce the wrong answer. Knowing that the Olympics are in Paris may lead to a reasonable recommendation, but it is not useful if the user dislikes sports. The system also needs preferences, interests, history, past actions, and facts such as budget. Del Balso describes a prompt containing both destination information and user data. Building it requires connections to a vector database, destination data in a data lake or other store, and user data. His central claim is direct: the quality of the response depends on the quality of the context supplied to the model.
Context personalization progresses through four data levels
Del Balso describes a progression from no context to batch, streaming, and real-time context. Batch data captures historical behavior, profile information, stated preferences, and favored activities. Streaming data captures recent intent, such as videos watched or searches made in the last few minutes. Real-time data includes the user's current location and external information such as weather and traffic. Each level can improve the recommendation, but each also adds engineering work. He later adds conversational memory as a bonus level, allowing the system to incorporate a user's correction, such as asking for a warmer destination.
Batch personalization still requires substantial data engineering
At the batch level, a system can use trip history, interests, stated preferences, and past activities from a data warehouse. The hard part is constructing the context and joining warehouse data with retrieved candidates so the model receives the intended prompt. Teams also need pipelines for retrieval and serving, plus historical datasets for evaluation and backfilling. Del Balso presents Tecton's workflow as a way to define a signal and its transformation in a small amount of Python, create an evaluation dataset, push the signal to production, and read it at inference time. The example changes a generic destination into a recommendation for Kyoto based on a preference for places with deep history.
Streaming data captures recent intent but raises operational risk
Streaming context records what the user has done recently, such as the locations they viewed or the topics covered by videos they watched during the past hour. This can make recommendations more relevant because it captures current intent rather than only long-term preferences. Del Balso is direct about the cost of this approach. Streaming pipelines remain difficult to build, operate, and maintain, and work at scale can create a serious performance-versus-cost tradeoff. A team focused on LLMs may not have streaming expertise, which increases the risk of delays or accidental overspending. He applies the same Python, evaluation, production, and inference workflow to streaming signals.
Real-time context adds current location and external signals
Real-time context uses information available at the moment of inference. Examples include the user's current phone location, their inputs, weather at that location, and traffic. Del Balso says these signals can reveal immediate intent and support recommendations such as a last-minute opportunity near the user. They also make the system harder to run because the team must maintain production data pipelines and integrate third-party sources, potentially through contracts with weather or traffic providers. The system must balance speed and cost. His example signal measures how close a user is to a destination or whether they are in the same country.
Prompt construction and context weighting remain open problems
During questions, Del Balso says the system still needs a method for combining all retrieved information into the final prompt. Teams need to compress the context into tokens and decide how much attention to give recent data compared with older batch data. They can tell the model to pay closer attention to recent information, but he describes the broader problem as still open. He also says vector database results can be kept lightweight and overridden with more accurate or trusted first-party data. The architecture is composable, so teams can add filters and handle product-specific behavior rather than following one fixed template.
"Streaming data pipelines still are really hard to build and operate and maintain for production."18:00
Who should watch
You are building an LLM product that gives generic answers because it lacks user history, preferences, or current signals.
Your team needs to connect batch, streaming, and real-time data to prompts without building every serving and evaluation system by hand.
You want a practical account of the engineering costs behind personalized RAG, including prompt size, data freshness, streaming operations, and third-party integrations.
