# Product Enrichment and Recommender Systems

Marc Lindner & Amr Mashlah, eezylife Inc. | MLOps Coffee Sessions | Episode 114 | 56:12
Hosted by Skylar Payne

Source: https://www.youtube.com/watch?v=2wU3qS7D49c
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/product-enrichment-and-recommender-systems
Published: 2022-08-10
Tags: data-engineering, embeddings, multimodal, recommender-systems

## TL;DR
- eezylife combines catalogs from services such as Ticketmaster, OpenTable, and Spotify with user preferences, location, weather, mood, and availability.
- The hardest part of recommendation is often describing the product accurately, since concepts such as an intellectually stimulating movie or a romantic restaurant have no simple ground truth.
- The team standardizes data into materialized PostgreSQL views called candidates, then combines feature-based, embedding-based, collaborative, and rule-based scores.

## Summary
Marc Lindner and Amr Mashlah describe how eezylife builds recommendations across movies, restaurants, recipes, museums, concerts, and other activities. The product collects catalog data from many providers, normalizes it, removes duplicates, enriches items with text and image features, and matches them with user preferences and current context. They explain why product understanding is harder than user profiling: someone can say they want an intellectually stimulating movie, while labeling a movie that way is subjective. Their system combines content features, text embeddings, collaborative filtering, and hard constraints such as opening hours. The team also relies on manual product judgment, onboarding data, user analytics, and iterative prototypes to catch poor recommendations. Marc and Amr discuss cold-start recommendations across domains, recommendations for couples, the limits of group recommendations, and the privacy question created by a system that knows a user's preferences well.

## Key ideas
### eezylife turns many product catalogs into one personal activity guide
[07:00](https://www.youtube.com/watch?v=2wU3qS7D49c&t=420s)
Marc Lindner explains that eezylife began with the idea of reversing the usual ad-platform relationship. Instead of companies targeting users, users can target products they might want. The team aims to collect catalogs from partners such as Ticketmaster, OpenTable, and Spotify, then match those products to a person's situation. Recommendations can include a restaurant table, a concert with tickets still available, a museum, a Netflix movie, or another activity. The system uses location, weather, mood, and stated preferences, so the same request can produce different results for different people.

### Product understanding is harder than collecting user preferences
[10:29](https://www.youtube.com/watch?v=2wU3qS7D49c&t=629s)
Marc Lindner describes two problems that pull in opposite directions. It may be easy to ask whether a user wants an intellectually stimulating movie, while it is difficult to decide which movies fit that description. The same issue appears with concepts such as a romantic restaurant. Amr Mashlah says the team extracts features from plot text, reviews, and other item information, including labels such as action-packed, based on a true story, intellectual, or easy to watch. Those features then have to be matched with the user's mood, the time of day, and other context.

### Multiple providers create identity, schema, and image-duplication work
[13:16](https://www.youtube.com/watch?v=2wU3qS7D49c&t=796s)
Amr Mashlah says that combining catalogs requires identifying when records from different sources refer to the same item. Movies often have IMDb IDs or other universal identifiers, while restaurants may need matching based on name and location. The work grows as more sources are added, so the team limits deduplication to suitable candidate subsets. Images create another problem because several providers may supply near-duplicates from the same angle. Amr says the team used CLIP embeddings to label and group images, and processed more than 20 million images as new data arrived.

### A managed GPU workflow lets the data team scale image enrichment
[15:31](https://www.youtube.com/watch?v=2wU3qS7D49c&t=931s)
Amr Mashlah describes using a managed infrastructure tool called Modal to run CLIP and other embedding models across multiple GPUs. The team can specify its computing needs without writing Docker files or managing individual instances. Models can come from TensorFlow Hub or Hugging Face. In the image workflow, the system reads image URLs from the database, runs labeling, and writes the results back. Marc Lindner says the tool gives a data scientist who does not specialize in cloud infrastructure a simpler way to run large inference jobs.

### Standard candidate views make new sources easier to ingest
[20:21](https://www.youtube.com/watch?v=2wU3qS7D49c&t=1221s)
Marc Lindner says the team stopped maintaining small machine-learning solutions that created more operational work than value. Instead, it defined a common path from incoming source data to recommendation-ready data. Their candidates are materialized PostgreSQL views that gather the required fields from many tables and put them into the format used by the recommender. A data team member maps a new provider's API output to that standard, and a scheduled job ingests it daily. Amr Mashlah says this iterative standardization made it easier to add sources, scrape a new area, or open a new city.

### The recommender combines several model families with hard constraints
[24:09](https://www.youtube.com/watch?v=2wU3qS7D49c&t=1449s)
Amr Mashlah says the system produces multiple scores and combines them with rules or an additional teacher model. Marc Lindner distinguishes feature-based scoring, text embeddings, collaborative filtering, and rules that act more like filters. A request for something tomorrow evening must exclude events that are not open or available then. Preferences can also transfer between domains. Food preferences can inform restaurant and recipe recommendations, while music preferences can identify concerts by favorite artists.

### The team uses cross-domain embeddings and fast prototypes for cold start
[29:02](https://www.youtube.com/watch?v=2wU3qS7D49c&t=1742s)
Marc Lindner says text embeddings can connect domains that users would not normally link directly. Someone's movie preferences can help produce museum recommendations, and the team found examples such as a relationship between The Godfather and an Italian restaurant. Amr Mashlah prototypes these ideas on small data sets, often inspecting CSV files or using a small interactive tool. They meet frequently, change the approach, and look at whether the results seem useful and understandable before scaling the idea. Marc is candid that intuition can fail when a promising prototype must later run over tens of millions of records.

### Bad recommendations require product rules and direct inspection
[31:45](https://www.youtube.com/watch?v=2wU3qS7D49c&t=1905s)
The team treats disappointment as a product problem as well as a modeling problem. Marc Lindner gives vegetarian restaurants as an example: restaurants may claim to offer vegetarian food even when the first image shows meat. eezylife also has to cope with empty local inventories, new users, and impossible conditions such as asking for cinemas during the COVID lockdown. Amr Mashlah says the first rule for a new user is to avoid something they explicitly disliked. The team records recommendation events, reviews them in Mixpanel, and has product staff inspect cases where a user's request produced no result or a poor result.

### Personalization creates a trust and privacy decision
[52:21](https://www.youtube.com/watch?v=2wU3qS7D49c&t=3141s)
Marc Lindner says the ethics depend partly on who holds the data and what they intend to do with it. A user may accept an assistant or partner knowing a great deal about them because the purpose is to help, while an advertising platform may use the same knowledge to keep the user engaged. He says eezylife has to remain clear about its purpose and keep the data with the company rather than sharing it. The product vision includes more dynamic conversations, full-day plans, friends who may want to join, and recommendations that surprise users, but Marc acknowledges that broader knowledge increases the need for trust.

## Notable quotes
- Marc Lindner: "Whenever you clearly know what the user wants it's very very hard to know that that product has that and whenever you very clearly know that this product has that it's very difficult to know if that's exactly what the user wants." (10:29)
- Amr Mashlah: "We need from the recommender people need to look at standardized view that gives you what can be recommended and data team work on reaching this standard view." (22:51)
- Amr Mashlah: "The first rule is not to disappoint and not to do something stupid before you start try to do something smart there." (35:28)
- Marc Lindner: "I don't think that there's ever gonna be a point where the recommender system is finished." (38:09)
- Marc Lindner: "If our intention is purely to get you to have a better time and live a more fulfilled life then you as user have to decide: do I give that up, that piece of privacy, because I want that and I know what their intention is?" (53:10)

## Tools & references mentioned
- eezylife Inc.
- Ticketmaster
- OpenTable
- Spotify
- Netflix
- Hulu
- HBO Max
- IMDb
- CLIP
- Modal
- TensorFlow Hub
- Hugging Face
- AWS
- PostgreSQL
- Mixpanel
- The OCEAN model
- Instagram
- TikTok
- LinkedIn

## Who should watch
- You are building recommendations over several product types and need to merge catalogs from different providers.
- Your recommender produces plausible scores but still shows obviously poor results, empty feeds, or items that conflict with user preferences.
- You are deciding how much context and personal data an assistant should use, and need to think about user trust alongside model quality.

## Related talks

- [RecSys at Spotify](https://mlopstalks.com/talks/recsys-at-spotify) (Sanket Gupta, Spotify, 50:25)
- [Recommender System: Why They Update Models 100 Times a Day](https://mlopstalks.com/talks/recommender-system-why-they-update-models-100-times-a-day) (Gleb Abroskin, FunCorp, 49:01)
- [System Design for Recommendations and Search](https://mlopstalks.com/talks/system-design-for-recommendations-and-search) (Eugene Yan, Amazon, 58:23)
- [Building a Movie Recommendation System on Tecton with Snowflake](https://mlopstalks.com/talks/building-a-movie-recommendation-system-on-tecton-with-snowflake) (David Hershey, Tecton, 54:08)
- [RagSys: RAG is just RecSys in Disguise](https://mlopstalks.com/talks/ragsys-rag-is-just-recsys-in-disguise) (Chang She, LanceDB, 12:45)