# Lessons from Building LLM-based Social Media Products

Faizaan Charania, LinkedIn | AI in Production 2024 | 19:46

Source: https://www.youtube.com/watch?v=v21fCCuVMQg
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/lessons-from-building-llm-based-social-media-products
Published: 2024-04-22
Tags: evals, product-strategy, prompt-engineering, structured-outputs

## TL;DR
- Faizaan Charania says teams should start with user problems and treat LLMs as tools rather than as the product itself.
- Prompt engineering often gives teams a faster and cheaper path than fine-tuning, but it needs a repeatable evaluation process before launch.
- Products using generative AI need to earn trust by limiting hallucinations, explaining their behavior, using differentiated data, and measuring how people actually use them.

## Summary
Faizaan Charania describes how he and his teams brought generative AI features to LinkedIn users. He starts with product discovery: talk to users, test assumptions, and explore interaction patterns beyond chatbots. During implementation, teams need to plan GPU capacity, while deciding whether prompt engineering or fine-tuning fits the use case. He recommends giving prompt engineering a serious trial because it can reduce technical effort and infrastructure needs. Prompts must then be treated like models, with fixed evaluation data, explicit criteria, and repeated testing. After launch, trust becomes a product concern. Teams need to reduce hallucinations, explain refusals and policy boundaries, and use their data and personalization capabilities to stand apart from similar foundation-model products. Success should be measured through adoption and funnel behavior before teams focus on scale and monetization.

## Key ideas
### Start with the user's problem instead of the model
[02:23](https://www.youtube.com/watch?v=v21fCCuVMQg&t=143s)
Faizaan says LLMs are a tool, so product teams should begin by understanding where users struggle and what job they would hire the product to do. User research for generative AI needs an extra step because new interaction patterns are possible. A chatbot is only one option. Teams can test how AI fits into an existing product through prototypes and conversations with users. In one January 2023 research effort, his team tested a few major assumptions before choosing between two directions. The focused research helped them learn what might work and saved time.

### Explore several model and interaction options before committing
[06:24](https://www.youtube.com/watch?v=v21fCCuVMQg&t=384s)
After research, Faizaan recommends prototyping with different approaches. Teams can try large models, smaller open-source models, and different ways of placing AI inside the product. The exploration stage should not be overly prescriptive. Early assumptions about how people use a product today may prevent teams from seeing how they could use it tomorrow. The point of prototypes is to give the team room to find an interaction that fits the user's need, rather than forcing every use case into a chatbot.

### Prompt engineering is often the first implementation choice
[07:05](https://www.youtube.com/watch?v=v21fCCuVMQg&t=425s)
GPU capacity and budget depend on decisions such as prompt engineering versus fine-tuning. Faizaan argues that prompt engineering is likely to work for most product use cases. It can shorten time to market, has a lower technical barrier, and uses fewer GPUs. Fine-tuning still fits narrow cases where the margin for error is very small. He gives an example from his work: after prompt improvements, GPT-3.5 Turbo performed at or above GPT-4 for their particular use case, even though GPT-4 was better out of the box.

### Evaluate prompts like machine learning models
[10:40](https://www.youtube.com/watch?v=v21fCCuVMQg&t=640s)
A prompt that looks good in a demo can expose hallucinations and logic gaps when used by thousands or millions of people. Faizaan recommends treating the prompt as a model and removing subjectivity from evaluation. Teams should create a comprehensive output evaluation framework, keep the validation data and rules fixed, compare each new prompt version against the same examples, and measure where the prompt fails after every iteration. For article generation, criteria might include language, structure, length, variety of subject matter, and a mix of price points.

### Build trust into the product's responses
[15:26](https://www.youtube.com/watch?v=v21fCCuVMQg&t=926s)
Generative AI produces content that can represent a company or brand, so trust matters before teams measure conventional product success. Faizaan uses ChatGPT's response to a harmful stereotype as an example. The system refused the premise, explained that the request could violate policy, and gave users a feedback path. He says products should protect against hallucinations, communicate their stance, and use adaptive feedback to improve inclusion. A product that produces nonsensical output can lose users who do not return.

### Differentiate through data and personalization
[17:11](https://www.youtube.com/watch?v=v21fCCuVMQg&t=1031s)
Many products can be powered by similar foundation models, which may make the model itself less distinctive over time. Faizaan recommends differentiating through proprietary or differentiated data and through personalized experiences. Once a product is online, teams should learn from feedback and communicate relevant changes to users. This can build trust while giving the product a reason to exist beyond access to a common model.

### Measure real usage before scaling and monetizing
[18:11](https://www.youtube.com/watch?v=v21fCCuVMQg&t=1091s)
At launch, learning is the main goal because a new tool can break the assumptions behind an old workflow. Teams should watch adoption, inspect the funnel, find bottlenecks, and compare actual behavior with their intended use. If users are not using the product as expected, Faizaan says teams should be willing to change direction. He puts product-market fit before scaling to millions of users, and scaling before monetization.

## Notable quotes
- "LLMs are just a tool and the core focus that we have should be on how it can be used to solve your user pain point." (04:23)
- "Prompt iteration is the key." (10:40)
- "Think of this prompt as your own machine learning model and remove subjectivity from the output evaluation." (11:19)
- "Build a good experience then scale and then monetize." (19:37)

## Tools & references mentioned
- LinkedIn
- Yahoo
- Microsoft
- Power of Prompting
- GPT-4
- GPT-3.5 Turbo
- GPT-4 Turbo
- Med-PaLM 2
- Hugging Face
- Expedia
- ChatGPT

## Who should watch
- You are deciding whether an LLM feature should be a chatbot, an embedded workflow, or something else inside an existing product.
- Your first prototype works in a demo, but you need a disciplined way to test prompts at larger user volumes.
- You are launching an AI product and need to connect trust, user feedback, adoption, and infrastructure costs.

## Related talks

- [Product Engineering for LLMs](https://mlopstalks.com/talks/product-engineering-for-llms) (Charles Frye, Full Stack Deep Learning & Sahar Mor, Stripe & Sarah Guo, Conviction & Shyamala Prayaga, NVIDIA & Willem Pienaar, 31:46)
- [Reliable LLM Products, Fueled by Feedback](https://mlopstalks.com/talks/reliable-llm-products-fueled-by-feedback) (Chinar Movsisyan, Feedback Intelligence, 49:17)
- [Pitfalls and Best Practices: 5 Lessons from LLMs in Production](https://mlopstalks.com/talks/pitfalls-and-best-practices-5-lessons-from-llms-in-production) (Raza Habib, Humanloop, 30:27)
- [From Building Self-driving Cars to Building LLM Applications](https://mlopstalks.com/talks/from-building-self-driving-cars-to-building-llm-applications) (Effy Zhang, Baserun, 10:45)
- [UX of an LLM User](https://mlopstalks.com/talks/ux-of-an-llm-user) (Misty Free, Jasper & Davis Treybig, Innovation Endeavors & Dina Yerlan, Adobe Firefly & Artem Harutyunyan, Bardeen AI, 31:48)