Generative AI models differ in size, architecture, training data, objectives, task focus, and ethical alignment, so companies need to choose them against their own data, cost, privacy, and performance needs.
2
Companies can adapt models through fine-tuning, instruction tuning, prompt engineering, prompt optimization, and reinforcement learning with human feedback.
3
Azin recommends a crawl, walk, run approach: begin with simple prompt-based experiments, establish a baseline, then consider chaining, agents, fine-tuning, foundation-model training, or distillation.
Summary
Azin Asgarian gives startups a way to think about the rapidly changing generative AI field. She compares models by size, architecture, training data, objectives, task coverage, and ethical training, then discusses how to choose among them using benchmarks, leaderboards, business requirements, cost, data volume, privacy, and performance. She explains several adaptation methods, from fine-tuning and instruction tuning to prompt optimization and reinforcement learning with human feedback. Larger models create memory, serving, and cost problems, which has led to tools and practices often called LLMOps. Azin also covers responsible AI, multimodal models, and platforms that help teams work with generative systems. For finding product opportunities, she separates pain relievers, such as interfaces that help customers search overwhelming documentation, from gain creators, such as tools that support creativity. Her practical recommendation is to start with fast, simple experiments and add complexity only after a useful baseline exists.
Model choice depends on the use case and its constraints
Azin says models differ in size, architecture, training data, pre-training objective, task focus, and ethical training. She contrasts task-agnostic and task-specific systems, and mentions Anthropic's focus on models that are helpful, harmless, and honest. Companies need to decide which model fits their use case by weighing data, cost, ethics, and performance. An off-the-shelf model may be easy to use, but she says simple solutions do not create much competitive advantage. The choice should begin with the company's actual problem and requirements.
Benchmarks and leaderboards give teams a place to start
Azin points to benchmarks and leaderboards as practical tools for comparing models. She mentions the Stanford HELM leaderboard, which evaluates models across tasks and metrics, as well as the Hugging Face leaderboard for models available on the Hugging Face Hub. She also describes human and GPT-based evaluations for chatbots. These comparisons can help a team narrow its options, although the final choice still depends on the intended application and constraints.
Model customization ranges from prompts to human feedback
The talk presents several ways to adapt a model. Traditional fine-tuning can use input-output examples, while instruction tuning gives a model instructions for desired behavior. Prompt engineering began with manually written prompts and has developed into prompt optimization, where algorithms generate or tune prompts. Discrete prompts can remain readable, while continuous or soft prompts may become uninterpretable strings that still improve performance. Azin also describes reinforcement learning with human feedback as a way to align a chatbot with user feedback and human values.
Serving and modifying large models becomes difficult as their memory footprint grows. Azin says copying or adjusting a very large model may not be feasible because of the cost and amount of memory required. This has produced the term LLMOps, which she describes as an extension of MLOps for large language models. She groups the available tools into no-code, local, and code-first platforms, along with frameworks that automate parts of training and agent creation. The right choice still depends on the problem and business requirements.
Responsible AI needs to be part of product decisions
Azin asks companies to consider the broader social impact of their systems, possible negative consequences, and guidelines that could reduce those risks. This matters especially for products that interact directly with end users. She uses Anthropic's work on making chatbots helpful, harmless, and honest through human feedback and selected data as an example. In her view, responsible design is both an ethical requirement and a factor that can affect a company's position in the market.
Multimodal models may unlock different applications
Azin discusses multimodal AI as a separate direction from language-only models. She expects access to GPT-4's vision capabilities to enable more visual-language work, including visual question answering, image analysis, and generating text from images. She advises teams to understand the current capabilities and limitations of multimodal systems, then consider whether new applications could affect their own work or business.
Generative AI opportunities can relieve pain or create gains
Azin groups product opportunities into pain relievers and gain creators. A pain reliever addresses a problem, reduces risk, or removes a negative experience. Her example is a documentation interface or agent that helps customers find answers when a product has too much information. A gain creator increases value, surprise, delight, or happiness. She gives the example of using generative AI to help someone who finds it hard to be creative on demand, such as by producing a marketing idea.
Start with simple experiments before adding complexity
Azin recommends a crawl, walk, run framework. Teams should run many fast iterations, fail quickly, and learn quickly instead of adding every capability at once. Crawling can mean learning prompt engineering and prompt tuning, then building a simple baseline. Walking can involve chaining, agents, or fine-tuning. Running may include training a foundation model on proprietary data or distilling models for efficiency. Her advice is to add these more demanding approaches only after the simpler foundation is working.
