PodcastFoundational Models are the Future but...ClaimAlex Ratner says fine-tuning on labeled data is generally needed to move foundation models from generic or generative behavior to accurate performance on difficult, custom enterprise tasks.27:32
Threads / Open weights against the API / 2023
Open weights against the API in 2023
44 sessions
PodcastFoundational Models are the Future but...
PodcastMLOps vs ML-as-a-ServiceClaimFoundational model applications have mostly been consumer-style applications rather than products tied to proprietary data or proprietary environments.8:13
MeetupLet's Talk About Raw DocumentsClaimPipeline notebooks are the source of truth for generating preprocessing APIs for specific domains.10:38
MeetupThe 7 Lines of Code You Need to Run Faster Real-time InferenceClaimOpenVINO is an open-source toolkit for optimizing and deploying AI inference.1:01
Solving the Last Mile Problem of Foundation Models with Data-Centric AIPushed backAlex argues that the most durable advantage will come from private data, domain-specific knowledge, and last-mile development rather than from closed general-purpose API models.7:54
Want High Performing LLMs? Hint: It Is All About Your DataPushed backVikram Chatterji rejects the idea that teams should exclusively prompt or exclusively fine-tune, arguing that the right balance depends on the use case.32:53
PodcastThe Birth and Growth of Spark: An Open Source Success StoryClaimMatei Zaharia says the open-source Spark community grew rapidly because companies and researchers wanted easier programming interfaces, libraries, and an ecosystem for large-scale data applications.17:09
DevTools for Language Models: Unlocking the Future of AI-Driven ApplicationsClaimLarge language models usually have a billion or more parameters and are designed to be adopted and fine-tuned for specific data.1:27
Building Defensible Products with LLMsClaimSupervised fine-tuning can use successful production examples to improve a model for a specific subset of tasks.13:32
MeetupFine-Tuning LLMs: Best Practices and When to Go SmallClaimMark Kim-Huang says fine-tuning open-source large language models is the focus of the session.1:42
Pitfalls and Best Practices: 5 Lessons from LLMs in ProductionPushed backRaza argues against starting with complex chains or agents before testing stronger models, prompt engineering, and fine-tuning, while acknowledging that agents and chains work in some situations.13:38
Taking ImgFlip's 'This Meme Does Not Exist' to the Next Level with a LLMClaimThe Llama model version they tried had a license that forbade commercial use.6:20
PodcastOpen Source and Fast Decision MakingClaimRob Hirschfeld and Dave McCrory patented methods for managing infrastructure and providing an API for infrastructure managed by another company.7:07
Build and Customize LLMs in Less than 10 Lines of YAMLPushed backTravis argued that fine-tuning a smaller model can match or outperform a much larger model for a sufficiently bounded task at lower latency and cost.17:17
Building and Curating Datasets for RLHF and LLM Fine-tuningClaimArgilla can be used to collect feedback for LLMs, fine-tuning, RLHF, and other NLP tasks.1:12
Understanding the LLM EconomicsPushed backNikunj Bajaj says smaller open-source models are not generally as capable as larger commercial models, but can perform comparably on simpler task-specific workloads after fine-tuning.28:00
PodcastPython Power: How Daft Embeds Models and Revolutionizes Data ProcessingPushed backSammy Sidhu disagreed with the common prediction that open-source models would become good enough within six months, estimating that it could take one to three years instead.48:25
Create a Contextual Chatbot with LLM and a Vector Database in 10 MinutesClaimRaahul Dutta says the team tested several language models and obtained good results from Falcon 7B after fine-tuning it on their dataset.5:02
Navigating Through the Generative AI LandscapeClaimAzin Asgarian identifies fine-tuning, prompt optimization, and reinforcement learning with human feedback as approaches for customizing models.6:44
Lessons Learned Productionising LLMs for Stripe SupportClaimFine-tuning the sequential GPT framework completely mitigated hallucinations in Stripe’s case.3:54
The Next Million AI AppsClaimThe workshop focuses on preparing data for fine-tuning, especially the quantity and quality of the data, and then running a fine-tuning job.1:25
Building Recommender Systems with Large Language ModelsClaimResearchers have fine-tuned LLMs with recommendation data by converting user-item interactions into instructions.5:50
RLHF Data Collection in PracticeClaimSurge AI has partnered with Anthropic, OpenAI, Google, and others to provide data for advanced systems.1:54
End-to-end Modern Machine Learning in ProductionClaimStarCoder is an open-source, community-led Copilot replication that can generate code in many programming languages.1:21
LIMA: Less is More for AlignmentClaimThe quality and diversity of annotated data, including domain and task diversity, are critical when creating a fine-tuning dataset.8:13
MLOps vs LLMOpsClaimAlex Ratner says LLM training workflows will focus more on data and context operations, including prompts, examples, fine-tuning, and external context, than on hyperparameter or architecture selection.11:52
Enabling Defense Missions with Local LLMsClaimHighly regulated environments have stringent deployment and operation requirements, including authorization to operate and artifact-control requirements for containers and model weights.5:09
LLM XGBoost: Can a Fine-Tuned LLM Beat XGBoost on Tabular Data?ClaimThe experiment translated CSV rows into natural-language sentences and fine-tuned a sequence-to-sequence model to predict yes or no labels.3:29
PodcastAll About Evaluating LLM ApplicationsPushed backShahul Es argues that fine-tuning is not an alternative to retrieval-augmented generation because fine-tuning does not inject new factual information.38:48
PodcastThe Centralization of Power in AIPushed backKyle Harrison disputed the idea that Meta's role in open-source AI is inexplicable, pointing to the company's lack of a cloud-computing business and its product incentives.22:40
Finetuning Open-Source LLMsPushed backSebastian Raschka disputes the implication that full fine-tuning is always necessary, saying that a few Transformer blocks can sometimes reach saturated performance.24:40
What Drives GenAI Development in the Next 3 YearsPushed backEuro Beinat says proprietary models are still better almost everywhere on benchmarks, but acknowledges that fine-tuned open-source models can exceed standard models for narrow tasks.12:05
PodcastMLOps vs ML OrchestrationPushed backKetan Umare disputes the idea that every requested capability should be added directly to open-source Flyte, because doing so could increase dependencies and hurt smaller installations.23:40
Efficient Serving of LLMs for Experimentation and Production with Fireworks.aiClaimFine-tuning can give businesses more control over model output than prompt engineering alone.1:36
LLM ValleyPushed backLilly Chen disputes the idea that incumbent companies automatically prevent B2B SaaS startups from winning; she says startups can win through rapid iteration and ground-up design.3:36
Data Quality's Impact on Large Language ModelsClaimMany companies are trying to use large amounts of old enterprise data for categorization, search, chat, and summarization, while often preferring open-source models because of privacy and licensing concerns.7:18
Deploying LLMs on Structured Data Tasks: Lessons from the TrenchesClaimIn-context learning lets one model perform many tasks without fine-tuning by changing the input text.1:56
AI Squared: Breaking LLMs out of the Chat ApplicationPushed backBenjamin Harvey disputes the expectation that an open-source model can become highly accurate without the extensive, continuing fine-tuning used by systems such as ChatGPT.37:37
The State of Open Source AI: Deployment Engines, Licences, & HardwarePushed backCasper da Costa-Luis disputes the idea that laws addressing software liability are aimed at open-source developers, saying some open-source developers are fighting on behalf of closed-source developers.6:32
MeetupLLMs Mini Summit // MLOps Mini Summit Meetup #3Pushed backThomas Capelle argues that LoRA usually provides nearly the same performance as full fine-tuning at much lower cost.15:07
PodcastDSPy: Transforming Language Model Calls into Smart PipelinesClaimDSP can compile the same program into different implementations, including zero-shot or few-shot prompts and fine-tuned smaller language models.49:22
PodcastLLMs in Focus: From One-Size Fits All to Verticalized SolutionsPushed backLaurel Orr argued that open models may close much of the performance gap with private models, while Demetrios Brinkmann suggested that OpenAI's large user base and continuous feedback could preserve its advantage.40:32
PodcastThe Role of Infrastructure in ML Leveraging Open SourceClaimNiels Bantilan considers the OpenAI developer announcements largely orthogonal to Union's work because many customers still build deterministic systems and models for tabular, image, video, audio, and other data.32:01