# Open weights against the API

110 sessions · follows the tags open-source, fine-tuning
Page: https://mlopstalks.com/threads/open-weights-against-the-api

When is a fine-tuned open model a better fit than a hosted API?

## 2020

8 sessions.

- [What is the open source ML framework Hermoine](https://mlopstalks.com/talks/what-is-the-open-source-ml-framework-hermoine) (Neylson Crepalde, A3Data). Claim: Hermoine is intentionally open enough that users can reshape it to fit their preferred way of working. [5:07](https://www.youtube.com/watch?v=H6if5q9Uo3w&t=307s)
- [MLflow Open Source Framework Hermione Demo](https://mlopstalks.com/talks/mlflow-open-source-framework-hermione-demo) (Neylson Crepalde, A3Data). Claim: The MLflow interface can be used to compare runs, inspect metrics and register a model for use through an API. [13:20](https://www.youtube.com/watch?v=NfEigZ5ayJE&t=800s)
- [Bring Your On-Prem ML Use Cases to Production on Google Cloud using Kubeflow](https://mlopstalks.com/talks/bring-your-on-prem-ml-use-cases-to-production-on-google-cloud-using-kubeflow) (Chanchal Chatterjee, Google). Claim: Chanchal Chatterjee's team created the open-source ML Pipeline Generator to help bring existing models into production on Google Cloud. [3:02](https://www.youtube.com/watch?v=JZAc_yZgByg&t=182s)
- [The Current MLOps Landscape](https://mlopstalks.com/talks/the-current-mlops-landscape) (Nathan Benaich, Air Street Capital & Timothy Chen, Essence VC). Pushed back: Timothy Chen and Nathan Benaich disagree with the idea that open source is always required for an MLOps business, while agreeing that it is increasingly advantageous for developer-focused products. [43:05](https://www.youtube.com/watch?v=i6HZ2vjFLIs&t=2585s)

4 more from 2020 on this thread: https://mlopstalks.com/threads/open-weights-against-the-api/2020

## 2021

5 sessions.

- [How to Avoid Suffering in MLOps/Data Engineering Role](https://mlopstalks.com/talks/how-to-avoid-suffering-in-mlops-data-engineering-role) (Igor Lushchyk, Adyen). Claim: Adyen replaced a homegrown scheduling system with open source tools because the old system was supported by only one person and had accumulated limitations. [17:41](https://www.youtube.com/watch?v=7L1W6Y1G-sI&t=1061s)
- [MLOps Investments](https://mlopstalks.com/talks/mlops-investments) (Sarah Catanzaro, Amplify Partners). Claim: Sarah Catanzaro is a partner at Amplify Partners, an early-stage venture capital firm that primarily invests in technical tools and platforms. [2:13](https://www.youtube.com/watch?v=twvHm8Fa5jk&t=133s)
- [Tour of Upcoming Features on the Hugging Face Model Hub](https://mlopstalks.com/talks/tour-of-upcoming-features-on-the-hugging-face-model-hub) (Julien Chaumond, Hugging Face). Claim: Julien Chaumond says the Transformers open-source software has more than 5,000 companies using it in production. [5:24](https://www.youtube.com/watch?v=03uallDwq6o&t=324s)
- [Wikimedia MLOps](https://mlopstalks.com/talks/wikimedia-mlops) (Chris Albon, Wikimedia Foundation). Pushed back: Chris Albon argues that open source is not automatically accessible and that accessibility requires making projects understandable and reproducible. [1:00:50](https://www.youtube.com/watch?v=TKVR5RYkqnc&t=3650s)

1 more from 2021 on this thread: https://mlopstalks.com/threads/open-weights-against-the-api/2021

## 2022

8 sessions.

- [Investing in MLOps](https://mlopstalks.com/talks/investing-in-mlops) (Leigh Marie Braswell, Founders Fund & Davis Treybig, Innovation Endeavors). Claim: Davis Treybig says open-source companies should plan how they might eventually monetize even if they do not build paid tiers for several years. [33:44](https://www.youtube.com/watch?v=pjZss-fnOac&t=2024s)
- [Building the World's First Data Engineering Conference](https://mlopstalks.com/talks/building-the-worlds-first-data-engineering-conference) (Pete Soderling, Data Council and Data Community Fund). Claim: Open-source and commercial businesses have different routes to market and revenue, and founders should understand the difference between them. [25:11](https://www.youtube.com/watch?v=RyeoFSctI-M&t=1511s)
- [Modern Data Science with Vaex](https://mlopstalks.com/talks/modern-data-science-with-vaex) (Maarten Breddels, Vaex.io & Jovan Veljanoski, Tiqets). Claim: Vaex is an open-source out-of-core DataFrame library that lets users process very large datasets locally without clusters or complex configuration. [4:06](https://www.youtube.com/watch?v=p__sVdwz8v8&t=246s)
- [Making MLflow](https://mlopstalks.com/talks/making-mlflow) (Corey Zumar, Databricks). Claim: MLflow was built as an open-source platform for the end-to-end machine-learning lifecycle, using open APIs and supporting many tools. [1:18](https://www.youtube.com/watch?v=odEWCeYPZkU&t=78s)

4 more from 2022 on this thread: https://mlopstalks.com/threads/open-weights-against-the-api/2022

## 2023

44 sessions.

- [Solving the Last Mile Problem of Foundation Models with Data-Centric AI](https://mlopstalks.com/talks/solving-the-last-mile-problem-of-foundation-models-with-data-centric-ai) (Alex Ratner, Snorkel AI and University of Washington). Pushed back: Alex argues that the most durable advantage will come from private data, domain-specific knowledge, and last-mile development rather than from closed general-purpose API models. [7:54](https://www.youtube.com/watch?v=-oDgV6q6KtI&t=474s)
- [Want High Performing LLMs? Hint: It Is All About Your Data](https://mlopstalks.com/talks/want-high-performing-llms-hint-it-is-all-about-your-data) (Vikram Chatterji, Galileo). Pushed back: Vikram Chatterji rejects the idea that teams should exclusively prompt or exclusively fine-tune, arguing that the right balance depends on the use case. [32:53](https://www.youtube.com/watch?v=XpeC1dqfiNo&t=1973s)
- [Pitfalls and Best Practices: 5 Lessons from LLMs in Production](https://mlopstalks.com/talks/pitfalls-and-best-practices-5-lessons-from-llms-in-production) (Raza Habib, Humanloop). Pushed back: Raza argues against starting with complex chains or agents before testing stronger models, prompt engineering, and fine-tuning, while acknowledging that agents and chains work in some situations. [13:38](https://www.youtube.com/watch?v=TRAUNcPj8KU&t=818s)
- [Build and Customize LLMs in Less than 10 Lines of YAML](https://mlopstalks.com/talks/build-and-customize-llms-in-less-than-10-lines-of-yaml) (Travis Addair, Predibase). Pushed back: Travis argued that fine-tuning a smaller model can match or outperform a much larger model for a sufficiently bounded task at lower latency and cost. [17:17](https://www.youtube.com/watch?v=De6RY2GN-e4&t=1037s)

40 more from 2023 on this thread: https://mlopstalks.com/threads/open-weights-against-the-api/2023

## 2024

25 sessions.

- [Vision Pipelines in Production: Serving & Optimisations](https://mlopstalks.com/talks/vision-pipelines-in-production-serving-optimisations) (Biswaroop Bhattacharjee, Prem AI). Pushed back: Basic generation with more configuration was considered insufficient for the required consistency and control, so fine-tuning was chosen instead. [3:31](https://www.youtube.com/watch?v=POL7gSRFXqE&t=211s)
- [Fine Tuning Llamas](https://mlopstalks.com/talks/fine-tuning-llamas) (Kai Davenport). Pushed back: Kai Davenport rejects the idea that fine-tuning is always better than retrieval-augmented generation and argues that the two approaches can be combined. [4:57](https://www.youtube.com/watch?v=7F0tnquy6t0&t=297s)
- [Ghostwriter - AI Writing That Learns From You](https://mlopstalks.com/talks/ghostwriter-ai-writing-that-learns-from-you) (Jonny Dimond, Shortwave). Pushed back: The speaker disputes the assumption that autocomplete can be made reliable through system-prompt instructions alone and says fine-tuning was needed. [10:08](https://www.youtube.com/watch?v=9B-BiKH_xjg&t=608s)
- [Alignment is Real](https://mlopstalks.com/talks/alignment-is-real) (Shiva Bhattacharjee, TrueLaw Inc). Pushed back: Demetrios Brinkmann questioned whether DSPy was too much of a research project to trust in production, while Shiva Bhattacharjee defended using a modified, self-hosted version of it. [4:57](https://www.youtube.com/watch?v=O0F3RAWZNfM&t=297s)

21 more from 2024 on this thread: https://mlopstalks.com/threads/open-weights-against-the-api/2024

## 2025

18 sessions.

- [I Let An AI Play Pokémon! - Claude plays Pokémon Creator](https://mlopstalks.com/talks/i-let-an-ai-play-pokemon-claude-plays-pokemon-creator) (David Hershey, Anthropic). Pushed back: Fine-tuning is not usually the best first step; prompting should generally be pushed further before fine-tuning is considered. [13:31](https://www.youtube.com/watch?v=nRHeGJwVP18&t=811s)
- [We're All Finetuning Incorrectly](https://mlopstalks.com/talks/were-all-finetuning-incorrectly) (Tanmay Chopra, Emissary). Pushed back: The idea that prompting or prompt fine-tuning is inherently undesirable was softened; prompting is useful before a system reaches its ceiling. [14:30](https://www.youtube.com/watch?v=mulsjmhXbaQ&t=870s)
- [Building Out GPU Clouds](https://mlopstalks.com/talks/building-out-gpu-clouds) (Mohan Atreya, Rafay Systems). Pushed back: Demetrios Brinkmann noted that fine-tuning is often criticized as difficult and potentially harmful when done incorrectly, while Mohan Atreya argued that it can become a zero-code workflow. [45:57](https://www.youtube.com/watch?v=86FVq-tK3aE&t=2757s)
- [Office Hours on DuckDB, AWS Glue, and Iceberg](https://mlopstalks.com/talks/office-hours-on-duckdb-aws-glue-and-iceberg) (). Pushed back: Eric said the distinction between Delta Lake and Iceberg was not one he could fully explain from memory, while Hossam framed them as competing open-table-format approaches. [15:27](https://www.youtube.com/watch?v=b8NrMgPEX_g&t=927s)

14 more from 2025 on this thread: https://mlopstalks.com/threads/open-weights-against-the-api/2025

## 2026

2 sessions.

- [Open vs Closed Source Agent Infra?](https://mlopstalks.com/talks/open-vs-closed-source-agent-infra) (Adel El Hallak, NVIDIA). Pushed back: Ben disputed the assumption that open-source tools let teams fix any problem, saying highly abstract tools can quickly become difficult to understand and can create a painful dead end. [11:03](https://www.youtube.com/watch?v=TTAwep2uwto&t=663s)

1 more from 2026 on this thread: https://mlopstalks.com/threads/open-weights-against-the-api/2026