# Fireside Chat - The Future of LLMs

David Hershey, Unusual Ventures & Daniel Jeffries, AI Infrastructure Alliance | LLMs in Production 2023 | 36:07

Source: https://www.youtube.com/watch?v=vFZB9scWdWA
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/fireside-chat-the-future-of-llms
Published: 2023-08-17
Tags: evals, human-in-the-loop, prompt-engineering, testing

## TL;DR
- LLM applications should solve specific problems well instead of trying to replace entire software systems.
- Language models can help people enter software and operations work, but production knowledge and judgment still matter.
- Testing LLM applications requires new methods because teams need to assess whether outputs are useful, safe, accurate, and aligned with the task.

## Summary
David Hershey and Daniel Jeffries discuss where language models are useful today and where expectations have run ahead of reality. They argue for focused applications and workflows instead of general systems that promise to solve every problem. Jeffries describes a newsletter workflow that breaks the job into retrieval, story selection, drafting, and human editing. Hershey says ambitious projects are still possible when the problem is constrained, such as using a language model to play Minecraft. They also discuss the limits of AI coding assistants. These tools can help less experienced developers explore Kubernetes and can automate parts of a workflow, but they do not replace knowledge of production systems, security, or software design. The final section focuses on evaluation. Traditional monitoring assumes that a system works and checks whether it keeps working. LLM teams often do not know whether the system works at all, so they need tests for qualities such as helpfulness, accuracy, toxicity, and logical consistency.

## Key ideas
### LLM products should focus on a narrow job
[01:15](https://www.youtube.com/watch?v=vFZB9scWdWA&t=75s)
David Hershey says the frameworks around language models may become commodities, with strong price competition. What matters to him is the application that lets someone press a button and get a useful result. Daniel Jeffries agrees that teams are distracted by the number of possible uses and try to solve everything at once. He compares the better approach to the Unix philosophy: build a small tool that does one thing well. This could produce software between large enterprise products and consumer applications, including workflows that were previously too expensive or awkward to build.

### Language models need layers around them before they can support reliable applications
[05:02](https://www.youtube.com/watch?v=vFZB9scWdWA&t=302s)
Jeffries rejects the idea that a language model is already the operating system for all future applications. He compares the current situation to moving up a software stack, from lower-level systems toward abstractions such as WordPress and Divi. He expects middleware, prompting practices, and other techniques to make models more useful. Researchers are working on how models reason, how they expose their world models and beliefs, and how questions can be phrased to reduce hallucinations. The speakers say these practices are still emerging.

### Breaking a workflow into designed steps works better than asking for everything at once
[07:47](https://www.youtube.com/watch?v=vFZB9scWdWA&t=467s)
Jeffries describes a newsletter writer built at the AI Infrastructure Alliance. The system retrieves recent papers and places them in a Telegram channel. Bots select six stories and run a tournament, then another prompt produces several summaries in a particular writing style. A human reviews the results, rejects weak wording, moves lines, and removes repeated source material. Jeffries says the team designed the reasoning behind the workflow instead of asking the model to invent the whole newsletter process. They compare this with AutoGPT becoming stuck in an unrelated loop after being asked to make a birthday cake.

### Ambitious AI projects work when the problem has clear boundaries
[11:57](https://www.youtube.com/watch?v=vFZB9scWdWA&t=717s)
Hershey says teams can still pursue difficult goals, as long as they define a constrained problem rather than asking one system to solve the whole universe of tasks. He points to the Voyager paper, where GPT-4 was used as a knowledge engine to play Minecraft. Playing a complicated game is a general task within a defined environment, which makes it a useful target. The speakers say language models bring reasoning and a partial model of the world to software, but applications still need a concrete, valuable problem to guide their design.

### The infrastructure is moving more slowly than the research
[14:09](https://www.youtube.com/watch?v=vFZB9scWdWA&t=849s)
Jeffries says language-model research feels extremely fast while the infrastructure remains immature. He gives the example of trying to run Falcon 40B, finding limited model availability through AWS, and considering services such as RunPod and DataCrunch. He also asks why no one has simply taken the leading open-source models on Hugging Face, parallelized requests across them, and sold access by the token. The gap between an idea and a working product includes infrastructure, tools, APIs, and the practical details of building with models.

### AI assistants help developers, but production skills still have to be learned
[18:32](https://www.youtube.com/watch?v=vFZB9scWdWA&t=1112s)
Hershey separates writing code from operating production systems. He says becoming a strong software engineer takes time because it includes learning how systems behave in production, while data scientists also need probability, experimentation, machine learning, and data knowledge. ChatGPT can give someone a useful starting point for a Kubernetes question, but that does not remove the need to understand production systems. Jeffries describes a similar future for developers: assistants can handle boilerplate and act like tutors for junior programmers, while experienced engineers still need to reason about scaling, security, novel attacks, and unusual failures.

### Human review remains necessary because model mistakes require subject knowledge
[22:12](https://www.youtube.com/watch?v=vFZB9scWdWA&t=1332s)
Jeffries gives writing as an example of collaborative use. A model improved a blog draft after he prompted it to vary paragraph length, use contractions, and avoid overly formal phrasing. It still failed to make the final point land, and he rewrote that part using judgment built from decades of writing. The same issue appears in code. Models can invent packages, use realistic but nonexistent API calls, or make errors that differ from ordinary human mistakes. Developers need enough knowledge to notice those failures.

### LLM evaluation asks whether the system works at all
[28:47](https://www.youtube.com/watch?v=vFZB9scWdWA&t=1727s)
Hershey says traditional observability assumes that a system works and checks whether it continues to work. With language-model applications, teams often do not know whether the system works in the first place. Evaluation may involve user behavior, benchmarks, thumbs-up and thumbs-down signals, or tests written before release. Jeffries gives examples such as checking whether a model follows the requested logic, detects prompt injection, and avoids factual or mathematical errors. They also discuss testing qualities such as toxicity, helpfulness, friendliness, intelligence, and accuracy. Prompt tests with collections of example uses are one early approach.

## Notable quotes
- David Hershey: "I really want the application, you know." (01:32)
- Daniel Jeffries: "People need to lean into the old Unix philosophy of do one thing and one thing well." (02:41)
- David Hershey: "The fundamentals of learning how to become and think about production systems you still have to learn that." (19:46)
- David Hershey: "People are shipping things to production language models and just don't know if they work period." (29:25)
- David Hershey: "We are evaluating on just a completely different set of dimensions than we really have before." (31:50)

## Tools & references mentioned
- Demetrios Brinkmann
- Daniel Jeffries
- David Hershey
- Tecton
- Determined AI
- AI Infrastructure Alliance
- AutoGPT
- GPT-4
- Voyager
- Minecraft
- Falcon 40B
- AWS
- RunPod
- DataCrunch
- Hugging Face
- Kubernetes
- ChatGPT
- GitHub Copilot
- Divi
- Wolfram Alpha
- New Relic
- Amplitude
- Mixpanel
- Maxine
- Apache Airflow
- Promptomized

## Who should watch
- You are deciding whether to build a broad AI platform or a focused application and want a practical argument for narrowing the problem.
- You are a data scientist or developer wondering whether language models remove the need to learn production systems, software fundamentals, or security.
- Your team has an LLM application in production but lacks a clear way to test whether its outputs are useful, safe, and correct.

## Related talks

- [From Building Self-driving Cars to Building LLM Applications](https://mlopstalks.com/talks/from-building-self-driving-cars-to-building-llm-applications) (Effy Zhang, Baserun, 10:45)
- [Evaluating LLM-based Applications](https://mlopstalks.com/talks/evaluating-llm-based-applications) (Josh Tobin, Gantry, 49:50)
- [Real LLM Success Stories: How They Actually Work](https://mlopstalks.com/talks/real-llm-success-stories-how-they-actually-work) (Alex Strick van Linschoten, ZenML, 48:20)
- [Systematically Test and Evaluate Your LLMs Apps](https://mlopstalks.com/talks/systematically-test-and-evaluate-your-llms-apps) (Gideon Mendels, Comet, 1:01:43)
- [LLMs For the Rest of Us](https://mlopstalks.com/talks/llms-for-the-rest-of-us) (Vikram Sreekanti, Aqueduct & Joseph Gonzalez, UC Berkeley and Aqueduct, 24:33)