# Everything We've Been Taught About ML is Wrong

Emmanuel Ameisen, Anthropic | LLMs in Production 2023 | 27:08

Source: https://www.youtube.com/watch?v=nnAY6tOPxh8
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/everything-weve-been-taught-about-ml-is-wrong
Published: 2023-08-11
Tags: evals, monitoring, prompt-engineering, synthetic-data

## TL;DR
- Emmanuel Ameisen argues that large language models have changed several standard ML rules because they reach useful results faster through general capabilities and prompting.
- Models can now generate training data and evaluate outputs in some situations, although humans are still needed for difficult or specialized judgments.
- ML engineering fundamentals remain: solve a useful problem, use good data, account for model errors, build around engineering constraints, and monitor systems after deployment.

## Summary
Emmanuel Ameisen revisits the ML rules he used to teach, including starting with a simple model, using task-specific data, avoiding model-generated data, and keeping models within their trained purpose. He argues that large language models have changed the practical meaning of simplicity. A complex general model can produce a useful baseline in minutes, faster than building a simple task-specific model. Training on broad data can also improve performance on specialized tasks, and models can sometimes generate training data or judge outputs. Ameisen is careful about the limits. Human review remains useful, especially for specialized domains, and model outputs still contain hallucinations, bias, and ordinary mistakes. He closes with advice that has not changed: work on a useful problem, improve the data and prompts, treat models as fallible, apply strong engineering practices, handle version changes, and monitor deployed systems because failures can remain hidden for months.

## Key ideas
### A complex model can now be the fastest way to get a useful baseline
[00:44](https://www.youtube.com/watch?v=nnAY6tOPxh8&t=44s)
Ameisen says he used to teach people to begin with the simplest possible model, such as logistic regression for tabular classification or bag-of-words for NLP. That advice was tied to getting an initial result quickly. The relationship has changed with language models. He now uses an LLM for tasks as simple as writing a regular expression, wrapping list elements in parentheses, or reformatting text. A logistic regression might have about a thousand parameters, BERT about 110 million, and common LLMs more than 100 billion, yet the larger model can produce a useful result sooner. His revised rule is to start with a competent model, except where the task cannot easily be given to an LLM or is unrelated to NLP.

### Broad training data can improve performance on a specialized task
[06:54](https://www.youtube.com/watch?v=nnAY6tOPxh8&t=414s)
Ameisen uses realistic Yelp review generation to explain why task-specific data is no longer always the right starting point. Training only on Yelp reviews gives a model limited linguistic and reasoning ability, while training on the whole internet can provide capabilities that transfer to the task. He connects this to transfer learning with ImageNet, where a model trained on a large dataset could be fine-tuned on a smaller, specific dataset. LLMs extend this idea through zero-shot and in-context learning. A large model can perform well without fine-tuning, although Ameisen says fine-tuning can still help. His qualification is to use all the good data that the model can actually use, rather than indiscriminately adding bad data.

### Model-generated data has become useful in cases where it once seemed absurd
[09:50](https://www.youtube.com/watch?v=nnAY6tOPxh8&t=590s)
Ameisen describes model-generated data as an approach that historically worked poorly and often required substantial effort. The old objection was that a model generating its own training data added no information and could lead to mode collapse. He then points to Constitutional AI as an example of a model reading a constitution, rating answers against it, and generating preferences used to train the model. The resulting model improves. This changes the old default that humans must create all training data. Ameisen does not claim that generated data works for every application. His point is narrower: the fact that models can generate useful training data at all is a major change in ML practice.

### Models can sometimes judge outputs better than human raters
[12:04](https://www.youtube.com/watch?v=nnAY6tOPxh8&t=724s)
Generative outputs are difficult to score with standard metrics because a correct paraphrase can receive a poor score when it differs from a reference answer. Ameisen says humans have therefore been the usual judge for tasks such as summarization and translation. He describes research in which GPT-4 ratings agreed more closely with the golden ratings than crowd-worker ratings did. He also cites an estimate that 33 to 46 percent of Mechanical Turk workers used LLMs while completing a task, which complicates the assumption that human ratings are always purely human. Ameisen still limits the claim to some cases. Models can evaluate specialized tasks through prompting and examples, but human experts remain relevant when context and domain knowledge are difficult.

### General models have capabilities that are not visible from their training objective
[14:47](https://www.youtube.com/watch?v=nnAY6tOPxh8&t=887s)
Older ML models were narrow tools. A fraud classifier was built for fraud classification, and nobody expected it to write poems or answer questions about unrelated subjects. Large language models can write and read emails, do math, write code, simulate code execution, summarize documents, generate training data, and evaluate model outputs. Ameisen calls these additional abilities model overhangs. Some capabilities are unexpected, while others remain undiscovered because there is no complete way to list everything a large model can do. That creates both useful applications and safety risks. A general model can be taught a purpose through prompting, so deployment teams need to consider uses beyond the task that first motivated the model.

### The old rules about model limitations still apply
[17:43](https://www.youtube.com/watch?v=nnAY6tOPxh8&t=1063s)
Ameisen says LLMs still hallucinate, make mistakes, and show bias. Product teams should assume that a model will sometimes be wrong and build checks around that assumption. For advice-related applications, he recommends verifying that an output is true before returning it to a user. He also rejects treating models as magic. Better data and better prompts may create more value than adding model complexity, especially when a prompt contains useful zero-shot examples. The practical work remains grounded in the quality of the problem definition, the data, and the surrounding product rather than in the model alone.

### Deployment still requires engineering, version handling, and monitoring
[19:46](https://www.youtube.com/watch?v=nnAY6tOPxh8&t=1186s)
The engineering burden has not disappeared because models have become more capable. Ameisen describes work around prompt construction, post-processing, latency spikes, and deploying a prompting server rather than only a model artifact. He says changing LLM versions can create breaking changes that teams will need to handle carefully. Monitoring remains necessary because ML systems can fail silently, with problems discovered three or six months later. His closing advice is to spend time solving a useful problem instead of merely experimenting with new tools, while keeping the operational discipline that has always been needed for deployed ML systems.

## Notable quotes
- "So start with a competent model, I think is really the new version of this." (06:20)
- "If you have a model that's large enough you should just train on all the data because it'll make you better at all the things." (08:08)
- "Use models to train your models, use models to evaluate your models, just models all the way down." (14:35)
- "Models aren't magic, yet." (18:33)
- "Monitoring always need to do it." (20:39)

## Tools & references mentioned
- Anthropic
- Stripe
- Building ML Powered Applications
- logistic regression
- BERT
- Yelp
- GPT-3
- ImageNet
- Constitutional AI
- Machiavelli paper
- GPT-4
- Mechanical Turk

## Who should watch
- You are deciding whether an LLM should replace a small task-specific baseline in an existing ML workflow.
- You need to evaluate generated text, create training data, or use a general model on a specialized task.
- You deploy LLM-based products and want a concise reminder about hallucinations, prompt work, version changes, latency, and monitoring.

## Related talks

- [Enterprises Using MLOps, the Changing LLM Landscape, MLOps Pipelines](https://mlopstalks.com/talks/enterprises-using-mlops-the-changing-llm-landscape-mlops-pipelines) (Chris Van Pelt, Weights & Biases, 47:51)
- [We're All Finetuning Incorrectly](https://mlopstalks.com/talks/were-all-finetuning-incorrectly) (Tanmay Chopra, Emissary, 1:00:31)
- [The Next Revolution in AI: LLMs and Beyond](https://mlopstalks.com/talks/the-next-revolution-in-ai-llms-and-beyond) (, 13:47)
- [Incorporating LLMs in High-stake Use Cases](https://mlopstalks.com/talks/incorporating-llms-in-high-stake-use-cases) (Yada Pruksachatkun, Moonhub, 11:01)
- [GenAI: An Unreliable Information Store](https://mlopstalks.com/talks/genai-an-unreliable-information-store) (Noble Ackerson, Venta, 27:55)