Prompting and retrieval-augmented generation are useful ways to work with pretrained language models, but fine-tuning adapts a model to a custom task.
2
LoRA factorizes weight updates into smaller matrices, which can reduce the resources needed to fine-tune a large model.
3
Full fine-tuning can be a useful baseline, while updating only selected layers or using LoRA can give a better fit for limited budgets and hardware.
Summary
Sebastian Raschka explains how pretrained language models can be used before discussing fine-tuning. Prompting changes the model's input, while retrieval-augmented generation adds information from a document collection. Fine-tuning changes the model for a target task, including classification and instruction following. He compares feature extraction, updating output layers, and updating the full Transformer. Full updates can improve performance, but they use more time, memory, and compute. Raschka then explains low-rank adaptation, or LoRA, which represents a weight update with two smaller matrices. In his examples, LoRA reduced a 7-billion-parameter training run from about nine hours on six GPUs to about one hour on one GPU. He also demonstrates an open-source Lightning AI repository for downloading models, preparing datasets, and running LoRA or full fine-tuning. His advice is practical: use full fine-tuning when resources allow, then test whether fewer layers or LoRA provide enough performance for the task.
Pretrained models support several workflows before fine-tuning
Raschka starts with prompting, where the wording and format of the input can change the result, especially for smaller open-source models. He then describes retrieval-augmented generation as a hybrid system. A company can split documentation into chunks, create embeddings, store them in a vector database, and retrieve similar passages for a user query. The language model uses that retrieved material when generating its answer. This is useful when the needed information already exists in company documentation and should not be regenerated from memory.
Classification is a practical reason to fine-tune a language model
Raschka argues that many real applications involve classification, including spam detection, fake-news detection, and toxic-content detection. One option is to extract embeddings from a frozen Transformer and train a separate classifier such as logistic regression or a support vector machine. A second option adds and fine-tunes output layers while keeping the Transformer frozen. A third option updates the entire language model. In his movie-review example with 50,000 examples, the small model reached about 90% accuracy after three minutes when only some parts were updated, and 93% when the whole model was updated.
Instruction fine-tuning teaches a model to produce task-specific responses
Instruction fine-tuning uses examples containing an instruction, an optional input, and the desired output. Raschka gives examples such as asking for a limerick about a pelican or identifying the odd item in a group. This differs from pretraining, where the model predicts the next word in unlabelled text. He compares a GPT-3 base model, the same model with prompting, and a model with supervised fine-tuning. The supervised version performs better in the comparison, which is why many high-performing models on leaderboards use supervised fine-tuning.
Low-rank adaptation, or LoRA, represents the weight update as two smaller matrices rather than storing a full update matrix. Raschka gives a 5,000 by 1,000 matrix as an example. The full matrix has 5 million parameters, while factorizing it into matrices sized 5,000 by 10 and 10 by 1,000 uses 60,000 parameters. LoRA can approach full fine-tuning performance for particular tasks, although Raschka does not present it as a general guarantee. The rank and the layers selected for updating are important hyperparameters.
The resource difference between full fine-tuning and LoRA is large
In Raschka's speed test, full fine-tuning of a 7-billion-parameter model took about nine hours on six GPUs. He needed CPU offloading because the full model did not fit across those GPUs. LoRA fit on one GPU and took about one hour for 50,000 training examples. Full fine-tuning used the available GPU memory, while LoRA used about 16 GB. Quantized LoRA reduced memory use further, to roughly 12 or 13 GB, although the extra quantization step added computation time.
An open-source training repository keeps the process inspectable
Raschka describes an open-source Lightning AI repository that provides scripts for fine-tuning models such as Mistral and Llama 2. He prefers it because users can inspect and modify the scripts instead of working through a fully hidden interface. The basic workflow is to clone the repository, install its requirements, download a model, convert its checkpoint, prepare a dataset, and run a training command. The repository supports built-in datasets and custom CSV files. Dataset preparation must match the model's tokenizer, since different models can tokenize the same text differently.
A small number of Transformer blocks may be enough
When asked about full fine-tuning, Raschka describes experiments that started with the output layer and added Transformer blocks one at a time. In the experiment he discusses, performance had already saturated after updating roughly two or three Transformer blocks, even though the model had more blocks available. He recommends starting with the output layer and gradually adding layers to see whether the extra updates are useful. The result depends on the dataset, so this is a testing strategy rather than a fixed rule.
The fine-tuning choice depends on budget and the target task
Raschka calls full fine-tuning a useful baseline when enough resources are available, but the cost can become substantial. He estimates that eight expensive GPUs can cost around 30 or 40 dollars per hour, so a ten-hour experiment can cost about 400 dollars. LoRA can provide a good approximation at lower cost, and he suggests that a team might prefer LoRA on a model ten times larger rather than full fine-tuning on a smaller model. He also says LoRA can reduce performance on tasks that were not part of the fine-tuning target, which may be acceptable when the application has a narrow goal.
