Current fine-tuning systems usually extend pretraining because they keep next-token loss and the original output head.
2
Production fine-tuning should use a loss function that matches the business objective and an output head limited to valid task outputs.
3
Fine-tuning is useful after a task stabilizes, enough data exists, and the team can define what good performance means.
Summary
Tanmay Chopra argues that most current fine-tuning is mislabeled extended pretraining. These systems keep next-token loss and the full language-model output head, so they mainly teach a model to produce a familiar style. A production model should instead optimize for the actual task, such as making a routing decision, and restrict its outputs to the valid classes. He explains this with a classifier example where classification loss is more meaningful than penalizing every incorrect character in the generated answer. A smaller specialized model can then improve consistency, reduce latency, and cost less. Chopra also gives conditions for using fine-tuning: the task should be stable, the team should have roughly 500 decent samples to start, and there must be a clear definition of good performance. He says prompting remains useful when data is scarce, while fine-tuning can gradually replace the prompt as more examples become available.
Fine-tuning should improve control over model behavior
Chopra frames prompting, retrieval, and fine-tuning as successive ways to steer a model. Prompting changes the input, retrieval adds external knowledge to that input, and fine-tuning changes the model weights. These approaches can be combined. A team can retrieve context and use a fine-tuned model at the same time. He describes prompting and retrieval as input-side changes, while fine-tuning lets backpropagation adjust the weights from input and expected-output pairs. This matters when a team cannot describe every rule that connects an input to the desired output.
Chopra says current fine-tuning endpoints usually keep the next-token loss used during pretraining and keep the output head unchanged. He argues that this is better described as more pretraining than true task-specific fine-tuning. Because the model is still rewarded for predicting the next word, the method is naturally associated with learning style. That limitation comes from the objective being optimized. Earlier machine learning systems changed the loss to match the task and adjusted the output head to match the expected outputs. Chopra says those two choices are still available for language models.
The loss function should match the business decision
For an enterprise task, Chopra says next-token prediction can waste learning capacity because the business may care about a decision rather than the exact wording of an answer. If a model reads an essay to make a decision, the training signal should indicate whether the essay supported the right decision. It does not need to penalize every word in the essay. The same principle applies to classification and routing. A meaningful task loss can update the model based on whether it selected the correct class, rather than treating an answer that differs by one character as almost entirely correct.
A task-specific output head can enforce valid answers
Chopra says language models normally produce probabilities over every token in their vocabulary, even when an application only needs a few classes or routes. For a router, a prompt that asks for the most likely class can produce arbitrary text and can score an answer based on every character. A specialized model can replace the output head with one containing only the acceptable classes. The model can then use classification loss, and its output is limited to the routes the application can actually use. In the discussion, Chopra says his platform replaces the language-model head from the options found in the uploaded dataset rather than masking tokens.
Constrained outputs can improve accuracy and latency
Chopra gives an insurance technology example where 300 samples were used to fine-tune a classifier, with performance improving by about three times after the output universe was constrained. He also describes a separate company that specialized a model for classification and reduced classification time to about one-third of its original duration. The smaller model used more samples and maintained accuracy. He attributes the speed gain to changing the output head so the model can produce the required classification in a single forward pass instead of repeatedly regressing over the full token vocabulary.
Fine-tuning works best after the task has stabilized
Chopra says fine-tuning reduces flexibility compared with prompting or retrieval, so teams should wait until the task is stable. A task that changes every day is a poor candidate because the model will quickly encode assumptions that are being revised. Changes every three or six months can still be manageable. He also says the team needs data and a definition of good performance. His starting point is at least 500 reasonably good samples, although the labels do not need to be perfect. The team does not need to use cross-entropy as its evaluation measure, but it must be able to say what a good result looks like.
Prompting bridges the gap before enough training data exists
In the question period, Chopra describes prompting as a way to compensate for limited data. A good prompt primes a large model to use knowledge it already has, which helps with the cold-start problem. He suggests writing a strong prompt when a team has around 500 samples and fine-tuning with that prompt. As the dataset grows to around 5,000 or 10,000 samples, the team may be able to remove the prompt and let the model learn the task from the data. Prompting and fine-tuning therefore form a progression rather than mutually exclusive choices.
The practical cost and effort can be lower than teams expect
Chopra says fine-tuning is not necessarily expensive, slow, or technically difficult. He reports that the examples he has seen beat the teams' existing metrics for under $500, with the highest examples reaching about $1,000. He says successful jobs generally require less than two days of compute, and a job with 1,000 to 2,000 samples, or sometimes 5,000 samples, on a one- or seven-billion-parameter model can take less than a day. Once the data is prepared, he says the first fine-tune can take less than 15 minutes and does not require knowledge of multivariable calculus.
