AI systems work best when their role matches the user's expertise, because experts need deterministic, accurate assistance while beginners can use AI as a coach.
2
Prompting can bootstrap an initial system, but owning the model lets a team optimize for its actual objective function instead of only predicting the next word.
3
Production AI systems need explicit evaluations, thresholds, fallback paths, and continuous improvement rather than a one-time model release.
Summary
Tanmay Chopra argues that many teams call extended pretraining or prompt changes fine-tuning, while missing the older machine learning practices that make systems improve for a specific task. Prompting is useful for reaching an initial version, especially when data is scarce, but it cannot change the model's objective. A specialized model can optimize for classifications, confidence thresholds, or whether generated content matches reality. Chopra also argues that LLMs should become components inside larger ML systems. A classifier can decide whether an LLM should answer, while deterministic retrieval or smaller local models can reduce uncertainty and latency. Teams should choose problems where they have a differentiated feedback loop and enough economic value to justify ongoing maintenance. The conversation ends with an infrastructure and language constraint: serious customization often requires a Python-based AI backend because much of the ML tooling is built around PyTorch, TensorFlow, and related libraries.
AI is useful in different ways for experts and beginners
Tanmay Chopra separates AI as a supervisor from AI as an assistant. A beginner can use code generation to get started, check the result, and learn when the system fails. An expert working in a familiar area may be slowed down because repeatability, determinism, accuracy, and latency matter more than conversational help. Chopra gives infrastructure engineering as an example: code generation can be valuable for an infrastructure engineer doing frontend work, while it can slow that same person down on a difficult infrastructure problem. The usefulness depends on the user's expertise and ability to verify the output.
General models create too much distance between a prompt and a usable workflow
Chopra says generalized systems are trained on broad knowledge, so they rarely produce the best workflow for a particular company. A system would need to see many examples pairing prompts with the ideal resulting pipelines to reduce the edits needed after its first output. He also describes a deeper mismatch with machine learning. Traditional ML takes inputs and outputs and searches for a function, while newer systems often ask users to explain their process in a prompt. That puts the burden of describing a workflow onto people who may not know the right terminology or process.
Teams need to define what good means before they can improve an AI system
Chopra says his company first asks customers to define success for their own system. That could mean having the right files, producing the right text, or matching a business outcome. He distinguishes an AI system being generally good from being good for a particular user or task. Evaluations based on gut feeling add another layer of uncertainty, yet deterministic evaluation remains uncommon. He says evaluation is difficult because the target can sit between a technical measure and a business metric. AI systems also differ from conventional software because they are perishable and need a plan for retraining and improvement from the start.
Prompting can bootstrap version zero, while model ownership changes the objective
Chopra does not reject prompting. He recommends using it until it stops being good enough, then deciding whether owning a model is justified. Prompt optimization cannot change the objective function, because the underlying model still cares about predicting the next word. Enterprise tasks may instead care about classification or whether an investment memo maps to reality. Owning the model allows optimization against that task-specific objective. He compares maintaining a specialized model to a lifelong commitment, since the team must retrain and care for it over time rather than release it once and walk away.
Specialized systems can use math to optimize a task instead of guessing through prompts
Chopra describes optimization as a mathematical way to reduce loss when a team can define what good looks like. He connects this to DeepSeek's GRPO approach and says the industry lost sight of older fine-tuning methods by focusing on generalized next-token prediction. Prompting guides still have a place because an LLM lets a team bootstrap an initial version with far less labeled data than older ML workflows required. That first version can then become a source of real user feedback. The long-term system should improve from that feedback rather than wait passively for the next foundation model.
LLMs should be components inside ML systems, with smaller models handling clear decisions
Chopra recommends treating an LLM as one node in a larger pipeline. In a chatbot, a classifier can decide whether a question is in scope before the LLM answers it. This lets the system reject uncertain requests or fall back to an older workflow. He says teams can convert an LLM into classifiers, regressors, and other traditional model types while retaining knowledge from the model's internal layers. With a confidence score and a threshold, an enterprise can measure error rates and adjust how often the system answers. This is safer than making an LLM answer with similar confidence when it knows the answer and when it is guessing.
System design can replace uncertain generation with deterministic retrieval and infill
For localized advertising, Chopra proposes retrieving a suitable human model from a database and then using inpainting rather than generating an entirely new person. The country lookup can be deterministic, while the image edit handles the flexible part. He describes this as turning a generation problem into retrieval plus infill, which reduces uncertainty and makes separate parts of the pipeline easier to optimize. The same design principle applies more broadly: use as little AI as possible, assign clear tasks to each component, and avoid adding uncertainty through generated orchestration when an expert can specify the workflow directly.
Serious customization often requires a separate Python-based AI backend
Chopra says teams can start by calling foundation-model APIs from any language, but they may hit a ceiling when they need local models, custom evaluation, model management, or lower latency. Embedding a query locally is one example where a small model can avoid a network call. The difficulty is that much of the ML ecosystem is built in Python, including PyTorch, TensorFlow, and transformer libraries. His recommendation for teams committed to AI is to consider a dedicated AI backend that owns evaluation, API calls, model management, and gateway management. An MVP can remain in an existing language if the team accepts that a later move may be required.
