Tuhin Srivastava thinks machine learning will become a standard part of every software engineer's toolkit, while the data scientist role shrinks from its earlier form.
2
General-purpose models can cover many low-value use cases, but specialized products will need smaller or adapted models for better speed, cost, and performance.
3
Deploying large models still creates difficult infrastructure problems around latency, cost, scale to zero, cold starts, and model versioning.
Summary
Tuhin Srivastava describes his move from investment banking to machine learning research and then to building production systems. He says that training a model in a notebook and showing a confusion matrix was never enough, so he learned software engineering to put models into products. His view of the field has changed: data scientists once carried most of the machine learning work, but software engineers are now adopting models directly and treating machine learning as part of product development. Tuhin expects general-purpose models to handle many low-value use cases. More valuable, specialized applications will need fine-tuning, smaller models, or other adaptations to meet their cost, speed, and performance needs. He also explains why large models create difficult serving problems, including cold starts, autoscaling, latency, and scale to zero. Baseten's approach is to expose models, logs, and serverless functions through APIs while keeping production versions separate from other versions.
Tuhin learned machine learning by moving from finance into applied research
Tuhin says investment banking taught him to work hard under unpleasant conditions, but he eventually left after two years and moved to Boston for biomedical research. Around 2011 and 2012, he used machine learning to predict the prognosis of neuromuscular disease so that intervention could happen earlier. The work involved support vector machines, with gradient-boosted trees becoming more popular at the time. His signal-processing background gave him probability and information theory foundations, and he spent his final months in banking relearning statistics on weekends.
Production work pushed Tuhin from machine learning into engineering
Tuhin worked on fraud detection, content moderation, and lead scoring, which he describes as common applied machine learning use cases in the early 2010s. Companies often hired a few machine learning people and hoped something useful would emerge, but he did not want to stop at model training. He learned how to build the surrounding product because a notebook and a confusion matrix were not the goal. At Gumroad, limited engineering support meant that he learned server-side and front-end engineering, while a colleague with a mathematics PhD also became an engineer.
Software engineers are taking on machine learning as a normal product skill
Tuhin says the field has shifted from data scientists learning engineering to software engineers learning machine learning. He thinks data scientists emerged because companies did not yet know how to turn machine learning into value, so analysts were asked to take on a research-oriented role. In his view, engineers have more ability to productize models without waiting for another team. Tuhin expects machine learning to become a large part of every engineer's toolkit, even when the engineer does not understand every internal layer of a model.
Engineers can create useful products without becoming model researchers
Tuhin uses Refusion as an example of engineers applying machine learning through product and domain knowledge. The project builds on Stable Diffusion and generates music from prompts, while its creators brought substantial experience in music. They fine-tuned models and used Weights & Biases, but Tuhin says they did not need a deep conceptual understanding of every internal detail to create value. He describes the field as moving from exploration toward exploitation, where higher-level abstractions let engineers treat models as black boxes and focus on what the product should do.
General models will cover broad use cases while specialized products go deeper
Tuhin agrees that off-the-shelf GPT-4-style APIs will cover much of the low-value, commoditized long tail. Applications with concentrated value, such as medical imaging or legal work, will need models adapted to their context. That adaptation might involve using a different model, fine-tuning, or choosing a smaller and more targeted system. He frames the decision around the task, then asks what matters for that task, including response time, cost, and performance. His restaurant analogy is that a specialist can usually make one type of food better than a restaurant covering every cuisine.
Large models create unsolved serving and scaling problems
Abi asks about the compute and storage costs of deploying increasingly large models. Tuhin says Baseten is working on serving a model with around 60 billion parameters, and that larger models create problems with scaling, scale to zero, cost, latency, and cold starts. A model that is needed for only a few hours each day may not be worth running continuously, but bringing it back online quickly is also difficult. He says OpenAI made its models usable through serving infrastructure, fine-tuning support, and documentation, while the broader industry still has work to do before engineers get the same simple experience.
Model documentation and version tracking remain open problems
Abi asks about Baseten's approach to documentation as code. Tuhin describes a workflow where engineers contribute living documentation as code, with the documentation connected to the development process. He says machine learning makes this especially difficult because models and other artifacts can disappear from a team's shared understanding. Teams may not know which model was trained when, which version matters, or which artifact is running in production. Tools such as Weights & Biases help with training records, but Tuhin says many teams still cannot identify the exact production artifact.
Baseten uses APIs and separate production versions to manage model services
Tuhin says Baseten treats models, model versions, logs, and serverless functions as API-based components. A team can add a serverless caching layer around a large model and scale that layer separately from the model itself. He acknowledges that microservices add operational complexity, so Baseten is working toward a more Git-based workflow. The intended model is similar to a web application: one primary production version receives live traffic, while other versions remain available for testing or explicit calls. Tuhin says this makes the problem closer to familiar software engineering workflows involving CI/CD, staging, and production.
"My take is actually what we're going to see now is that the data people who became machine learning people, I think that's actually going to shrink back, they're just going to go back to being data people and machine learning is going to become just like a massive massive part of every engineer's toolkit."Tuhin Srivastava18:51
Who should watch
You build machine learning products and still treat deployment as the final step after notebook work.
Your team is deciding whether a general-purpose model is enough or whether a specialized, smaller model is worth the extra engineering.
You are dealing with model serving costs, cold starts, version tracking, or documentation that does not fit your existing software workflow.
