Podcast

Reinvent Yourself and Be Curious

Stefano Bosisio, SynthesiaEpisode 264 · 57:16 · Sept 2024 · 272 viewsHosted by Demetrios Brinkmann
Thumbnail for Reinvent Yourself and Be Curious Watch on YouTube
TL;DR
  1. 1

    Stefano Bosisio moved from biomedical engineering and computational chemistry into data science, then into ML platform engineering.

  2. 2

    His first major ML platform had model tracking, monitoring, data validation, and automated training, but people did not use it because he had not explained or supported it well enough.

  3. 3

    A useful MLOps platform depends on business needs, communication, cost control, and careful choices about what to build internally or buy.

Summary

Stefano Bosisio describes a career that moved from biomedical engineering to a chemistry PhD, then data science in banking, and finally ML platform engineering. His transition began when notebook-based model development and a difficult handoff to operations made it hard to deliver business value. He learned AWS and built an ML platform from basic services, including Lambda functions, Step Functions, Glue, databases, training, tracking, monitoring, and data validation. The platform was technically strong, but adoption was poor. Stefano attributes that failure to weak internal communication, limited demonstrations, incomplete documentation, and a failure to understand the needs of data scientists, analysts, product managers, and other users. He also discusses build-versus-buy decisions, cloud costs, autoscaling failures, and the different demands of tabular, computer vision, and large language model workloads. His course teaches the MLOps mindset through a practical platform built with Kubeflow and Google Cloud.

Key ideas
19:00

A research career can build technical depth while leaving communication gaps

Stefano began with biomedical engineering, then pursued a PhD in computational chemistry because he wanted to do research and discover new things. During the PhD, he moved from almost no coding experience into Python, C, C++, multiprocessing, and statistics. He says this built many skills, but academic work can also make people assume that everyone understands their specialist language. He had fewer soft skills and less experience interacting with people outside his research setting. Moving into industry exposed those gaps quickly. He encourages people who enjoy research and continued study to consider a PhD, while also developing communication and other soft skills alongside the technical work.

09:19

The move from data science to MLOps started with a painful production handoff

After his PhD, Stefano joined a fintech company as a data scientist while the company was forming a new data science team. He found the transition from academia difficult because colleagues used financial terminology and the work ran on tightly controlled Windows laptops. Models were developed in Python notebooks, then handed to a DevOps or data operations team that struggled to run the model binaries in real time. His manager asked him to develop an ML platform on AWS. Stefano spent about three months learning the platform and breaking the problem into data processing, model training, and model execution. That work connected his research engineering background with the practical need to deliver business value.

17:20

A technically polished platform can fail when users do not understand it

Stefano built what he describes as the company's first ML platform, with automated model training, experiment and model registration, monitoring, and data validation. He expected the data science team to adopt it, but people were not using it. He spent weeks questioning why an apparently strong product had no users. His conclusion was that he had not communicated its value or invested enough time in adoption. Meetings and standups were insufficient. He needed to give demonstrations, record tutorials, write documentation, offer help, and show how users could ship a training pipeline or deploy a model. The lesson extended beyond data scientists to product managers, user experience teams, and data analysts.

23:15

Platform engineers need to adapt their communication to each stakeholder

Stefano says engineers can quickly build an experiment tracker, a large cluster, or an autoscaling system, but often struggle to explain the business value. He advises putting oneself in another person's position and paying attention to emotional intelligence. Different colleagues may need different explanations, so platform engineers must adjust how they communicate. He compares this work to building a Ferrari: data scientists are the drivers, while MLOps engineers are the mechanics who help the car perform. Without internal visibility and support, the platform becomes an expensive infrastructure project that nobody uses.

28:08

Build-versus-buy depends on speed, team capability, and cost

Stefano has seen external providers present expensive infrastructure diagrams for systems that a capable internal engineering team could build. He says buying can make sense for a startup that needs to move quickly and cannot afford an engineering team. When the team exists, leaders should understand what it can build before committing to an external product. He also supports FinOps and reviewing existing tools for lower-cost alternatives. The right decision depends on the business, the required delivery speed, and the team's abilities. He is especially wary of paying for infrastructure or services without understanding whether they solve the actual problem.

30:30

Some platform patterns generalize, while data and deployment still vary

For tabular data and computer vision, Stefano starts by separating the problem into smaller parts. Data platforms, metadata, querying, validation, and training pipelines can often follow similar patterns even when the raw data changes from tables to videos. A video system still needs to locate the video and query its metadata, but video-specific processing depends on the business case, such as object detection or another form of analysis. Training pipelines also share a broad structure: retrieve data, process it, train a model, and produce the model artifact. Deployment is less standardized because teams may use Vertex AI, their own Kubernetes infrastructure, or specialized machines for computer vision.

38:05

LLM platforms add infrastructure, availability, and cost decisions

Stefano partly agrees with the idea of an ML platform as an ecosystem that can support traditional ML, computer vision, and LLM workloads. Teams can call external model APIs or use models from Hugging Face through a simple service. That approach creates questions about provider downtime, service-level commitments, request volume, and cost. At Synthesia, serving large numbers of users makes external GPU costs a serious consideration, so the team must think about spot instances and sharing GPU resources across processes. He argues that companies should connect these choices to business value. Moving from no spend to a large monthly bill can happen quickly when teams add LLM workloads without understanding the cost.

42:49

Autoscaling mistakes can become expensive production lessons

Stefano agrees that engineers who understand autoscaling can be valuable because incorrect settings can raise cloud bills quickly. In his experience, Kubernetes autoscaling sometimes scaled far beyond expectations or failed to scale because the settings did not behave as intended. He recalls cases where a misconfigured background function or data pipeline consumed tens of thousands of dollars over a weekend. He also shares a security mistake from an early move from AWS to Google Cloud, when he accidentally pushed Google Cloud keys publicly to GitHub. The point is practical rather than heroic: production mistakes happen, and teams need safeguards, alerts, limits, and a culture where engineers can report errors.

"If you can't exploit as much as you can the business value out of your data, out of your work, then unfortunately there's a lot of things to do because we do need the business value."Stefano Bosisio11:01
Who should watch
  • You are moving from academia or data science into ML engineering and want an honest account of the skills that change outside research.
  • You are building an internal ML platform and need to improve adoption, documentation, demos, and stakeholder communication.
  • Your team is choosing between cloud services, external model providers, and internal engineering, with cost and operational risk in the balance.