# Building for Small Data Science Teams

James Lamb, SpotHero | MLOps Coffee Sessions | Episode 69 | 52:26
Hosted by Adam Sroka

Source: https://www.youtube.com/watch?v=yAsPfhI5Jd8
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/building-for-small-data-science-teams
Published: 2021-12-20
Tags: classical-ml, experiment-tracking, orchestration, platform-teams

## TL;DR
- James Lamb is deliberately starting SpotHero's machine learning platform with cloud-based experimentation rather than deployment and monitoring.
- A four-person data science team can accept more local conventions, such as Python-only workflows and pickled scikit-learn-compatible artifacts, because it has less code and fewer models to maintain.
- Machine learning engineers help data scientists and engineers work together by translating requirements and building self-service systems around clear contracts.

## Summary
James Lamb describes how SpotHero is designing machine learning infrastructure for a small data science team. When he joined, four data scientists and a small data engineering team were using bespoke Airflow jobs for batch model scoring. SpotHero had no shared machine learning platform yet. Lamb argues that the team should first improve experimentation, with cloud training, access to larger resources, and orchestration for hyperparameter sweeps. Deployment and monitoring can follow once the team has real model use cases. He compares this approach with a previous job where strict guardrails made one class of model easy to productionize but failed when models became more varied. The conversation also covers LightGBM's design and open-source community, model packaging, when small teams should write their own tools, and the role of machine learning engineers as translators between data science and engineering. Lamb is candid that team size and company priorities should shape architecture decisions.

## Key ideas
### LightGBM speeds tree training by reducing the data considered at each split
[03:40](https://www.youtube.com/watch?v=yAsPfhI5Jd8&t=220s)
James Lamb explains that LightGBM is a framework for supervised learning on tabular data with gradient boosting and tree-based learners. Its major innovation is to bucket continuous features into histograms before training. With a default of 255 bins, the algorithm considers histogram boundaries instead of every possible threshold. This makes training faster and reduces the in-memory size of the training data. The smaller representation also makes distributed training more practical because machines exchange less information. Lamb says LightGBM supports binary and multiclass classification, regression, and learning-to-rank problems.

### LightGBM's open-source community is small and layered
[09:27](https://www.youtube.com/watch?v=yAsPfhI5Jd8&t=567s)
Lamb says the LightGBM repository can intimidate new contributors because it includes a non-trivial C++ library, a C API, JVM bindings, R and Python packages, Docker files, shell scripts, and CMake-based builds. He wants to make contributions possible without understanding every component, but says he has seen fewer small contributions than in the Dask ecosystem. He describes a small group of core maintainers, active contributors who work on larger features, and occasional committers. He estimates two to four very active maintainers, another four to five people with commit rights who contribute less often, and about eight to twelve active contributors who appear every few months.

### SpotHero is prioritizing experimentation before production infrastructure
[14:15](https://www.youtube.com/watch?v=yAsPfhI5Jd8&t=855s)
When Lamb joined SpotHero, the machine learning platform was still at the stage of architecture proposals and vendor discussions. The team had four data scientists, with one or two additional people doing applied machine learning work. Model deployment consisted of bespoke Airflow jobs and batch scoring. Lamb's first priority was cloud-based training that could use larger resources or GPUs, plus experiment orchestration for runs such as a 50-job hyperparameter sweep. He would be satisfied if the team later had many validated experiments but still had difficult productionization work, because those experiments would reveal what deployment and monitoring systems actually needed to support.

### Real model variety should shape production systems
[18:48](https://www.youtube.com/watch?v=yAsPfhI5Jd8&t=1128s)
Lamb recalls a previous platform with extensive guardrails and an easy production path for a particular class of tabular classification models. Problems appeared when a model produced multiclass probability arrays or when a forecasting model returned arrays of timestamps and values. The existing monitoring and deployment assumptions did not handle those outputs well. This experience led him to prefer validating more kinds of experiments before fixing the production path at SpotHero. The team can then choose deployment and monitoring designs based on models it really expects to run.

### Small teams can accept simple local conventions for longer
[29:41](https://www.youtube.com/watch?v=yAsPfhI5Jd8&t=1781s)
SpotHero uses Python for machine learning, pickled scikit-learn API-compatible objects as scoring artifacts, and Airflow pipelines that load those files from S3. Lamb acknowledges that this prevents the team from using other languages or frameworks without extra wrapper work. He thinks the trade-off is acceptable because the team has four data scientists and is not working with hundreds of models. A larger team might assign people to maintain a shared data-loading library, while a small team may reasonably let data scientists write SQL directly against its data stores.

### Internal tools can express company data in business terms
[34:53](https://www.youtube.com/watch?v=yAsPfhI5Jd8&t=2093s)
Lamb sees a possible future where SpotHero builds an experiment system around its existing Airflow knowledge. Data scientists could provide Docker images, Airflow could run them with the Kubernetes pod operator, and a shared library could store and version artifacts and metrics. He also sees value in middleware between data stores and training code. Such libraries could expose concepts such as customer data, purchase data, home addresses, or geographic points instead of forcing every data scientist to write raw SQL. He points to Salesforce's higher-level feature engineering approach as an example of this kind of abstraction.

### Machine learning engineers translate between technical groups
[43:50](https://www.youtube.com/watch?v=yAsPfhI5Jd8&t=2630s)
Lamb describes translation as one of the most important parts of a machine learning engineer's job. The role turns data science requirements into engineering requirements and explains engineering constraints in terms data scientists can use. At SpotHero, machine learning engineering sits within data engineering, which Lamb describes as an enablement team. It builds self-service libraries, services, and infrastructure with explicit contracts. A joint group of analysts, data scientists, data engineers, and the machine learning engineer also meets regularly, giving the teams a place to explain their work and ask for help.

### Team alignment can matter more than a particular technical design
[48:09](https://www.youtube.com/watch?v=yAsPfhI5Jd8&t=2889s)
Lamb says he is most proud of work at a previous company where engineering and data science rebuilt a platform together. Engineers learned what was needed to power machine learning models, while data scientists understood why some engineering decisions were necessary. He contrasts this with an earlier system that stored high-volume time-series data efficiently but gave data scientists opaque UUIDs with no service to explain what they represented. His positive example came from teams discussing trade-offs together instead of sending requests between disconnected groups.

## Notable quotes
- James Lamb: "I think that that translation part is really really important that you're able to sort of like you know for example if you ask a data science team hey do you need another three months of history for this data set they're not gonna say no." (44:13)
- James Lamb: "I would be so happy if six months from now we had a ton of ideas that we felt were sort of proved out in the lab and we were ready to productionalize and productionalizing them still sucked." (17:54)
- James Lamb: "We have a small data science team a small data engineering team relative small relative to the amount of data we're collecting and the like sort of opportunity space." (26:21)
- James Lamb: "I think that the thing I'm most proud of in my career so far is how do i explain this at a previous job." (48:09)

## Tools & references mentioned
- SpotHero
- LightGBM
- Microsoft Research
- Saturn Cloud
- Dask
- XGBoost
- CatBoost
- Nadia Eghbal
- Working in Public
- Apache Airflow
- Seldon Core
- MLflow
- ONNX
- PMML
- PFA
- Python
- R
- Julia
- scikit-learn
- Amazon S3
- Kubernetes
- Salesforce

## Who should watch
- You are building machine learning infrastructure for a small team and need to decide what to standardize now.
- Your organization is choosing between improving experimentation and building a polished deployment and monitoring path.
- Data scientists and engineers are separated by unclear requirements, unfamiliar tools, or hand-offs between teams.

## Editor's note

James Lamb says SpotHero should improve cloud-based training and experiment orchestration before building deployment and monitoring systems. ZenML lets teams write these workflows as Python pipelines whose runs record steps, inputs, outputs, and code versions, while unchanged steps are cached. That gives a small team a traceable record of experiments without forcing production infrastructure before it knows what models it will run.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Machine Learning at Reasonable Scale](https://mlopstalks.com/talks/machine-learning-at-reasonable-scale) (Jacopo Tagliabue, Coveo, 1:04:32)
- [Platform Thinking: A Lemonade Case Study](https://mlopstalks.com/talks/platform-thinking-a-lemonade-case-study) (Orr Shilon, Lemonade, 51:42)
- [A Journey in Scaling ML](https://mlopstalks.com/talks/a-journey-in-scaling-ml) (Gabriel Straub, Ocado Technology, 52:41)
- [Lessons from Studying FAANG ML Systems](https://mlopstalks.com/talks/lessons-from-studying-faang-ml-systems) (Ernest Chan, Duo Security, 45:32)
- [Machine Learning Engineering in Action](https://mlopstalks.com/talks/machine-learning-engineering-in-action) (Ben Wilson, 53:05)