# MLOps #37 When You Say Data Scientist Do You Mean Data Engineer? Lessons Learned From Startup Life

Elizabeth Chabot, Deloitte | MLOps Meetup | Episode 37 | 1:00:43
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=v2HzCcAT1t8
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/mlops-37-when-you-say-data-scientist-do-you-mean-data-engineer-lessons-learned
Published: 2020-10-12
Tags: data-engineering, data-quality, governance, monitoring

## TL;DR
- A production data product needs software checks, testing, monitoring, maintenance, and MLOps support alongside machine learning.
- Teams need to collect and govern data with its quality, integrity, reliability, and lineage in mind before they build predictive products.
- Companies should define data roles clearly and use machine learning only when it adds value beyond a simpler rule-based solution.

## Summary
Elizabeth Chabot draws on startup and consulting work to explain why data products often fail before the model is the real problem. Companies frequently hire data scientists when they need data engineering, application development, data governance, monitoring, or MLOps. They also assume that collected data is useful without checking how it was gathered, whether fields changed over time, or whether the data is still reliable. Chabot argues that leadership must own the wider data collection effort, while data scientists should have room to test algorithms and build predictive tools. Early products may need hard-coded rules or human review before machine learning is justified. Tool choice depends on the team, the use case, cost, and risk. A tool can fill a skills gap when the risk is low, but security and other high-risk areas need qualified people. Role definitions should change as a startup moves from an MVP to a product with real customers.

## Key ideas
### Production data products need more than a model
[01:35](https://www.youtube.com/watch?v=v2HzCcAT1t8&t=95s)
At a startup building Bookkeeping AI for small and medium-sized businesses, Elizabeth Chabot found that different financial structures forced the team to create many variations of its algorithms. The team had statisticians and people who understood machine learning, but they lacked the software engineering experience to maintain pipelines and fit them into the application. Leadership kept suggesting more data scientists as the answer. Chabot argued that the real need included software checks, testing, monitoring, maintenance, and someone who could connect the model to the product. She told startups that they would eventually need an MLOps person, or someone who could combine MLOps and data engineering skills.

### Bad data collection blocks predictive products before modeling starts
[06:21](https://www.youtube.com/watch?v=v2HzCcAT1t8&t=381s)
Chabot describes companies that wanted reporting, insights, or predictive analytics without having collected the required data. One company had not kept timestamps or useful attributes, and some identifiers stopped being unique. Another startup believed it had the right data for an algorithm, but its collection processes were not correcting the data properly. Her first questions are simple: what do you want to do, and do you have the data? Supervised machine learning needs historical information that was collected properly, with sufficient quality, integrity, reliability, and lineage. Startups change their processes quickly, so a field collected two years earlier may not mean the same thing today.

### Data collection and governance belong across the organisation
[11:24](https://www.youtube.com/watch?v=v2HzCcAT1t8&t=684s)
Chabot says data collection cannot usually be left to the first data scientist, especially when that person was hired to develop algorithms. Collection, governance, integrity, and reliability need leadership support because they affect the whole company. She recommends setting a shared baseline of terms, then checking whether the organisation has a data dictionary, business glossary, and active reporting that people actually review. In older companies, leadership may believe a field is still being collected while engineers know that a bug has caused it to output zeros for years. Chabot prefers addressing the full gap directly rather than revealing problems one at a time, because gradual disclosures can damage trust.

### A full-stack data scientist becomes a bottleneck when models need care
[19:10](https://www.youtube.com/watch?v=v2HzCcAT1t8&t=1150s)
Chabot warns against centring a company on one person who is expected to do everything. A data scientist who creates algorithms, monitors them, maintains them, and handles application integration soon stops being available for new work. She says the team may need to divide responsibilities among data science, data engineering, software engineering, DevOps, security, and MLOps as the company grows. Early employees can combine roles, but real customers bring service expectations that require dedicated time. She also describes a case where a data engineer and MLOps person maintained the database, created reporting, and built a machine learning pipeline, even though the first algorithm needed more iteration.

### A pipeline is worthwhile when the product already provides user value
[20:27](https://www.youtube.com/watch?v=v2HzCcAT1t8&t=1227s)
Chabot contrasts two startup hiring mistakes. One company devoted half its 20-person team to an algorithmic product that had not been tested well enough with users and was not clearly producing value. It had to cut half the team and reconsider the product. In another case, the team had a viable use case, positive user response to a semi-automated output, and a pipeline that could support further algorithms. The model was not yet as effective as hoped, but the surrounding system made iteration possible. Her distinction is that a pipeline is not the problem when the underlying product has value. Hiring a whole team before testing that value is the problem.

### Education helps leaders and practitioners agree on what data work requires
[26:18](https://www.youtube.com/watch?v=v2HzCcAT1t8&t=1578s)
Chabot says the gap between executives and people collecting data is mainly an education problem. Leaders may be uncomfortable with engineering, analytics, or code, while practitioners may not understand the business context. She recommends defining basic terms without assuming prior knowledge, then using the product roadmap to work backward from the outputs the company wants. Teams can identify the data points needed for a future result and discuss them with product owners and engineers. This exercise may take several sessions, especially when the company is changing direction, but it gives people a shared way to discuss collection, metrics, timing, and outputs.

### Machine learning should follow a demonstrated product need
[42:42](https://www.youtube.com/watch?v=v2HzCcAT1t8&t=2562s)
Chabot argues that investors and product teams often ask for machine learning when they really need an automated output. An initial MVP may be better served by a human review step or a simple hard-coded rule. If users confirm that the output saves them work or provides value, the company can then bring in a data scientist to test and improve the solution. She describes a data scientist who produced three algorithms in a Jupyter notebook, after which executives assumed the work was finished. The algorithms still needed engineering and production work. A data scientist hired for advanced modeling should not quietly become the organisation's data governance caretaker.

### Tool and hiring decisions depend on skills, cost, and risk
[45:50](https://www.youtube.com/watch?v=v2HzCcAT1t8&t=2750s)
Chabot used SageMaker for an MVP, but it did not fit the team's longer-term iteration needs. Her rule is to use a tool when the team lacks a skill, the tool can solve the problem, and support is available from a service representative or engineer. A tool may cost less than hiring a person, but the organisation still needs enough people who understand how it works. She makes a stricter distinction for high-risk areas such as security. A company should hire qualified people rather than assume a security product closes every gap. Teams should also post an MLOps role before customers use the system and the team can no longer track who is testing what.

## Notable quotes
- Elizabeth Chabot: "The problem is it can also break very quickly and that's fine if it's not in production if it's in production and you have more than 100 users it's not always great." (01:35)
- Elizabeth Chabot: "So universally we're just bad at knowing what we need to collect before we actually want to build the thing." (08:29)
- Elizabeth Chabot: "If you don't have the person and a tool will solve the issue and you know that you can get a service rep and a service engineer to help you with it then it's completely okay." (46:59)
- Elizabeth Chabot: "If it's your first product and you know that it's an if else solution and you can put it in place and you don't know anything after that is hiring at a data scientist at that point a good idea?" (53:42)

## Tools & references mentioned
- Deloitte
- Bookkeeping AI
- MLOps
- MLOps Community
- SageMaker
- Jupyter Notebook
- DevOps
- machine learning
- deep learning
- logistic regression
- decision science
- data science
- data engineering
- Power BI
- Tableau

## Who should watch
- You are hiring your first data or machine learning employees and need to separate data science, data engineering, software, DevOps, and MLOps work.
- Your organisation wants predictive features but has not checked whether the required data is collected consistently and can be trusted.
- You are deciding between a managed tool, a new hire, and a simpler rule-based MVP.

## Editor's note

Elizabeth Chabot says a team with machine learning skills can still lack the software engineering needed to maintain pipelines and fit them into the application. ZenML turns workflows into Python pipelines whose runs record steps, inputs, outputs, and code version, so the team can trace each result back through the work that produced it.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [The Journey from Data Scientist to MLOps Engineer](https://mlopstalks.com/talks/the-journey-from-data-scientist-to-mlops-engineer) (Ale Solano, 41:30)
- [Organisational Challenges of MLOps](https://mlopstalks.com/talks/organisational-challenges-of-mlops) (Adam Sroka, Origami Energy, 54:29)
- [MLOps: Isn't That Just DevOps?](https://mlopstalks.com/talks/mlops-isnt-that-just-devops) (Ryan Dawson, Seldon, 1:06:32)
- [MLOps: A Leader's Perspective](https://mlopstalks.com/talks/mlops-a-leaders-perspective) (Stephen Galsworthy, Quby, 54:08)
- [MLOps Insights](https://mlopstalks.com/talks/mlops-insights) (David Aponte-Demetrios Brinkmann-Vishnu Rachakonda, 37:47)