Data scientists need to distinguish fast exploratory code from code that is ready to run repeatedly in production.
2
Data scientists do not need to become Kubernetes experts, but they should understand software engineering language, system boundaries, security, testing, and deployment practices.
3
Large language models are bringing statistical analysis and machine learning knowledge back together because evaluating their outputs across many tasks is difficult.
Summary
Catherine Nelson argues that data scientists need software engineering skills because exploratory work increasingly becomes part of production systems. She explains that notebooks and quick scripts are useful while testing an idea, but they need to be refactored before other people depend on them. Her book focuses on Python developers who are not building web applications and want practical guidance on APIs, testing, security, code structure, Git, object-oriented programming, and CI/CD. Nelson does not think every data scientist should learn Kubernetes. Instead, teams should provide support and clear system requirements, while data scientists learn enough of the surrounding system to communicate and understand production code. She also says large language models have changed the shape of data science. Evaluating one model across many tasks requires both statistical thinking and knowledge of how machine learning systems work.
The book exists because standard software engineering material was inaccessible to data scientists
Catherine Nelson says she wrote the book after discovering that her early data science training had not prepared her to work on a software team. She knew how to do data science, but did not know basic terms such as API or test. The books she found were aimed at web developers and often used Java or other languages that added unnecessary barriers. Her guide focuses on people writing Python who are not developing for the web. It explains software engineering ideas in situations that match data science work, so readers can learn how to write better code without first becoming web developers.
Exploration and production require different standards for code
Nelson accepts that data science often starts with long scripts and notebooks because the work is exploratory. At that stage, the main question is whether an idea is worth pursuing, so code quality is less urgent than learning about the data and testing assumptions. Once the result needs to run in production, the standard changes. The code must be easier to understand, maintain, test, and run repeatedly. Nelson says data scientists should know which code is temporary and which code needs refactoring. Reusable pieces should become efficient, tested libraries rather than being passed on as unfinished spaghetti code.
Security gaps come from the training environment data scientists usually see
Nelson identifies security as another area that data science courses often omit. Training commonly uses open datasets, open-source packages, and simple model serialization, so students may never consider the risks of handling private data or downloading dependencies. Those assumptions change when a data scientist works with real people's data inside a software team. Nelson says practitioners should understand terms such as risk, threat, and vulnerability, know where to find relevant security issues, and be aware of attacks against machine learning models. Monitoring model behaviour and analysing performance can also help detect problems such as data poisoning.
Moving from data science to machine learning engineering requires a change in mindset
Nelson describes data science as a job built around uncertainty and exploration. Machine learning engineering places more weight on making a process run repeatedly, scale, and remain efficient, tested, and reliable. The transition is therefore about more than learning another tool. People need to become interested in how each task is implemented, how pieces fit together, and how the system will behave beyond the first successful experiment. Experiments still happen, but they take place inside a narrower and more standardised process. Nelson has seen people make this transition by paying closer attention to the code they write and learning how to improve it.
Production work needs an explicit refactoring stage after the proof of concept
Nelson is firmly in favour of refactoring messy exploratory code instead of putting it directly into production. She says teams have discussed for years whether notebooks should run in production or whether they should be converted into code that fits the wider system. The practical problem is that many data science project plans do not include time for this conversion. Refactoring can require skills that are outside a data scientist's usual work, so teams need to plan for it. Keeping the data scientist involved also protects the reasoning behind the model and makes later updates easier.
Data scientists need enough system knowledge to work with engineers
Nelson does not argue that every data scientist must become an expert in every part of a production system. She wants them to understand the language and structure well enough to read code, communicate with software and data engineers, and understand system design documents. Useful foundations include object-oriented programming, unit and integration tests, CI/CD, Git, and standardisation. She also recommends understanding the incentives of other disciplines, since software engineers may care about maintainability and consistency in ways that differ from an exploratory data scientist's immediate goals. A data scientist can remain strongest in exploration while still understanding the production code built from that work.
Data science job descriptions often ask one person to cover too much
The conversation returns several times to the vague meaning of the data scientist role. Nelson says it is unfair to expect one person to handle modelling, data pipelines, CI/CD, Kubernetes, monitoring, and every other part of the machine learning lifecycle. She understands why this happens because data science attracts people who enjoy learning new areas, and some people willingly grow into broad roles. Still, the work may amount to several jobs. Her preferred shape is for someone to understand the full lifecycle while developing deeper expertise in one area. Teams should also provide support instead of treating Kubernetes or platform operations as an automatic requirement for every data scientist.
Large language models are reconnecting statistical and machine learning skills
Nelson says her view of data science changed after she began consulting for a generative AI startup. She had expected the field to split further between analytics and machine learning engineering. Large language models made her reconsider that view because evaluating them requires statistical knowledge and an understanding of model behaviour. A traditional model usually targets one problem, which makes evaluation more direct. An LLM may perform many tasks, and a change can improve one task while hurting another. Evaluation therefore becomes a difficult data science problem, especially when outputs need to be assessed as a large corpus rather than one answer at a time.
"There's an initial setup cost to setting up tools and to automate processes that you do repeatedly, but once you've done that and you can just press a button and run your test and lint your code and run the formatting, it's great."Catherine Nelson51:57
Who should watch
You are a data scientist whose notebooks or scripts may soon become part of a production system.
You work with data scientists and need a shared vocabulary for APIs, tests, security, Git, CI/CD, and system design.
You are deciding how much platform knowledge, including Kubernetes, a modelling-focused data scientist should be expected to learn.
