Machine learning teams lose track of work when tasks, requirements, datasets, and experiments stay on personal notes or local machines.
2
Flexible planning methods such as Kanban fit research better than rigid Scrum plans because new findings can create blocking tasks during a sprint.
3
Shared versioning and experiment tracking reduce repeated work and let teams reproduce results after people, data, or project conditions change.
Summary
Kseniia Melnikova walks through an ML development process based on CRISP-DM and focuses on mistakes in planning, data processing, and experiments. She argues that teams need shared task lists, common requirements, accessible documentation, versioned datasets, and a central record of experiments. Scrum can be difficult for research because new results often create work that was not in the original sprint plan, so she prefers Kanban for its flexibility. For datasets, she recommends DVC because it tracks versions and stores only changes instead of copying every full dataset. For experiments, she discusses MLflow, Sacred, and Omniboard, showing how shared tracking records code, parameters, metrics, and experiment sources. She also describes a Samsung project where paper-based records made it difficult to collect and reproduce results. Her advice is practical: automate repetitive records where possible, share information across the team, and keep stakeholders informed.
An ML project needs a shared process because its development loops create many versions
Kseniia describes an ML lifecycle that moves from goals and requirements through data collection, cleaning, preparation, training, evaluation, tuning, testing, and production monitoring. The process can loop back when testing shows that more data is needed, or when tuning requires repeated training. She says these loops create many versions that teams must manage and track if they want reproducible and automatable work. She focuses on the development stages rather than production, which she describes as a separate process concerned mainly with monitoring and synchronization. After monitoring, a team may also need to define new goals.
Personal notes and private task lists make team responsibilities disappear
The first mistake is treating work as a personal record instead of a team record. People may memorize tasks, write notes for themselves, discuss work orally with a manager, or leave tasks in a private paper or group chat. Kseniia says that when work is not tracked and shared, responsibility is unclear and tasks are easy to lose or forget. She recommends tools such as Jira, Confluence, Azure DevOps, ClickUp, or even Excel files. Small tasks should be recorded too. A Kanban view can show what each person is doing, while shared requirements and documents give the team one place to check goals and decisions.
Research teams need planning methods that can absorb new work
Kseniia explains why she does not prefer Scrum for much of ML research. A Scrum plan may set a task list for two, three, or four weeks and make it difficult to interrupt the backlog. Research can produce new tasks immediately, and a result may create a blocking task that was not known at the start of the sprint. A research task may also take longer than its planned time. She prefers Kanban because it is more flexible. She also mentions TDSP, although its amount of documentation can be a disadvantage, and Lean as another approach teams may consider.
Unversioned datasets create storage waste and make results hard to share
The data processing mistake is chaos among dataset versions. Two data scientists may copy shared data to local storage and work on separate branches without a reliable way to align them. Teams may also duplicate an entire dataset for every small change, which increases storage use and makes it difficult to know which version produced a result. Kseniia recommends DVC because it controls versions, stores them in one place, and caches changes instead of copying the same dataset repeatedly. If work must happen on a local machine, results and final dataset versions should be copied to a shared network. Every team member should be able to access every dataset version.
Experiment records belong in a shared system rather than on paper or in fragile files
Kseniia calls missing experiment tracking a serious risk because teams can lose good models and parameters, and they cannot easily compare work. She recalls teams at Samsung recording results on real paper. Manual spreadsheets, CSV files, or scripts also create opportunities for input errors, such as a missing zero or an incorrect decimal mark. They make visualization harder and can leave the team without one common source of experiment results. She recommends shared systems such as MLflow, or Sacred with Omniboard. Her example MLflow interface records the user, code source, version, hyperparameters, and metrics, giving the team a common place to inspect experiments.
Information sharing must include stakeholders as well as engineers
Kseniia identifies a lack of information sharing as mistake zero. Online work removes informal office conversations, so she recommends more meetings and stand-ups. Her team held a stand-up three times a week, with each member giving a short status update. She also advises mapping stakeholders by their interest and power. A team manager may need close involvement, while a manager's manager may only need periodic updates about results. The point is to decide how each stakeholder receives information instead of sharing everything in the same way. Shared wikis should contain requirements, research papers, processes, and other information that teammates may need.
Reproducibility still matters when one engineer owns the ML work
In response to a question about solo ML engineers, Kseniia says experiment tracking helps explain work to management and reduces repeated verbal explanations. The engineer may also need to return to an old result months later, or hand the project to someone else when leaving the company. She prefers an experiment tracking tool to manual Confluence updates because automatic logging takes less effort and can record results directly from code. She says tools such as MLflow can be added with a small amount of code. The underlying concern is continuity: a result that nobody else can reproduce is difficult for the company to use.
Paper-based records forced Samsung teams to repeat work and blocked collaboration
Kseniia describes finding that teams at Samsung used paper to record tasks and experiment results. This made it hard to collect results across projects and prevented colleagues in other locations from reproducing experiments. She says the cost includes cloud compute time spent rerunning experiments, the salaries of people trying to recover lost information, and extra storage purchased for unmanaged dataset copies. A model that cannot be reproduced or shared is not useful as production work, in her view. She also agrees with Demetrios Brinkmann that teams need to establish the business objective and understand stakeholder goals before building.
