Data science teams should choose problems based on potential business return and their likelihood of solving them with data science.
2
A project is unfinished or unsuccessful when it has not changed business operations and produced measurable results, even if the model works.
3
Quarterly OKRs create focus, while a version-controlled handbook creates consistent processes as the team grows.
Summary
Luigi Patruno describes how his work changed after moving from individual contributor to data science leader. He starts with the business problem, defines a measurable outcome, and then decides whether machine learning is an efficient way to address it. His team prioritizes problems with large potential returns, enough data, and a reasonable chance of success. He measures success by whether another business team changes its operations and sees results, rather than by model accuracy alone. Luigi also explains how stakeholder commitment affects project success, why his team mixes safer projects with exploratory work, and how quarterly OKRs keep projects focused. A shared handbook, stored as version-controlled Markdown files, documents the team's processes and reduces repeated questions. He closes with management advice on setting clear goals, giving specific positive feedback, correcting problems quickly, and coaching employees toward solutions instead of taking their work from them.
Business problems come before machine learning techniques
Luigi says successful projects keep their goal at the forefront. The team begins with a real business problem and defines a measurable objective that shows whether the problem has been solved. Machine learning is only one possible tool. Traditional software engineering may be more cost effective. His internal optimization team looks for problems that cannot be handled well with traditional software and may benefit from learning from data. They then ask whether solving the problem could have an outsized return and whether the team has a decent chance of succeeding.
A working model is not the same as a solved business problem
Luigi rejects measuring a team's success by model metrics alone. A model can achieve a particular accuracy while the underlying business problem remains untouched. He considers the work a cost to the business until the team has helped another department change its operations and produce measurable results. This makes the data science team a group of change agents. They need to work from the partner's perspective, help implement the change, and calculate the return against the time and effort spent. Efficiency matters alongside impact.
Stakeholder commitment predicts whether an internal project can work
Luigi distinguishes between stakeholder buy-in and stakeholder commitment. Buy-in means people agree that a problem matters. Commitment means they will spend time helping solve it and will help their team operate differently. Projects tend to work better when teams ask for help because they already feel the problem strongly. Projects become harder when the data science team invents a solution and tries to sell it to a group that has not asked for it. His team uses early conversations to identify whether stakeholders are invested before spending an individual contributor's time on modeling.
The team combines product, data science, and engineering roles
Luigi's data science organization includes product managers, data scientists, and machine learning engineers. Some data scientists work in centralized project teams, while others are embedded in product squads. Product managers understand stakeholder needs, scope projects, help set OKRs, and track business objectives. Data scientists build models and conduct analysis. Machine learning engineers work vertically inside projects and horizontally on the internal machine learning platform. Everyone uses the platform, while the team structure determines how people work with business stakeholders.
OKRs give uncertain data science work a clear operating focus
Luigi uses strict quarterly objectives and key results. Key results are tangible and measurable, and each one has a single directly responsible individual. That person is accountable for whether the effort succeeds. Luigi's role is to make sure the set of projects is ambitious enough and worthwhile for the organization. He describes taking a three-week vacation without logging in and returning to find that work had continued because people knew their goals and responsibilities. For uncertain projects, the team can run a quarter-long proof of concept before deciding whether to increase investment.
Project portfolios should include both likely wins and uncertain bets
Data science projects have more uncertainty than ordinary development work, so Luigi does not put every investment into one initiative. His team works with different parts of the business and mixes projects with a strong history or close precedent with newer projects whose success is harder to predict. A new initiative can begin as a limited proof of concept rather than a promise of a full solution. This approach lets the team pursue large potential impact while limiting the damage if one project fails.
A version-controlled handbook creates discipline as the team grows
Luigi says processes become necessary as a team grows. His team borrowed the handbook model from GitLab and keeps its data science handbook as Markdown files in GitHub, where changes are version controlled. The handbook documents quarterly planning, project selection, data science product development, analysis expectations for different audiences, and technical guidance. Every team member can contribute through a pull request, and everyone is expected to contribute. Luigi says the handbook has grown to roughly 30 to 50 pages, with each team member contributing at least one page. People answer fewer repeated questions because they can point colleagues to a documented process.
Managers should coach employees instead of taking over their work
Luigi says managers should set direction, reinforce good work, and correct problems quickly. He gives specific praise for behavior that others should imitate, such as a thorough pull request review, and gives critical feedback privately. He has also learned to resist his perfectionist impulse to take over. When an employee asks what to say to a stakeholder, he asks what the employee thinks should be said and helps refine the answer. He wants people to bring a proposed solution even when their confidence is low, since confidence can be developed through coaching.
Luigi sees excessive conservatism as a leadership blind spot
Luigi says his desire to succeed can make him favor projects with familiar stakeholders, proven techniques, and prior data about success rates. That reduces risk, but it can also keep the team from making larger bets. He points to language model work as an area where 2U had a successful internal hackathon but had not yet started an actual proof of concept. His challenge is to keep the team's success rate high without avoiding projects whose potential is large but whose outcome is harder to predict.
"Machine learning in my opinion is just one tool. The thing that's more important is what's the problem and how do you know if you've solved it."Luigi Patruno10:14
Who should watch
You lead a data science or machine learning team and need a way to connect project work to business results.
Your internal data team struggles to get stakeholder commitment or to move a successful model into day-to-day operations.
You are taking on management responsibilities and want practical advice on goals, feedback, delegation, and team processes.
