Podcast

ML Platforms, Where to Start?

Olalekan Elesin, HRS Product Solutions GmbHEpisode 118 · 52:52 · Aug 2022 · 849 viewsHosted by Vishnu Rachkonda
Thumbnail for ML Platforms, Where to Start? Watch on YouTube
TL;DR
  1. 1

    Olalekan Elesin started the Scout24 AI platform by interviewing data scientists, testing prototypes, and solving the workflows that caused the most delay.

  2. 2

    The platform focused on notebook startup time, model deployment, and machine learning knowledge because surveys showed these mattered more to users than data quality.

  3. 3

    Olalekan Elesin evaluates build-versus-buy choices by asking whether a system differentiates the company and how well it fits the existing ecosystem.

Summary

Olalekan Elesin describes how he moved from engineering into technical product management while building an AI platform at Scout24. The team began with discovery because data scientists worried that a platform team would replace their work. Interviews and a survey helped them choose practical priorities: faster environments, faster model production, and better machine learning knowledge. They reduced notebook startup time, created a command-line workflow, added automatic shutdown and notebook saving, and published internal examples. The conversation also covers product adoption, emotional attachment to existing tools, and why internal platforms still need customer research. Olalekan explains how he compared SageMaker with MLflow, giving engineers room to test both while considering integration with the company's existing systems. He also discusses AutoML, AWS Glue, his AWS community work, and his approach to career growth. His advice is direct: start with user problems, measure time to value, and avoid building infrastructure that does not improve the business.

Key ideas
05:19

An AI platform needs discovery before implementation

At Scout24, the AI platform existed as a pillar on an organizational diagram, but nobody had defined what it should do. Olalekan Elesin moved from data engineering into a technical product management role and began with discovery. The team had clashes with data scientists because they feared losing control of their work. Instead of building a platform from an idea, the team investigated the problem, ran retrospectives, and learned how people wanted to engage. Olalekan says the first group to solve for was data scientists who were willing to work with the team. Only after that could the platform expand to other engineers and parts of the company.

09:04

Adoption depends on improving a familiar workflow

The existing notebook environment ran on AWS EMR and took roughly 10 to 15 minutes to start. Data scientists often needed a simple EC2 instance and Jupyter, rather than Spark, so some created their own setup with SSH tunneling. Olalekan's team built a command-line workflow that started a notebook without requiring users to manage scripts or move files between machines. He demonstrated it during customer interviews instead of asking people to adopt it from a description. Users who initially rejected the idea tried it after seeing it, then asked whether they could deploy a model from the same environment.

12:28

Internal platforms must respect users' attachment to their tools

Vishnu Rachkonda points out that technical professionals are closely tied to the tools that shape their daily work. Olalekan agrees and describes this as a challenge for an engineer learning to think from the customer's perspective. In his later data platform work, a data scientist told the team that people had used their existing technology for at least five years and wanted to know why they should change. Olalekan says internal teams cannot assume colleagues will use a new platform simply because it was built for them. They have to work backwards from the existing workflow and make the new experience better.

19:04

The platform's first priorities came from user research

Olalekan recommends starting with interviews that collect the problems people describe, then using a survey to rank them. At Scout24, the team spoke with about 10 colleagues and surveyed 40 people. The survey produced three leading needs: reducing the time to get an environment running, reducing the time to put a machine learning model into production, and increasing knowledge about building machine learning models. Data quality was among the least selected needs, despite being a common complaint in machine learning discussions. Because the team had only a few engineers, they focused on the top priorities rather than attempting to cover every platform concern.

21:20

Small workflow changes can produce measurable gains

The team reduced the generic environment's startup time from about 10 minutes to roughly three minutes or less. They began with a PowerPoint mock-up, moved to a CLI script, and then created a Python command-line application with commands for starting and shutting down notebooks. Interviews showed that users did not want to remember shutdown steps, so the team added automatic shutdown after an idle period and at the end of the day. Notebooks were saved automatically so users could resume their work. The broader production metric also improved from at least four weeks to at most three weeks for one data scientist, while one engineer reached production in about 24 hours or a few days.

24:12

Product management gives engineers room to test while keeping direction

Olalekan describes technical product work as repeated interviews, quick prototypes, and close collaboration between product and engineering. He brought an engineer into user interviews so the engineer could hear the problem directly and develop empathy for the people using the platform. The team tested SageMaker notebook behavior, built internal examples, and created SageMaker containers when that addressed a real user need. Olalekan also gave engineers guided autonomy when comparing MLflow with SageMaker. Letting an engineer evaluate MLflow created participation in the decision, while the final choice still depended on the time required to integrate the tool into the existing environment.

39:08

Platform choices should fit the systems that already run the company

When choosing between AWS services and focused tools, Olalekan first asks whether building the capability would differentiate the company. If it would not, he considers whether AWS solves the problem and then compares other products. At Scout24, the team found that SageMaker integrated more quickly than MLflow, even though MLflow appeared to offer more capabilities at the time. He also warns that a multi-cloud decision has application and data-transfer consequences. Moving machine learning workloads to Google Cloud while applications and data remain in AWS can create costs and design problems that teams may overlook when they follow product hype.

44:35

AutoML can help engineers use machine learning without replacing specialists

Olalekan sees an AI platform as serving data scientists who build business differentiation and engineers who want to apply machine learning without deep statistical training. He created Continuous Development for AutoML after reading a projection about AutoML adoption and wanting to bring that future closer. His aim was to let users specify datasets and features while the system handled parts of model selection and feature engineering. He argues that AutoML makes data scientists more efficient. Domain knowledge still matters because users need to know what they want to predict and which business problem they are solving.

"You have to walk backwards from that emotional connection and making it a lot better for them."Olalekan Elesin13:41
Who should watch
  • You are deciding whether an internal machine learning platform should exist and need a way to choose its first problems.
  • Your platform team has strong engineers, but users are resisting a new workflow or continuing to build their own tools.
  • You are comparing managed cloud services with focused products and need to account for integration, migration, and data-transfer costs.