Maxime Beauchemin argues that putting data in one warehouse makes sense because data tends to be joined, while running machine learning there depends on the available compute and programming semantics.
2
Entity-centric modeling groups an entity with its attributes, facts, and time-windowed metrics, which can support feature stores, model training, scorecards, and broader data use.
3
Fast-moving data teams need to answer this week's question while building systems that can answer the same question reliably next year.
Summary
Maxime Beauchemin explains why the data warehouse has become a natural home for analytics and some machine learning work. Data wants to be brought together, and moving compute to the data can be easier than moving data elsewhere. He then describes entity-centric modeling, where a user, listing, or other entity is associated with demographics, facts, and time-windowed metrics. This pattern supports feature stores, but Maxime says its value extends to other uses, such as generating a user's scorecard. The conversation also covers metadata warehouses, lineage, ownership, and the need to make data assets understandable as platforms grow. Maxime distinguishes short-term analysis from longer-lived data engineering work. He also explains why Airflow attracted users beyond data engineering, and why he chose to build Preset around Superset instead. His advice for machine learning teams is grounded in data engineering: use immutable data blocks, deterministic tasks, provenance, and reproducibility when preparing training data.
Keeping data together can make warehouse-based machine learning practical
Maxime says data has a kind of gravity. It tends to be joined, unioned, and brought together, so putting it in a warehouse or lake can make sense. Running compute where the data lives may also be easier than moving the data to another system. He points to clusters in Snowflake and BigQuery, along with the possibility of running Python or machine learning functions inside SQL workflows. The choice still depends on whether SQL provides the right semantics and whether the platform can safely run code. His answer is open rather than absolute, but he sees warehouse-based machine learning as a reasonable direction.
Entity-centric modeling attaches features and facts to a clear entity
Maxime describes a machine learning dataset as a clear entity surrounded by many attributes, facts, and metrics. A user might have demographic attributes, seven-day visits, 28-day photo uploads, or a year's worth of Airbnb bookings. Dimensional modeling separates fact tables from dimension tables, while machine learning workflows often bring those values together around an entity such as a user or listing. Time-windowed metrics are especially useful because they can become predictors. Maxime says this arrangement is often called a feature store, but the same organization can support other work beyond feeding a model.
Entity-centric data has uses beyond model training
Maxime says an entity-centric feature store can hold much more value than training models. If a company precomputes a large set of attributes for a user, it can retrieve a scorecard or inspect what is happening with that user without rebuilding every metric from raw data. The model is one consumer of the data. Other consumers may need the same facts and time-windowed measurements for analysis, operational decisions, or understanding an account. The organizing idea is a clear protagonist, such as a user, plus a large collection of attributes and metrics associated with that entity.
Growing data platforms need a warehouse for metadata
Maxime agrees that companies can end up needing metadata about their metadata. He describes metadata about lineage, provenance, operational runs, ownership, labels, and business definitions. As a platform becomes more complex and accumulates history, people need to know where a table came from, what sits upstream and downstream, who owns it, and whether it can be trusted. He mentions Amundsen and DataHub as open source projects that collect this information in a central graph of data assets. Without that information, teams rely on tribal knowledge and repeatedly ask colleagues whether a table or feature is reliable.
Short-term answers and durable data systems require different work
For a fast-growing business, Maxime separates the person who answers this week's question from the person who builds a system that will work for years. An analyst or analytics engineer may query raw tables and produce a partial answer quickly. A data engineer may then turn that work into a repeatable process with source control, code review, data modeling, cost checks, and service-level expectations. The two time frames often need to run in parallel, although teams do not always have enough people to do both. In practice, the durable version may wait until an executive or an operational failure makes it unavoidable.
Machine learning and data engineering share infrastructure while keeping different concerns
Maxime says machine learning has its own concerns and practitioners, but it shares resources with the broader data platform. Notebooks and open-ended interpreters are useful on the machine learning side, while data engineers may spend more time in SQL. Some tools and workflows overlap. He refers to a modern data stack diagram that first combined machine learning with the rest of the platform, then separated it into its own diagram because the combined view had become too complicated. The split reflects different backgrounds and time frames, even though the systems still depend on one another.
Airflow was designed as a general scheduler and attracted machine learning users naturally
Maxime started Airflow at Airbnb in 2014 after seeing Facebook build a new data platform around systems such as Hive and Presto. Facebook had several internal schedulers for directed acyclic graphs of batch jobs expressed with SQL or Python. Those systems supported general computation, data science, machine learning data preparation, and training parallelism. Maxime describes Airflow as a way to express time-bound jobs and their dependencies, with the work inside them limited mainly by what Python can trigger. That general design allowed data scientists and machine learning engineers to use it even though data engineering was the initial setting.
Maxime chose Superset and Preset because data consumption matched his interests
Maxime says he became more interested in Superset than in orchestration because visual, interactive, and colorful data work appealed to him. Superset is an open source platform for dashboards, data exploration, and visualization, and Preset is the commercial open source company he started around it. He sees a higher risk and reward in challenging established business intelligence vendors with open source and a software-as-a-service model. Preset offers a free version for up to five users, so teams can connect data, build dashboards, and test value before speaking with sales. He also wants to improve the early user experience and encourage people to return to shared dashboards.
Machine learning data preparation should follow software engineering discipline
Maxime connects machine learning training data with established data engineering practices. Provenance and reproducibility matter because teams need to know what data trained a model and whether they can reproduce the result. In his discussion of functional data engineering, he recommends pure functions, immutable data blocks, and deterministic results. Running the same function with the same immutable input should produce the same output. This approach gives teams a way to explain results and build confidence in data preparation. The discipline applies before model training, where many failures can begin, rather than only inside the model code.
"Airflow will just allow you to express sets of jobs that depend on each other and provide guarantee about the order of those jobs and when they're gonna run."Maxime Beauchemin25:29
Who should watch
You are deciding whether analytics and machine learning workloads should move into a shared data warehouse.
Your data team is growing quickly and needs to balance urgent analysis with systems that can be maintained over time.
You are building training-data pipelines and need clearer provenance, reproducibility, and ownership.
