Podcast

Data Mesh: Data Quality Control Mechanism for MLOps?

Scott Hirleman, DataStaxEpisode 77 · 57:03 · Jan 2022 · 632 views
Thumbnail for Data Mesh: Data Quality Control Mechanism for MLOps? Watch on YouTube
TL;DR
  1. 1

    Data mesh organizes analytical data into products owned by the domains that understand the data best.

  2. 2

    Data products need documentation, usage agreements, consumer access, and standards that make data interoperable across domains.

  3. 3

    Teams should move toward data mesh through small, useful changes rather than trying to build the whole model at once.

Summary

Scott Hirleman explains data mesh as a way to organize analytical data around data products owned by business domains. A data product includes the raw data, its transformations, documentation, access patterns, and agreements about freshness and correctness. Domain teams own the data because they understand its meaning, while a central platform team provides shared infrastructure, standards, and guardrails. For ML teams, this can make training data easier to find and reuse. It can also give data producers more visibility into downstream consumers, reducing accidental breaks to models and pipelines. Scott is direct about the limits. Data mesh is still new, tools are incomplete, interoperability is difficult, and implementation differs widely between companies. He recommends starting with a few willing domains and a concrete problem, then expanding from there. A full implementation may be unsuitable for a small startup or for companies whose main problem is data quality rather than centralization.

Key ideas
05:50

Data mesh is a network of interoperable data products

Scott describes data mesh as a set of data products that work together, similar to how a service mesh coordinates services. A data product contains multiple related sets of data around a topic, rather than one table or stream. It is built for analytical use and should support questions that were not defined in advance. Consumers can combine products from different domains to answer new questions, instead of relying on a warehouse designed around a fixed set of expected queries.

08:17

A data product includes its transformations and consumer experience

Within data mesh, the data itself is the product. The product brings in raw data, controls its transformations, and defines how people consume it. A data scientist may need a repeatable way to pull the data rather than a CSV, while an analyst may prefer SQL. The product also describes its source, update schedule, trust level, and service-level agreements. Data updated every few minutes has different correctness expectations from data updated once a day.

11:22

Data ownership moves to the domains that understand it

The domains that know the business data best should own it and structure it with enough context for others to use. Scott says this is difficult without domain-driven design. Domain teams need data engineering support, a self-service platform, and shared infrastructure so they can focus on producing data products instead of managing Kafka, Spark, Snowflake, or other underlying systems. Shared standards are also needed, because high-quality data silos still create problems when products cannot be combined.

16:46

ML teams benefit from reliable sources and change awareness

For model development, data mesh should make it easier to discover available data and return to the same source repeatedly. Once a model is in production, data producers should understand which consumers depend on their data and communicate changes that could break those consumers. Scott says ML models should be treated as first-class consumers. The live inputs to a serving model do not necessarily come through the mesh, since he describes data mesh as focused on analytical data, but the mesh can support continued training and model updates.

17:33

Exploration still needs a human collaboration path

Adam asks what happens when a data scientist discovers an external source or needs data that is not in an existing product. Scott rejects the idea that the request should simply become a ticket dumped onto another team's backlog. The consumer should talk with the domain team, test the data in a less polished form, and decide together whether it belongs in a maintained product. That conversation can prevent a useful request from arriving months after the model or experiment has moved on.

22:15

Documentation requires resources and common blueprints

Scott says organizations cannot assign data ownership to domains without giving those teams time, people, and tooling. Documentation should live with the code and be available wherever the data is accessed. The platform team can provide blueprints for a standard data product and its documentation. Definitions also need care because terms such as customer can mean different things in marketing, sales, and support. A shared manual and clear standards reduce the burden on each domain.

24:57

The tooling is still largely assembled by each company

There is no product that an organization can deploy to create a data mesh automatically. Scott recommends starting with the tools a company already uses and asking how they can support the problem at hand. Companies may use streaming systems, Spark, Snowflake, object storage, or other components, but much of the integration is still their own work. Existing data tools often lack source-to-consumer lineage and ways to predict whether a schema change will break downstream users.

30:13

Data mesh is a gradual choice with real failure modes

Scott describes data mesh as an early, evolving practice rather than a finished blueprint. Its four broad pillars are domain ownership, data as a product, a self-service platform, and federated computational governance. Organizations can adopt the cultural shift without implementing every pillar immediately. A small startup may not have enough data engineering capacity for a full platform, while a large company with serious centralization problems may need more of the model. If teams create isolated products without interoperability, the result can be a data mess with more work than before.

50:17

A concrete business problem is the safest place to start

Scott recommends finding a few domains that want to serve their data and have a clear reason to collaborate with consumers. He cites examples from Intuit and DBG Media, where data was spread across acquired companies and product lines. Shared customer views created practical value, including easier access to subscribed products and better recommendations. If a company cannot find a problem with willing participants, or if its main issue is data quality rather than centralization, other approaches may be more appropriate than data mesh.

"The data product itself controls all of the transformation and then you also think about how it's going to be consumed is part of that product."08:54
Who should watch
  • You are an ML engineer who spends too much time finding training data or checking whether a source can be trusted.
  • Your data platform is centralized, and domain teams repeatedly change schemas without knowing which models or pipelines depend on them.
  • You are considering data mesh and need a candid view of its unfinished tooling, organizational cost, and practical starting point.