# Data Mesh: Data Quality Control Mechanism for MLOps?

Scott Hirleman, DataStax | MLOps Coffee Sessions | Episode 77 | 57:03

Source: https://www.youtube.com/watch?v=OAIT3WlDyx4
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/data-mesh-data-quality-control-mechanism-for-mlops
Published: 2022-01-28
Tags: data-engineering, data-quality, governance, platform-teams

## TL;DR
- Data mesh organizes analytical data into products owned by the domains that understand the data best.
- Data products need documentation, usage agreements, consumer access, and standards that make data interoperable across domains.
- Teams should move toward data mesh through small, useful changes rather than trying to build the whole model at once.

## Summary
Scott Hirleman explains data mesh as a way to organize analytical data around data products owned by business domains. A data product includes the raw data, its transformations, documentation, access patterns, and agreements about freshness and correctness. Domain teams own the data because they understand its meaning, while a central platform team provides shared infrastructure, standards, and guardrails. For ML teams, this can make training data easier to find and reuse. It can also give data producers more visibility into downstream consumers, reducing accidental breaks to models and pipelines. Scott is direct about the limits. Data mesh is still new, tools are incomplete, interoperability is difficult, and implementation differs widely between companies. He recommends starting with a few willing domains and a concrete problem, then expanding from there. A full implementation may be unsuitable for a small startup or for companies whose main problem is data quality rather than centralization.

## Key ideas
### Data mesh is a network of interoperable data products
[05:50](https://www.youtube.com/watch?v=OAIT3WlDyx4&t=350s)
Scott describes data mesh as a set of data products that work together, similar to how a service mesh coordinates services. A data product contains multiple related sets of data around a topic, rather than one table or stream. It is built for analytical use and should support questions that were not defined in advance. Consumers can combine products from different domains to answer new questions, instead of relying on a warehouse designed around a fixed set of expected queries.

### A data product includes its transformations and consumer experience
[08:17](https://www.youtube.com/watch?v=OAIT3WlDyx4&t=497s)
Within data mesh, the data itself is the product. The product brings in raw data, controls its transformations, and defines how people consume it. A data scientist may need a repeatable way to pull the data rather than a CSV, while an analyst may prefer SQL. The product also describes its source, update schedule, trust level, and service-level agreements. Data updated every few minutes has different correctness expectations from data updated once a day.

### Data ownership moves to the domains that understand it
[11:22](https://www.youtube.com/watch?v=OAIT3WlDyx4&t=682s)
The domains that know the business data best should own it and structure it with enough context for others to use. Scott says this is difficult without domain-driven design. Domain teams need data engineering support, a self-service platform, and shared infrastructure so they can focus on producing data products instead of managing Kafka, Spark, Snowflake, or other underlying systems. Shared standards are also needed, because high-quality data silos still create problems when products cannot be combined.

### ML teams benefit from reliable sources and change awareness
[16:46](https://www.youtube.com/watch?v=OAIT3WlDyx4&t=1006s)
For model development, data mesh should make it easier to discover available data and return to the same source repeatedly. Once a model is in production, data producers should understand which consumers depend on their data and communicate changes that could break those consumers. Scott says ML models should be treated as first-class consumers. The live inputs to a serving model do not necessarily come through the mesh, since he describes data mesh as focused on analytical data, but the mesh can support continued training and model updates.

### Exploration still needs a human collaboration path
[17:33](https://www.youtube.com/watch?v=OAIT3WlDyx4&t=1053s)
Adam asks what happens when a data scientist discovers an external source or needs data that is not in an existing product. Scott rejects the idea that the request should simply become a ticket dumped onto another team's backlog. The consumer should talk with the domain team, test the data in a less polished form, and decide together whether it belongs in a maintained product. That conversation can prevent a useful request from arriving months after the model or experiment has moved on.

### Documentation requires resources and common blueprints
[22:15](https://www.youtube.com/watch?v=OAIT3WlDyx4&t=1335s)
Scott says organizations cannot assign data ownership to domains without giving those teams time, people, and tooling. Documentation should live with the code and be available wherever the data is accessed. The platform team can provide blueprints for a standard data product and its documentation. Definitions also need care because terms such as customer can mean different things in marketing, sales, and support. A shared manual and clear standards reduce the burden on each domain.

### The tooling is still largely assembled by each company
[24:57](https://www.youtube.com/watch?v=OAIT3WlDyx4&t=1497s)
There is no product that an organization can deploy to create a data mesh automatically. Scott recommends starting with the tools a company already uses and asking how they can support the problem at hand. Companies may use streaming systems, Spark, Snowflake, object storage, or other components, but much of the integration is still their own work. Existing data tools often lack source-to-consumer lineage and ways to predict whether a schema change will break downstream users.

### Data mesh is a gradual choice with real failure modes
[30:13](https://www.youtube.com/watch?v=OAIT3WlDyx4&t=1813s)
Scott describes data mesh as an early, evolving practice rather than a finished blueprint. Its four broad pillars are domain ownership, data as a product, a self-service platform, and federated computational governance. Organizations can adopt the cultural shift without implementing every pillar immediately. A small startup may not have enough data engineering capacity for a full platform, while a large company with serious centralization problems may need more of the model. If teams create isolated products without interoperability, the result can be a data mess with more work than before.

### A concrete business problem is the safest place to start
[50:17](https://www.youtube.com/watch?v=OAIT3WlDyx4&t=3017s)
Scott recommends finding a few domains that want to serve their data and have a clear reason to collaborate with consumers. He cites examples from Intuit and DBG Media, where data was spread across acquired companies and product lines. Shared customer views created practical value, including easier access to subscribed products and better recommendations. If a company cannot find a problem with willing participants, or if its main issue is data quality rather than centralization, other approaches may be more appropriate than data mesh.

## Notable quotes
- "There is no silver bullet when you want to say, 'All right, what's the nitty-gritty? How can I get down and dirty with this data mesh?'" (02:01)
- "The data product itself controls all of the transformation and then you also think about how it's going to be consumed is part of that product." (08:54)
- "If you try and do it all up front, you're gonna fail." (52:13)
- "You don't have to put in all of the work up front. You can evolve towards this." (51:08)

## Tools & references mentioned
- data mesh
- data products
- service mesh
- Jamac
- domain-driven design
- data catalogs
- Kafka
- Spark
- Snowflake
- S3
- data lineage
- federated computational governance
- GDPR
- Saxo Bank
- Glue
- Flexport
- dbt
- JPMorgan Chase
- Intuit
- DBG Media
- Team Topologies
- Data Mesh Learning
- Data Mesh Radio

## Who should watch
- You are an ML engineer who spends too much time finding training data or checking whether a source can be trusted.
- Your data platform is centralized, and domain teams repeatedly change schemas without knowing which models or pipelines depend on them.
- You are considering data mesh and need a candid view of its unfinished tooling, organizational cost, and practical starting point.

## Editor's note

Scott Hirleman says ML models should be treated as first-class consumers of data, with producers aware of the consumers that depend on their data. ZenML records each run's inputs, outputs, and code version, so a model or artifact can be traced back to the data and code that produced it. This gives ML teams a record of model lineage.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Scaling Machine Learning with Data Mesh](https://mlopstalks.com/talks/scaling-machine-learning-with-data-mesh) (Shawn Kyzer, Thoughtworks, 53:54)
- [How Data Platforms Affect ML & AI](https://mlopstalks.com/talks/how-data-platforms-affect-ml-ai) (Jake Watson, The Oakland Group, 39:12)
- [Exploring SQLmesh](https://mlopstalks.com/talks/exploring-sqlmesh) (, 1:18:58)
- [Data Engineering for ML](https://mlopstalks.com/talks/data-engineering-for-ml) (Chad Sanderson, Convoy, 57:54)
- [Building an ML Platform from scratch](https://mlopstalks.com/talks/building-an-ml-platform-from-scratch) (, 1:46:08)