Bad data is inaccurate, incomplete, or misleading, and its cost includes wasted work, lost revenue, damaged trust, legal exposure, and harm caused by biased decisions.
2
Data quality problems can enter at every stage of the data lifecycle, from defining a feature to sharing an analysis or dashboard.
3
Teams should diagnose problems by tracing the lifecycle backwards, then improve quality through shared ownership, documentation, reusable code, alerts, reconciliation, automation, governance, and regular audits.
Summary
Shailvi defines bad data as information that is inaccurate, incomplete, or misleading. Data that produces an unwanted result is not automatically bad if it is accurate and complete. She explains why quality matters through the costs of wasted engineering time, lost revenue, lost opportunities, damaged trust, compliance failures, biased decisions, and low team morale. The talk follows the data lifecycle from definition and logging through transformation, analysis, and sharing. Each stage can introduce problems such as inconsistent definitions, missing events, faulty pipelines, unclear rules, logical errors, biased algorithms, faulty charts, or unintended reuse. For diagnosis, teams should first check whether the data is actually wrong, then trace the lifecycle backwards, form hypotheses, and test them. Improving quality requires work across product, marketing, design, and data teams. Shailvi recommends documentation, reusable modules, pipeline alerts, reconciliation, automation, governance, and a plan to measure and audit quality issues.
Bad data is inaccurate, incomplete, or misleading, rather than merely inconvenient
Shailvi defines bad data as data that is inaccurate, incomplete, or misleading. A result that people dislike is not automatically a quality problem. If the data is accurate and complete, it remains valid even when it challenges expectations. This distinction helps teams avoid treating inconvenient evidence as an error. She says poor quality creates costs through extra investigation, reconciliation, maintenance, and repeated fixes. It can also cause lost revenue, such as when an incorrect automated pricing model sets the wrong price, or lost opportunities when analysis points the business toward the wrong choice.
Data quality affects trust, compliance, ethics, and team morale
Quality problems damage internal and external trust. Internal stakeholders may reject valid results because they have seen unreliable data before. Customers may hesitate to follow product guidance, while competitors can take advantage of that lack of confidence. In regulated industries, poor data can create compliance and legal problems. Shailvi also connects hidden bias in data to direct harm for real people. Internally, repeated manual investigation is frustrating for data professionals. She argues that teams lose talent when people spend their time fixing recurring issues that could have been addressed through shared processes and ownership.
Every stage of the data lifecycle can introduce a different failure
Shailvi presents a lifecycle with definition, logging, transformation, analysis, and sharing. Definition can fail when teams use different meanings for the same feature, choose a definition that is too narrow, or enter incorrect parameters. Logging can miss an entire source, break a mobile pipeline, or use inconsistent storage periods, aggregation rules, or time zones. Transformation can suffer from unclear rules, ambiguous labels, undocumented assumptions, meaningless aggregations, and logic that was tested without real-world cases. Analysis can answer different interpretations of the same question, use the wrong model or formula, or embed bias through data selection.
Clear definitions and lineage reduce ambiguity during transformation
Transformation turns logged data into a usable format by applying business rules and preprocessing. Shailvi recommends data dictionaries and deliberate documentation so people do not silently make different assumptions about labels or rules. She warns that inconsistent intermediate aggregations make it difficult to find where a result went wrong. A team may have to work backwards through every processing step to locate the break. She credits dbt with making lineage easier to inspect by showing what data started as and what happened to it over time. Predictable results require teams to define and agree on the recipe used to transform data.
Sharing data creates quality risks through faulty reporting and misuse
A correct dataset can still produce problems when it is presented badly or used outside its intended purpose. Shailvi shows a chart that does not add up properly and describes dashboards that stop being maintained while people continue to treat them as authoritative. Viewers can also misread overlapping categories or make assumptions the creator did not intend. Once information is published, the team cannot control every later use. It can document what the data is suitable for, what it is not suitable for, and which uses should be avoided. Those caveats reduce unintended downstream interpretations.
Diagnosis starts by checking whether the suspected problem is actually an error
When results do not match a trusted source, or look suspicious, Shailvi recommends checking the basic assumptions first. Two numbers may be correct while representing different populations or definitions. The dataset may also have been built for another use case, or a known pipeline bug may explain the difference. If those checks do not resolve the issue, teams should inspect the lifecycle in reverse order. At each stage, they verify whether the previous phase contains the right data. They then form hypotheses, test them, and repeat until one explanation accounts for the observed problem.
Improving quality requires shared ownership and repeatable engineering practices
Shailvi says data quality is a cross-functional effort because the assumptions behind a problem may come from product, marketing, or design. Useful checks include comparing sources, finding missing or duplicate data, and comparing trends across dimensions. She recommends reusable code modules, accessible documentation for distributed teams, and alerts for important pipeline failures. Teams should maintain and reconcile data, automate where possible, simplify processes, and establish governance. They also need a concrete plan to audit and measure quality issues rather than waiting for each problem to appear in production.
