GoCardless uses data contracts to make data quality the responsibility of the teams that produce the data.
2
A data contract defines a dataset's schema, documentation, ownership, exposed interfaces, privacy metadata, and controls.
3
The main value of data contracts is the collaboration they create between data producers and consumers, rather than any particular implementation.
Summary
Andrew Jones describes how GoCardless moved away from copying upstream PostgreSQL databases into a central BigQuery warehouse through change data capture. That approach let upstream schema changes break downstream pipelines, while consumers had to reconstruct undocumented business meaning and add increasingly complex transformation logic. Data scientists consequently spent time repairing and reshaping data instead of building features and models. GoCardless's data contracts define what a service exposes, including its schema, descriptions, ownership, privacy classification, delivery interface, and operational controls. Contracts also give data consumers a migration path when data changes. The implementation creates isolated Google Cloud projects and resources owned by producer teams, while central catalogues and BigQuery's cross-project querying preserve discovery and analysis. Andrew is clear that the contract format matters less than agreeing on the problem, bringing producers and consumers together, and changing the culture around data ownership.
GoCardless depends on data for internal risk models and customer products
GoCardless uses machine learning in several important parts of its payments business. Andrew names an internal fraud model, Success+, which retries failed payments at a time when they are more likely to succeed, and Protect+, which helps protect merchants from customers who default. These models affect GoCardless itself and products sold to merchants. Their performance depends on having data that the teams can trust and understand. Andrew frames the data problem in practical terms: a model cannot be effective when its inputs are unreliable, unavailable, or poorly understood. Data quality therefore affects both internal financial risk work and the machine learning features offered to customers.
Copying operational databases into a warehouse makes downstream consumers fragile
The older GoCardless architecture copied upstream PostgreSQL databases into BigQuery through a change data capture service. Airflow and dbt then transformed and joined that data for analytics and model features. When an upstream schema changed, downstream ETL and transformations could fail without advance warning. Consumers had to investigate what changed, who changed it, and how to work around the new shape. They often added conditional logic or other patches, which made pipelines more complex and less reliable. The source systems were designed to run transactional services, so their data was not designed for general consumption by analysts, engineers, or data scientists.
Good data needs documentation, discovery, versioning, and an owner
Andrew defines good quality data in terms of how people consume it. Consumers should be able to discover the data and understand it without relying on institutional knowledge or asking for explanations in Slack. Data should be versioned so that changes are visible, especially breaking changes, and consumers should have a migration path to a new version. It also needs to be reliable enough for people to build and deploy models with confidence. Responsibility belongs with the producers because they know how their data has evolved and what they intend to change. This moves quality work closer to the source instead of adding more repair logic downstream.
A data contract gives shared data an API-like boundary
Andrew compares a data contract with an API between software services. A service would not normally give another service direct access to its internal database because internal changes would break consumers. An API provides a documented and relatively stable interface that can evolve through new versions and a migration period. Data should have the same boundary. A contract can define the schema, documentation, exposed data, support expectations, and other guarantees between producers and consumers. The comparison also changes the conversation inside an organisation. Data producers need to decide deliberately what they offer, while consumers can understand what they are allowed to depend on.
GoCardless contracts describe both the data and how consumers receive it
At GoCardless, a contract belongs to a service and has an explicit owner. It contains field definitions with names, types, and descriptions, along with classifications such as whether a field is a direct or indirect identifier. Andrew says this privacy metadata is required when data is defined because adding it later becomes unwanted busywork. The contract also specifies the interfaces used to expose the data, such as a Pub/Sub topic or a BigQuery table, and includes controls around access and compliance. The format is implemented with Jsonnet because it fits GoCardless's infrastructure, although Andrew does not present that format as a general requirement.
Producer-owned resources support autonomy while central catalogues preserve discovery
When a contract is merged into the main code repository, GoCardless can deploy matching Google Cloud resources, including a Pub/Sub topic, a BigQuery table, and Kubernetes services. These resources live in the producer team's own Google Cloud project. The team owns the table, access decisions, and supporting services instead of waiting for a central data team. Data remains discoverable through central catalogues, metrics, dashboards, and lineage information. BigQuery also allows queries across projects, so isolation does not prevent consumers from joining data. The architecture replaces a single central warehouse fed by CDC with multiple producer-owned data interfaces that consumers can still find and combine.
The value of contracts comes from producer-consumer collaboration
Andrew says the implementation is less important than agreeing on the problem. GoCardless had producers changing internal models without knowing that those models fed reports or fraud features, while consumers had little confidence in the resulting data. The contract process makes those dependencies visible and gives both sides a reason to work together. Andrew found that product teams were willing to participate once they understood the failures caused by the old CDC approach. He also describes this as a new skill for data scientists and BI engineers, who had become accustomed to repairing whatever data they received. They now have to ask producers for data that meets their needs.
Data contracts are part of a culture change around ownership
GoCardless uses data contracts to ask engineering teams to design data for consumption instead of treating warehouse exports as a byproduct of operating a service. Teams responsible for payment states and balances are expected to provide information that other teams can use for analytics and machine learning. The contracts encourage producers and consumers to discuss requirements directly, and they make ownership and responsibility explicit. Andrew describes the aim as becoming a data-driven organisation that gets business value from its data. The contract is therefore a mechanism for changing how teams treat data, alongside the technical definitions and deployment tooling.
