Convoy found that improving model deployment was less urgent than making upstream data dependable and clearly owned.
2
Data contracts treat business data as an API, with software engineers responsible for producing agreed schemas and communicating changes.
3
A shared enterprise data model can reduce duplicated warehouse logic and give machine learning, analytics, and experimentation teams consistent definitions.
Summary
Chad Sanderson describes Convoy's shift from rapid model deployment toward data quality. Convoy uses machine learning for pricing, ETA prediction, offer relevance, and simulation in its freight marketplace. As the number of models grew, upstream production changes began breaking models, while downstream warehouse datasets often encoded business logic inconsistently. Sanderson argues that data should be treated like an API, with explicit ownership, schemas, versioned expectations, and communication about changes. Convoy's data contracts connect software engineers who produce events with data scientists and ML engineers who consume them. The contracts are based on an enterprise map of entities and semantic events, and they can define how data should appear in the warehouse. Sanderson also explains Convoy's feature store and metrics layer, both designed to reduce handoffs between data science and software engineering. He is candid that some human judgment remains necessary, especially when deciding how to model new concepts.
Convoy's business depends on machine learning predictions
Convoy is a two-sided freight marketplace that connects shippers with carriers and tries to reduce empty truck miles. Its models price shipments, predict freight ETAs, rank offer relevance, and simulate future conditions when the company bids on freight months in advance. Sanderson says machine learning and data are the main difference between Convoy and traditional freight brokers. The company therefore needs data quality for models, reporting, analytics, and experimentation, rather than treating it as a narrow machine learning concern.
Model deployment matured before the data foundation did
Convoy built an internal machine learning platform on Kubernetes with a command-line deployment tool, scheduled training jobs, monitoring, shadow testing, and traffic splitting. That made it easy to release and test models. The team then found that upstream production changes could break models without the model owners being told. Data was also not treated like an API, so teams had to depend on informal communication. Downstream training datasets often contained business concepts reconstructed from data that was never designed for machine learning or analytics.
Data quality needs operational measures such as SLAs and repeatable tests
Sanderson puts the existence and performance of a real SLA first. A team may say that data is important and must arrive on time, but that statement is weak if nobody controls or owns the commitment. Convoy also tests whether repeated model runs on the same type of shipment produce excessively different results. A rare Atlanta-to-New-York shipment can act as a repeatable test case. If outputs diverge, the training data may be inconsistent or unusual, so regular test sets can expose that problem.
COVID exposed the limits of long-range predictions and delayed data
Convoy's models performed very badly during COVID because prices were based on conditions several months ahead, while the models could not anticipate an extreme external event. The company changed its business process and repriced with shipper partners instead of relying on the old commitment. Sanderson also says Convoy lacked the right real-time data and depended too much on lagging indicators. The experience made faster updates and better data a central part of the later quality effort.
Data contracts make production data an explicit API
A data contract is an agreement between a producer and a consumer about the data needed and the shape in which it should arrive. Convoy implements contracts with Apache Kafka and an SDK similar to protobuf. Engineers produce data for broad consumption while considering machine learning requirements. The contract makes ownership clearer: software engineers are responsible for the quality and communication of service data, while data scientists and ML engineers consume a defined interface instead of depending on implementation details.
An enterprise data model gives entities and events shared meaning
Convoy mapped the major entities and events in its system, including shipments, offers, RFPs, emails, trucks, and carriers. The model contains roughly 120 to 150 entities, with each entity ideally tied to a service and production table. Sanderson distinguishes semantic events, such as a shipment going off hold or an offer being confirmed, from simple CRUD updates. Data scientists helped define the schemas they needed, and engineers then implemented the most important contracts in production. The rollout covered about 30 models at the time of the talk.
A feature platform can remove a slow software-engineering handoff
Convoy's feature platform lets data scientists query Kafka topics with SQL or Python, create feature sets, and receive an automatically generated endpoint through an SDK. This avoids asking a software engineer to reimplement a feature in TypeScript and then sending it back for validation. Sanderson says that translation was slow and could produce small differences between training and serving logic. He also acknowledges the cost of maintaining an internal system, especially while the company is investing heavily in data quality.
The metrics layer applies shared definitions to experiments and analytics
Convoy stores metric definitions in a Git repository as SQL plus YAML metadata, including how a metric is calculated, whether an increase is good, and who owns it. For experiments, the system joins treatment and control entity IDs to those definitions, computes the metrics, aggregates the results, and runs a statistical test. Sanderson sees the metrics layer and feature store as the same type of surface for different serving needs. He describes plans to expose metric and dimension combinations through an API.
Automation should stop when domain judgment is needed
Sanderson says Convoy optimizes for efficiency, completeness, accuracy, and quality rather than automation by itself. A person may need to decide how a requested concept should be modeled. For example, a data scientist asking for a user-phone entity might need a join, a combination of existing events, or a new service. An intermediary can judge which design is scalable and fits the existing business model. He also changed his view of data scientists, describing their work as focused on delivering business value rather than maintaining production code.
