Lyft lets teams define features with JSON and SQL files in Git, then publishes them to offline and online stores through Airflow.
2
A low barrier to creating features led to 2,400 features, uneven ownership, weak data quality, and reduced trust in reusable data.
3
Lyft improved the platform by tracking ownership, deprecating unused features, exposing metadata, and sampling online reads to discover actual consumers.
Summary
Devon Mittow describes how Lyft's feature service grew into a mature platform used by machine learning models, marketing, driver incentives, fraud detection, and dispatch. Teams define a feature in JSON and SQL, and Airflow publishes the results to an offline warehouse and the DS Features online DynamoDB store. The same ease of use that helped adoption also created problems. Software engineers, analysts, and other users could create features without the data ownership skills needed to manage upstream dependencies, quality checks, alerts, and failures. Lyft had 2,400 features, but only 55% listed an owner. Mittow explains how the team improved ownership coverage through usage logs and deprecation, added dashboards and metadata access, and captured sampled read activity through the DS Features SDK. His main advice is to build governance into the platform early and make deprecation as easy as creation.
Lyft's feature service turns Git configuration into offline and online data
Users define a feature with a JSON file and a SQL file in a Git repository. Those configurations connect to Amundson, Lyft's data discovery platform, and are picked up by Airflow. Airflow builds the required DAGs, executes the feature SQL, and publishes the results to both an offline data warehouse and DS Features, an online DynamoDB table for low-latency key-value fetches. The service supports machine learning models, marketing and communications, driver incentives, fraud detection, and dispatch.
Feature owners carry the business and data responsibilities
The JSON configuration includes ownership, upstream data dependencies, and data quality checks. Feature owners choose the data source, define business logic, configure ownership and alerting, and decide which checks should block publication. The platform team handles online and offline synchronization, orchestration, and service-level expectations for data fetches. This division only works when feature owners understand data contracts, quality failures, Airflow incidents, and on-call work.
Easy feature creation brought many users without the needed data skills
Lyft's platform had 2,400 defined features, and its low barrier to entry meant that software engineers, analysts, and people building marketing audiences could all create them. Those users understood their business needs, but they did not always know how to manage upstream commitments, write useful quality checks, respond when checks failed, or investigate a failed Airflow task. As a result, ownership and data quality became platform problems.
Missing ownership reduced trust and slowed platform changes
When Devon joined the platform, ownership was optional and only 55% of features listed an owner. Even listed owners might lack the tools or skills to manage the feature. Consumers could not tell whether a feature was healthy, arriving on time, or well maintained, so they were less willing to reuse existing data. Missing ownership also made migrations and other platform changes slower because the team had to find and communicate with affected users.
Usage data gave Lyft a way to remove features that no longer mattered
Lyft improved ownership coverage by examining read logs and asking who would care if a feature were turned off. The threat of deprecation encouraged some users to accept ownership. In other cases, nobody depended on the feature, so Lyft turned it off. The team also improved Amundson with ownership and tiering information and built dashboards for troubleshooting and understanding feature health.
Platforms should make governance and deprecation easy
Devon argues that data governance should be built into the platform from the beginning. Ownership information needs to stay aligned with changes in the organization, and deprecating a feature should be as easy as creating one. The platform cannot choose the right data source for every user, but it can expose dataset metadata so users can make informed decisions without extensive data engineering background.
Sampled reads reveal actual consumers when manual metadata becomes stale
Manual ownership and dependency fields become unreliable as people leave, teams change, and organizations are reorganized. Lyft therefore samples about one in every 1,000 fetches from the DS Features online store and records an analytical event with the calling service. Because requests pass through Lyft's SDK and feature registry, the platform can use those reads to see who actually fetches each feature, instead of relying only on declared dependencies.
"We can't fire an analytical event on every read activity, but on a sample of like one 1,000th of fetches to the DS Features online data store we just fire an analytical event that tells us what service is calling that and who's reading it."11:15
Who should watch
You run a feature store where many teams can create data assets, and ownership or quality failures are becoming the platform team's problem.
Your organization relies on manually declared dependencies, but team changes make those records unreliable.
You are deciding how to retire unused features and want practical ideas for combining usage logs, metadata, and deprecation workflows.
