# Just Fetch the Data and then...

David Bayliss, LexisNexis Risk Solutions | MLOps Coffee Sessions | Episode 110 | 51:56
Hosted by Vishnu Rachakonda

Source: https://www.youtube.com/watch?v=m9EuvBomb3A
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/just-fetch-the-data-and-then
Published: 2022-07-29
Tags: data-engineering, governance, privacy, search

## TL;DR
- David Bayliss says data systems should be built around collecting, integrating, and understanding durable data rather than around today's changing use case.
- LexisNexis Risk Solutions separates creative data science from tightly controlled production work through sandboxed environments, metadata, permissions, and a domain-specific language.
- The TARDIS abstraction layer lets LexisNexis change databases, hardware, and third-party libraries without rewriting the data algorithms that depend on them.

## Summary
David Bayliss describes the data infrastructure behind LexisNexis Risk Solutions, where information is used for identity checks, credit, insurance, fraud detection, background checks, and other decisions that affect people. He says the company ignores narrow product requests and builds around data that will remain useful as use cases change. The scale is large, with billions of records arriving daily and highly sensitive information governed by different legal permissions. Data scientists experiment inside controlled environments, while production systems use metadata to record lineage, explain decisions, enforce access, and test model coverage. Bayliss also explains TARDIS, an abstraction layer that separates data algorithms from particular databases, hardware, and third-party tools. This lets teams move workloads across implementations and preserve systems over decades. His approach gives specialists freedom during experimentation while keeping data, models, and deployment under strict control.

## Key ideas
### Durable data matters more than the current use case
[07:49](https://www.youtube.com/watch?v=m9EuvBomb3A&t=469s)
David Bayliss says the simplest way to handle many use cases is to ignore them at first. Product managers change what they want, while the incoming data remains. His method is to collect and integrate the data, establish what it is saying, and build a model of the world. Once that foundation exists, different teams can ask new questions without rebuilding the system around every request. He describes this as focusing on the data rather than on a product manager's temporary description of the problem.

### Data integrity and legal permissions are part of the same system
[10:27](https://www.youtube.com/watch?v=m9EuvBomb3A&t=627s)
Bayliss describes his role as a split responsibility. Hundreds of data scientists work in specialist fields and solve customer problems, while his team keeps the underlying data integrated and usable. The data is highly sensitive, so access depends on what a customer is legally allowed to receive. If two banks have different permissions, the same person may produce different answers for each bank. Bayliss says the system must enforce those differences rather than simply collect every possible piece of information.

### Production systems need explainable lineage
[15:31](https://www.youtube.com/watch?v=m9EuvBomb3A&t=931s)
LexisNexis cannot rely on black boxes for decisions that affect people's lives. Bayliss says algorithms are encoded in a domain-specific language so the company can trace a result from the model back through the data used to produce it. Some algorithms must also be registered with a state or customer. The same metadata tracks production queries and controls what data reaches an individual data scientist. Permission information filters the data before it appears in the scientist's working environment.

### A customer model is tested against the past and kept inside an iron box
[20:53](https://www.youtube.com/watch?v=m9EuvBomb3A&t=1253s)
Bayliss explains an insurance example in which a data scientist receives customer data with known outcomes, cleans and links it, then recreates what would have been available at an earlier date. The system can travel back through ten years of data, which the team calls the TARDIS. Scientists can use the tools they prefer during experimentation, but the work stays inside a tightly controlled environment. When they have a model, they encode it in Bayliss's meta-language so it can move into production with its data use and regulatory constraints known.

### TARDIS separates algorithms from changing infrastructure
[24:21](https://www.youtube.com/watch?v=m9EuvBomb3A&t=1461s)
The TARDIS is an insulation layer between data algorithms and their implementation. Bayliss says LexisNexis has products that predate him and expects to support systems for decades, while acquired startups often arrive needing a technology refresh because their third-party tools have become unsupported. The abstraction layer lets the same code work with SQL, non-SQL systems, in-memory databases, and caches. Teams can test different library sets and deploy different implementations without changing the underlying data logic.

### Specialists can make shared low-level code faster
[29:57](https://www.youtube.com/watch?v=m9EuvBomb3A&t=1797s)
Bayliss compares the data science virtual machine with a JVM. It provides operations for fetching, sending, reading, writing, and indexing records, while hiding the underlying implementation. The metadata and DSL are compiled into in-memory structures at application load. This allows Bayliss to work locally on a laptop and then send the same work to the cloud for much larger data. He argues that a small group of specialists writing low-level access routines can produce better results than thousands of data users duplicating that work.

### The iron box is a controlled cloud sandbox
[35:17](https://www.youtube.com/watch?v=m9EuvBomb3A&t=2117s)
Bayliss says the experimental environment now runs in the cloud and is configured so data can enter and leave only through defined paths. Users cannot change many subscription attributes, and scratch space is scrubbed when they finish. The environment gives scientists access to the data they are allowed to see while preventing that data from leaving. Once a model is ready, the metadata and production process bring it back under regulation.

### Synthetic tests expose gaps in model training data
[38:05](https://www.youtube.com/watch?v=m9EuvBomb3A&t=2285s)
Because model decision trees are stored as metadata, Bayliss's team can generate fake customer data designed to exercise the tree's decision points. This provides a coverage measure that ordinary model training does not provide. The generated cases can reveal parts of the tree that the original training data never reached. He also describes privacy controls that replace personally identifiable information with an internal identifier and use differential privacy techniques when apparently harmless attributes could identify someone.

### Search became faster when the problem was narrowed to identity matching
[41:41](https://www.youtube.com/watch?v=m9EuvBomb3A&t=2501s)
Bayliss challenged the assumption that search results cannot be precomputed because users can enter infinitely many search terms. He separated broad search from linking and matching, where the customer is looking for a particular entity. With a known universe of people, his team can precompute the valid ways each person can be searched for. This removes irrelevant responses and reduces the work needed at query time. Bayliss says the change brought latency down to about three milliseconds while handling 40,000 to 50,000 transactions per second.

## Notable quotes
- David Bayliss: "What doesn't change is the data that's coming in." (08:12)
- David Bayliss: "My job is to be the most boring person on the planet and to get the answer right when it really matters." (10:31)
- David Bayliss: "They can do what they want in there, they can produce an algorithm, but they're in an iron box, it stays there." (22:55)
- David Bayliss: "I specifically don't care what it is they're using to do the experimentation." (24:19)
- David Bayliss: "We have a data science virtual machine." (33:08)

## Tools & references mentioned
- LexisNexis Risk Solutions
- Hadoop
- TARDIS
- Doctor Who
- JDBC
- SQL Server
- Python
- SAS
- Amazon
- Java
- Apache Apex
- MySQL
- Oracle
- IBM 370
- differential privacy
- Hairy Kneecap Theory
- Jeremy Howard

## Who should watch
- You are building data systems whose product requirements will change faster than the underlying data.
- Your team needs to let data scientists experiment while enforcing privacy, audit, lineage, and production controls.
- You are deciding whether an abstraction layer is worth the cost of supporting systems and dependencies over many years.

## Editor's note

David Bayliss says TARDIS keeps data algorithms independent of databases, hardware, and third-party tools, so long-lived systems do not need to be rewritten when their infrastructure changes. ZenML uses configurable stacks so the same Python pipeline can run on a laptop, Kubernetes, Airflow, Kubeflow, or a cloud provider's services. Each run records its steps, inputs, outputs, and code version.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [Deep in the Heart of Data](https://mlopstalks.com/talks/deep-in-the-heart-of-data) (Carl Steinbach, LinkedIn, 55:27)
- [Mid-Scale Production Feature Engineering](https://mlopstalks.com/talks/mid-scale-production-feature-engineering) (Dr. Venkata Pingali, Scribble Data, 1:01:35)
- [Data Engineering + ML + Software Engineering](https://mlopstalks.com/talks/data-engineering-ml-software-engineering) (Satish Chandra Gupta, Slang Labs, 57:05)
- [Data Engineering for ML](https://mlopstalks.com/talks/data-engineering-for-ml) (Chad Sanderson, Convoy, 57:54)
- [How Data Platforms Affect ML & AI](https://mlopstalks.com/talks/how-data-platforms-affect-ml-ai) (Jake Watson, The Oakland Group, 39:12)