# Explainability in the MLOps Cycle

Dattaraj Rao, Persistent | MLOps Podcast | Episode 138 | 46:08
Hosted by Vishnu Rachakonda

Source: https://www.youtube.com/watch?v=8Io-EITWDJk
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/explainability-in-the-mlops-cycle
Published: 2022-12-27
Tags: drift, experiment-tracking, governance, monitoring

## TL;DR
- Dattaraj Rao focuses on knowledge graphs, responsible AI, MLOps, and privacy-preserving AI because regulated industries need context, explanations, and data governance.
- A knowledge graph can make language-model applications more useful when users need structured answers with concrete evidence, such as tax rules by county and date.
- Deploying a machine learning model follows many of the same patterns as deploying a rule-based system, including packaging intelligence, running inference, monitoring drift, and checking for bias.

## Summary
Dattaraj Rao explains how his work moved from mechanical engineering and rule-based monitoring systems at GE to computer vision, responsible AI, knowledge graphs, and MLOps at Persistent. He describes three areas of current work: knowledge platforms and knowledge graphs, responsible AI with explainability and interpretability, and privacy-preserving AI. Banking and healthcare customers need to justify decisions such as loan approvals and diagnoses, while data governance limits what information they can use. Rao uses tax-document question answering to show where a language model alone can fall short. Extracting entities and relationships into a knowledge graph makes structured queries and evidence possible. He also compares old rule-based monitoring with modern machine learning deployments. The model replaces manually written rules, while packaging, deployment, inference, alerts, drift checks, fairness checks, and governance remain part of the system. He argues that MLOps should focus on reusable patterns rather than tools that may change.

## Key ideas
### Rao's engineering career shaped his interest in practical machine learning systems
[06:40](https://www.youtube.com/watch?v=8Io-EITWDJk&t=400s)
Dattaraj Rao began in mechanical engineering and taught himself computer science because he wanted to build software rather than stay with formula-based systems. At GE, he worked on design software, then remote monitoring for gas turbines. Those systems used domain rules, thresholds, and fuzzy logic to trigger alerts. Later, at GE Transportation, his team built computer vision for cameras mounted on locomotives to inspect railway tracks, work that led to 11 machine learning and computer vision patents. At Persistent, he leads an AI lab working on federated learning, responsible AI, MLOps, knowledge graphs, and next-generation NLP systems.

### Knowledge graphs help regulated applications preserve context and evidence
[13:28](https://www.youtube.com/watch?v=8Io-EITWDJk&t=808s)
Rao says large language models are useful, but they do not always fit banking and healthcare tasks where the system must capture context and explain a recommendation. His team works on knowledge platforms and knowledge graphs that capture entities, relationships, and supporting facts. This lets a system answer why it produced a result. The approach matters when customers need to explain why a loan was approved or rejected, or why a diagnosis was given. Rao is direct that Persistent has interesting offerings in this area, but has not completely solved the problem.

### A tax question-answering system became more useful after its problem was restructured
[17:31](https://www.youtube.com/watch?v=8Io-EITWDJk&t=1051s)
Rao describes a system for searching tax bulletins and answering questions about updates. A language model could find relevant text from documents, but the team could not justify the answers clearly. They changed the task into named entity recognition and relationship extraction. The system identified entities such as a county, a tax percentage, and an imposition type, then stored their relationships in a knowledge graph. This allowed queries such as finding counties with more than six percent sales tax since a given date. The graph provided structure, semantics, and a more inspectable path to the answer.

### Language models and knowledge graphs can be combined according to the task
[21:44](https://www.youtube.com/watch?v=8Io-EITWDJk&t=1304s)
Rao rejects the idea that language models are simply bad for explainability. He says they work well for use cases such as question answering and conversational AI. When the task requires extracting structured knowledge, exploring relationships, and giving concrete evidence, a knowledge graph is a better fit. The graph does not replace language models. Rao says models can help extract entities and relationships, including through transformer-based methods for named entity recognition. His objection is to using a language model blindly for every problem. He expects applications that combine language models with a contextual knowledge graph to grow.

### Machine learning models replace hand-written rules inside a familiar deployment pattern
[23:53](https://www.youtube.com/watch?v=8Io-EITWDJk&t=1433s)
In GE's remote monitoring systems, domain experts wrote rules such as triggering an alert when a turbine's input temperature passed a threshold at a particular operating point. The rules were packaged for a device, sent to the system, and run against live machine data. In a machine learning system, training data replaces the manual rule-writing process. A team builds and validates a model, then sends it to the cloud or an edge device, where it runs continuously and produces alerts. The model can capture patterns from many examples, while the surrounding delivery and inference flow remains similar.

### Fairness, explainability, and drift apply to both rules and learned models
[27:26](https://www.youtube.com/watch?v=8Io-EITWDJk&t=1646s)
Rao says the same operational concerns remain even when the system's intelligence changes from rules to a model. A human-written rule can contain an unwanted bias, and a model can learn one from its data. Both require fairness checks and explainability. Production data also changes. A turbine may physically degrade over time, which can require updating the rules or collecting new data and retraining the model. Rao describes this as the same infrastructure and operational space, with machine learning adding a different way to create the deployed intelligence.

### MLOps guidance should describe patterns that outlast individual tools
[32:00](https://www.youtube.com/watch?v=8Io-EITWDJk&t=1920s)
Rao wrote Keras to Kubernetes to explain how to take a model into production, but he says a future book would focus less on particular tools. The patterns he sees across customer work include drift monitoring, explainability, interpretability, model packaging, validation, and error analysis. He also points to data cataloging, model registries, model governance, model serving, and multi-tenancy. Tools such as Keras, PyTorch, SageMaker, and Vertex AI can implement these patterns, but the pattern should come first. This is why he thinks many of the book's fundamentals remain useful even as the tooling changes.

### Model registries and data catalogs are part of the operating system around models
[38:53](https://www.youtube.com/watch?v=8Io-EITWDJk&t=2333s)
Rao argues that teams need a managed way to publish data versions so data scientists can find and reuse them. A model registry gives the organization an organized catalog for models and supports governance. He also describes a growing interest in treating models as services or marketplace assets, especially in banking and healthcare, where models are trained on valuable organizational data. Multi-tenancy matters when several customers use the same model. Rao says teams should also produce audit reports or model cards for bias and fairness, even though some customers hesitate because an audit may expose an issue.

## Notable quotes
- Dattaraj Rao: "I get scared that one day I'll wake up and I won't be able to learn new things." (07:22)
- Dattaraj Rao: "You use LLMs to extract knowledge, not just blindly use LLM to solve any problem." (22:22)
- Dattaraj Rao: "The Ops part remains the same." (26:27)
- Dattaraj Rao: "Instead of some domain person writing rules, you have given it data and made it learn." (26:29)
- Dattaraj Rao: "We can't be biased if we don't audit it." (43:23)

## Tools & references mentioned
- Persistent
- GE
- General Electric
- Keras to Kubernetes
- Keras
- Kubernetes
- knowledge graphs
- large language models
- federated learning
- confidential computing
- OpenCV
- Java
- Fortran
- O'Reilly
- Algorithmia
- DataRobot
- Chip Huyen
- Machine Learning Design Patterns
- SageMaker
- Vertex AI
- H2O.ai
- PyCaret
- DVC
- Snowflake
- model registry
- feature stores
- vector databases

## Who should watch
- You are building machine learning systems for banking, healthcare, or another regulated setting and need explanations for model decisions.
- Your team is moving from hand-written rules to learned models and wants to understand which deployment and monitoring practices still apply.
- You are choosing MLOps tools and would benefit from a pattern-first view of data catalogs, model registries, drift checks, fairness checks, and model serving.

## Editor's note

Dattaraj Rao says teams need model registries and data catalogs to publish data versions, reuse them, and govern models. ZenML records each pipeline run's steps, inputs, outputs, and code version, so a model can be traced to the data and code that produced it. Its model registry keeps that lineage with the model.

Written by the MLOps Talks editors (the ZenML team), not by the speaker.

## Related talks

- [MLOps vs LLMOps](https://mlopstalks.com/talks/mlops-vs-llmops) (Richa Sachdev, JPMorgan Chase & Willem Pienaar, Feast & Chris Van Pelt, Weights & Biases & Aparna Dhinakaran, Arize AI & Alex Ratner, Snorkel, 34:57)
- [Practical MLOps Part 2](https://mlopstalks.com/talks/practical-mlops-part-2) (Alfredo Deza, Author and Speaker, 1:01:38)
- [Design and Development Principles for LLMOps](https://mlopstalks.com/talks/design-and-development-principles-for-llmops) (Andy McMahon, Barclays Bank, 1:10:18)
- [Enterprises Using MLOps, the Changing LLM Landscape, MLOps Pipelines](https://mlopstalks.com/talks/enterprises-using-mlops-the-changing-llm-landscape-mlops-pipelines) (Chris Van Pelt, Weights & Biases, 47:51)
- [The Current MLOps Landscape](https://mlopstalks.com/talks/the-current-mlops-landscape) (Nathan Benaich, Air Street Capital & Timothy Chen, Essence VC, 58:31)