# What is AI Quality?

Mohamed Elgendy, Kolena Inc. | MLOps Podcast | Episode 229 | 45:58
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=-Jdmq4DiOew
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/what-is-ai-quality
Published: 2024-05-03
Tags: evals, governance, guardrails, testing

## TL;DR
- AI quality depends on the domain, application, intended function, and risks that come with deployment.
- A useful quality standard defines functional requirements, associated risks, testing methods, metrics, and acceptable thresholds.
- Testing should use scenario-based coverage and become part of the engineering process, so teams can find failures, prevent regressions, and improve models systematically.

## Summary
Mohamed Elgendy defines AI quality through two questions: does the product perform the function it was built for, and are its risks understood and controlled? He compares a $1 pen with a $100 pen to explain why quality depends on intended use. A gaming chatbot, a banking support bot, and a financial advice system need different requirements, including different limits for latency, accuracy, hallucinations, and jailbreaks. Mohamed proposes standards organized by domain and application rather than one universal checklist. Each standard should describe functional specifications, risks, testing methods, metrics, and acceptable thresholds. He and Demetrios Brinkmann are using the AI Quality Conference as the start of a nine-month program involving builders, regulators, and infrastructure companies. The conversation also covers model cards, test coverage, regression testing, diverse representation, and the value of involving subject-matter experts. Mohamed argues that systematic testing can move machine learning from repeated experimentation toward an engineering process.

## Key ideas
### AI quality depends on what the product is meant to do
[03:25](https://www.youtube.com/watch?v=-Jdmq4DiOew&t=205s)
Mohamed says testing alone does not provide quality. Quality depends on the application, the domain, and the problem being solved. His pen example makes the distinction concrete: a $1 pen can meet its quality bar even though a $100 pen would fail that same bar. The cheaper pen is disposable and is not expected to work for years or under severe weather. An AI product needs the same context. A gaming chatbot can have different requirements from a banking chatbot, while a financial advice system must meet stricter rules around precision and prohibited advice.

### A quality product performs as intended and has no unknown risks
[07:27](https://www.youtube.com/watch?v=-Jdmq4DiOew&t=447s)
Mohamed reduces AI quality to two main sections. The first is the product's intended specifications, including the use cases and functions it should support. The second is the risks associated with the technology and the industry. Hallucinations and jailbreaks are examples for generative AI. A chatbot may confidently answer a question outside its retrieval system, or a user may push it outside guardrails designed to prevent financial advice. Quality means benchmarking the product against both its intended function and these risks.

### Quality requirements include the surrounding product and deployment
[09:11](https://www.youtube.com/watch?v=-Jdmq4DiOew&t=551s)
Demetrios points out that quality involves more than the machine-learning model. Mohamed agrees that software quality assurance and the product itself matter too. Deployment conditions can add requirements. A model running on edge hardware may need to respond within half a second, or 500 milliseconds, while a cloud application may tolerate several seconds. The acceptable latency depends on the hardware, the application, and whether the use case is time-sensitive.

### Standards should branch by domain and application
[10:53](https://www.youtube.com/watch?v=-Jdmq4DiOew&t=653s)
Mohamed does not expect one standard for every individual use case. He proposes a hierarchy that starts with a broad generative AI standard, then narrows into domains such as banking or financial services, and then into applications such as customer support or financial advice. This structure preserves common requirements while allowing domain-specific risks and limits. A customer support bot might accept an answer after five, six, or seven seconds, while another application may need a different latency threshold and stricter requirements for correctness.

### Builders, regulators, and infrastructure teams need to write standards together
[15:15](https://www.youtube.com/watch?v=-Jdmq4DiOew&t=915s)
Mohamed describes the AI Quality Conference as the start of a nine-month program. The work brings together AI builders, regulatory bodies, and MLOps or infrastructure companies. Builders understand the products and risks. Regulators focus on protecting people. Infrastructure teams can make requirements automatable without adding a large governance burden to data scientists. Mohamed divides the program into discovery of target domains, definition of risks and application guidelines, and definition of processes, tools, and infrastructure that teams can apply.

### A standard needs a shared structure for requirements and testing
[25:31](https://www.youtube.com/watch?v=-Jdmq4DiOew&t=1531s)
Mohamed says the program must avoid producing another list of disconnected standards. His proposed layout contains functional specifications, potential risks, a testing methodology, metrics, and acceptable guidelines or thresholds. Different risks require different methods. Testing personally identifiable information leaks is different from testing correctness or factualness. Agreeing on this structure gives providers and teams a common language, even when they still disagree about specific metrics or acceptable ranges.

### Scenario-based test coverage is more useful than one aggregate score
[30:45](https://www.youtube.com/watch?v=-Jdmq4DiOew&t=1845s)
Mohamed says a benchmark score such as 90% does not explain which situations were tested or whether the result is good enough. Teams should stratify prompts and data into test cases that reflect functional requirements and risks. He uses diversity and factualness to explain why tests may need to check several properties together. Test cases should include negative examples and verify that requirements hold across relevant situations. This approach also makes it possible to track regressions as models and applications change.

### Testing can turn machine learning into a repeatable engineering process
[28:09](https://www.youtube.com/watch?v=-Jdmq4DiOew&t=1689s)
Mohamed says teams often believe they already test their systems, but regular automated testing is still missing. Scenario-based tests show exactly where a model fails instead of leaving improvement to trial and error. They can also reduce wasted upstream work by helping teams label only the data they need. Over time, teams may grow from a small initial set of test cases to hundreds, building a shared understanding of their domain. Product managers, customers, leaders, and subject-matter experts can review the cases and identify edge conditions that technical teams missed.

## Notable quotes
- Mohamed Elgendy: "We are providing testing to our customers, we're not providing quality." (04:49)
- Mohamed Elgendy: "The quality means that my product performs as intended, safely, that no risks are associated with it that we don't know about." (08:09)
- Mohamed Elgendy: "The builders, the AI builders, regulatory bodies, and the MLOps or the tooling and the infrastructure layers." (15:56)
- Mohamed Elgendy: "This is the gold standard of the gold standard, if you will." (26:35)
- Mohamed Elgendy: "It turns the experimentation nature of machine learning into an engineering discipline where you find a bug and test it, and so on, and you improve it and test it." (28:49)

## Tools & references mentioned
- Kolena
- NIST
- National Institute of Standards and Technology
- AI Quality Conference
- MLOps Community
- Gemini
- GPT-3
- GPT-4
- OpenAI
- model cards
- RAG
- Washington Post
- The Information
- Cruise
- Stanford
- EU Act
- N regulations

## Who should watch
- You build or operate AI products and need to decide what quality means for a specific domain or application.
- Your team has benchmark scores or model cards but cannot tell which scenarios were covered or what an acceptable result is.
- You work on AI regulation, MLOps infrastructure, or testing and want a shared process that includes builders and subject-matter experts.

## Related talks

- [Data Quality = Quality AI](https://mlopstalks.com/talks/data-quality-quality-ai) (Samuel Partee, Redis & Chad Sanderson, Gable & Joe Reis, Ternary Data & Maria Zhang, Proactive AI Lab Inc & Pushkar Garg, Clari Inc, 27:15)
- [Managing Data for Effective GenAI Application](https://mlopstalks.com/talks/managing-data-for-effective-genai-application) (Anu Arora & Anass Bensrhir, QuantumBlack AI by McKinsey, 51:01)
- [AI Operations Without Fundamental Engineering Discipline](https://mlopstalks.com/talks/ai-operations-without-fundamental-engineering-discipline) (Nikhil Suresh, Hermit Tech, 49:28)
- [Scaling AI in Production](https://mlopstalks.com/talks/scaling-ai-in-production) (Srivatsan Srinivasan, AIEngineering, 51:56)
- [Build a Culture of ML Testing and Model Quality](https://mlopstalks.com/talks/build-a-culture-of-ml-testing-and-model-quality) (Mohamed Elgendy, Kolena, 51:14)