# Agile AI Ethics: Balancing Short Term Value with Long Term Ethical Outcomes

Pamela Jasper, Jasper Consulting Inc | MLOps Meetup | Episode 51 | 1:06:51
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=WwwV3VOxdcE
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/agile-ai-ethics-balancing-short-term-value-with-long-term-ethical-outcomes
Published: 2021-02-15
Tags: governance, human-in-the-loop

## TL;DR
- Pamela Jasper argues that ethics should be built into machine learning work through user stories, acceptance criteria, data practices, testing, and governance.
- Agile work can start with minimally viable data or models, then use feedback to find bias, privacy problems, and other risks before the system grows.
- The level of ethical review depends on the use case, with high-risk systems needing stricter thresholds, documented limits, diverse teams, and human governance.

## Summary
Pamela Jasper explains how machine learning teams can add ethical work without treating it as a separate stage that arrives after development. She proposes adding ethical acceptance criteria to individual user stories, so privacy, bias, explainability, and other concerns are attached to specific outcomes. Teams can start with minimally viable data or models, gather feedback, and expand their work through repeated iterations. That approach can expose problems earlier, although it does not mean that a high-risk product can remain incomplete. The required level of accuracy and fairness depends on the use case. Jasper also discusses Kanban as an alternative to fixed Scrum sprints, since some machine learning work does not fit a two-week cycle. She sees some ethical checks becoming automated tasks, while full audits still require people to examine the whole technical and governance program. Her FAIR framework draws on banking model risk management and SR 11-7.

## Key ideas
### Agile is a set of values that machine learning teams can apply without adopting full Scrum
[05:55](https://www.youtube.com/watch?v=WwwV3VOxdcE&t=355s)
Jasper defines agile through the 2001 Agile Manifesto rather than through one mandatory process. Scrum and Kanban are frameworks built around agile ideas, but a team can use a hybrid approach and still work in an agile way. She points to valuing working software over extensive documentation and individual contributions over long requirements exercises. Machine learning teams already run experiments, change data or model parameters, and use feedback to guide the next attempt. In her view, this iterative work fits agile principles even when the team does not use daily stand-ups, velocity points, or fixed two-week sprints.

### Ethical acceptance criteria belong inside individual machine learning user stories
[10:30](https://www.youtube.com/watch?v=WwwV3VOxdcE&t=630s)
Jasper maps ethics onto the machine learning lifecycle, from problem definition and data preparation through feature engineering, model design, testing, approval, and deployment. Her concrete proposal is to add ethical acceptance criteria alongside ordinary performance criteria in a user story. A story might require that an outcome does not violate a person's data privacy when personal data is used. That requirement travels with every use of the story through the pipeline. The FAIR framework also places model tiering at a higher lifecycle level, because models have different risks and do not all need the same explainability, model cards, or data sheets.

### Starting with minimally viable data can expose ethical problems earlier
[21:50](https://www.youtube.com/watch?v=WwwV3VOxdcE&t=1310s)
Jasper adapts the agile idea of a minimally viable product to machine learning through minimally viable data and minimally viable models. She uses a fruit-classification example: a team could begin with apples and oranges, define the classification task, test the model, and add more categories later. Working with a smaller initial dataset gives the team feedback sooner. That feedback can reveal bias, privacy issues, and technical problems before the team has invested in a much larger system. The final model still needs to satisfy its complete requirements. Starting small is an iteration strategy, not permission to deploy an unfinished high-risk product.

### Ethical thresholds depend on the harm and purpose of the use case
[25:50](https://www.youtube.com/watch?v=WwwV3VOxdcE&t=1550s)
Jasper says machine learning teams often accept that models will not be 100 percent accurate, while some ethical risks allow far less room for error. Facial recognition used in criminal justice or predictive policing needs performance as close to 100 percent as possible because the potential harm is high. Other contexts use different measures. For hiring, she discusses concerns about facial analysis even when the tool performs accurately, since disability or expression can affect its output. She also cites the EEOC four-fifths rule, where an 80 percent threshold can be used to assess disparate impact. Teams should define thresholds and product limitations at the user-story level.

### Kanban can fit machine learning work better than fixed Scrum sprints
[33:42](https://www.youtube.com/watch?v=WwwV3VOxdcE&t=2022s)
Jasper distinguishes Scrum's roughly two- to four-week sprints from Kanban, which does not require time-boxed sprints. A team might spend several weeks moving through a data or pipeline phase, adding variables beyond an initial minimally viable dataset. A Kanban board can represent the machine learning phases and move stories through them as work finishes. This lets a team remain agile without forcing research, data preparation, and model work into an arbitrary two-week window. Jasper presents this as one way to adapt agile practice to the shape of machine learning operations.

### Ethics can be added to agile boards as recurring tasks and checks
[37:17](https://www.youtube.com/watch?v=WwwV3VOxdcE&t=2237s)
Jasper describes ethical tools and techniques as work that belongs inside the team's existing agile process. For a high-harm use case involving lending, housing, education, or hiring, the team could use a checklist during the relevant phase to check for discrimination against protected groups. Data cleaning may include checking for bias, while later stories may cover explainable AI, model sheets, data sheets, or algorithm audits. The team identifies the ethical risk earlier, at the design or epic level, then records specific requirements in stories. Ethics becomes part of the team's ordinary work rather than a review that appears only after deployment.

### Automation can handle narrow ethical tasks, while full audits need human governance
[55:45](https://www.youtube.com/watch?v=WwwV3VOxdcE&t=3345s)
Jasper expects some specific checks to become reusable or automated. If one project identifies correlations between variables or finds that data is heavily weighted toward a protected class, the next project can repeat that check automatically. She draws a boundary around full audits. An audit covers the whole lifecycle and the governance program, including policies, procedures, model tiers, and the structure of the governance board. Jasper says an organization needs an ethical board that includes an AI ethicist, technical staff, and people from across the organization. Automation can assist with tasks, but it cannot replace deliberate human review of the wider program.

### Team composition and model grading can make diversity an explicit responsibility
[43:38](https://www.youtube.com/watch?v=WwwV3VOxdcE&t=2618s)
Jasper connects agile's preference for flat, collaborative teams with the need for diverse perspectives in high-harm machine learning work. She recommends seeking racial and gender diversity across the team rather than allowing the highest-paid or most senior person to dominate decisions. She acknowledges that startups may have fewer hiring options, then suggests using diverse consulting firms and open competitive bids. Her model risk example adds a formal incentive: a validation committee can grade a model, and team diversity can be one criterion in that grade. That makes team composition part of the model's documented evaluation rather than an informal aspiration.

## Notable quotes
- Pamela Jasper: "What I'm suggesting is that ethics become baked in almost to everything, to the mindset of a machine learning engineer and of a data scientist and of the machine learning operational processes or MLOps." (10:30)
- Pamela Jasper: "What I'm proposing is that in addition to acceptance criteria in terms of whether or not it meets certain performance measurements, we add an ethical acceptance criteria." (12:54)
- Pamela Jasper: "Doing it in an agile way means that you're not using the entire data set of available data variables, you're starting small." (23:45)
- Pamela Jasper: "I think that you need people who are looking at this deliberately and that's why I think that ethics needs to be embedded at user stories, at your minimally viable data, at your minimally viable models, during your model testing." (59:55)

## Tools & references mentioned
- FAIR, Framework for AI Risk
- SR 11-7
- Agile Manifesto
- Scrum
- Kanban
- MLOps
- Jira
- Trello
- Asana
- Weapons of Math Destruction
- EEOC
- Shea Brown
- Amazon
- robotic process automation
- capability maturity model

## Who should watch
- You are building machine learning systems and want a practical way to attach privacy, bias, or explainability checks to development work.
- Your team is debating whether Scrum, Kanban, or another agile process fits experimentation and data preparation.
- You are setting up AI governance and need to understand where automation stops and human review begins.

## Related talks

- [What Does Best in Class AI/ML Governance Look Like in Fin Services?](https://mlopstalks.com/talks/what-does-best-in-class-ai-ml-governance-look-like-in-fin-services) (Charles Radclyffe, Technology Governance and ESG Specialist, AI Ethics, 1:03:42)
- [AI Operations Without Fundamental Engineering Discipline](https://mlopstalks.com/talks/ai-operations-without-fundamental-engineering-discipline) (Nikhil Suresh, Hermit Tech, 49:28)
- [Extending AI: From Industry to Innovation](https://mlopstalks.com/talks/extending-ai-from-industry-to-innovation) (Sophia Rowland & David Weik, SAS, 1:01:37)
- [Building Trust Through Technology: Responsible AI in Practice](https://mlopstalks.com/talks/building-trust-through-technology-responsible-ai-in-practice) (Allegra Guinan, Lumiera, 47:09)
- [GenAI: An Unreliable Information Store](https://mlopstalks.com/talks/genai-an-unreliable-information-store) (Noble Ackerson, Venta, 27:55)