# MLSecOps is Fundamental to Robust AI Security Posture Management

Sean Morgan, Protect AI | MLOps Podcast | Episode 257 | 42:36
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=hUsuSst_K_M
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/mlsecops-is-fundamental-to-robust-ai-security-posture-management
Published: 2024-08-30
Tags: governance, security

## TL;DR
- Security needs to be integrated throughout the machine learning lifecycle instead of added when a model is ready for production.
- Data sets, foundational models, libraries, containers, model registries, and inference servers all add security risks to the supply chain.
- Teams should assign clear ownership, automate scans in their existing processes, patch exposed services, and inspect model repositories before using artifacts.

## Summary
Sean Morgan explains why machine learning teams need MLSecOps, which applies security practices throughout the AI and machine learning lifecycle. The risks include poisoned training data, untrusted data lineage, impersonated models on Hugging Face, malicious open-source dependencies, vulnerable containers, and exposed inference or model-management servers. Sean argues that security cannot depend on one control or on developers remembering every manual check. Teams should track which models were built from which data, scan code, notebooks, model artifacts, containers, and trained models, and patch known vulnerabilities. He also recommends controls that fit existing workflows, such as vetted artifact stores and proxies between developers and model repositories. Sean is direct about the organizational problem: someone must own the process, and scans should happen automatically in deployment and registration pipelines. The conversation closes with examples involving Ray, MLflow, PyTorch, and Hugging Face.

## Key ideas
### AI teams often trade away security for speed
[03:17](https://www.youtube.com/watch?v=hUsuSst_K_M&t=197s)
Sean says the pace of AI and machine learning innovation pushes organizations to build quickly, and security is often one of the first things dropped when speed is required. Teams then pay for that choice later. He says security is not embedded in MLOps in the same way it is in DevOps. MLSecOps addresses this by integrating security into every phase of model development, rather than trying to attach security to a model just before production. The goal is to make security part of the development process while the model, data, and dependencies are still being assembled.

### Machine learning expands the software supply chain
[04:11](https://www.youtube.com/watch?v=hUsuSst_K_M&t=251s)
Traditional DevOps threats still apply, including vulnerable library dependencies, but machine learning adds data sets and foundational models. These artifacts often come from repositories where it is difficult to verify who created them, how they were made, or whether they contain vulnerabilities. Sean says this increases the attack surface and makes the machine learning lifecycle harder to secure. The concern applies to enterprise teams and hobbyists. Data sets can contain poisoned samples, while model files and dependencies can execute unwanted code when loaded.

### Data lineage lets teams respond after a problem is found
[06:28](https://www.youtube.com/watch?v=hUsuSst_K_M&t=388s)
Sean does not claim that teams can easily prove a large data set contains no poisoned data. Instead, he recommends recording how models are created and which data sets went into them. If a data set later turns out to have a problem, the team should be able to find the models that used it and take appropriate action. He describes this lineage as the first step in a broader set of defenses. Data set scanning remains difficult at scale, so teams also need trusted sources, copyright checks, and records of how data entered the pipeline.

### Security has to cover every stage of the model lifecycle
[09:07](https://www.youtube.com/watch?v=hUsuSst_K_M&t=547s)
Sean describes security checks for model development, model registries, inference containers, inference servers, and red teaming. Model builders should scan static code and notebooks for API keys or other sensitive information before committing them to a repository. When pulling a foundational model, teams should verify the source and understand what code is loaded. Model repositories can contain impersonation attacks, just as package repositories can contain malicious packages. At inference time, exposed API endpoints can allow attackers to compromise a server and move toward users or systems with access to valuable data.

### Model repositories create impersonation and code-execution risks
[12:08](https://www.youtube.com/watch?v=hUsuSst_K_M&t=728s)
Sean gives an example from Hugging Face involving an organization named Facebook Llama, which was created by a security researcher to demonstrate an attack. A user searching for a familiar model or organization could select the wrong artifact. He says attackers can also persuade people to copy code from Stack Overflow, pull the wrong model, and execute arbitrary code. He compares these attacks with name squatting and dependency attacks on npm and PyPI. Model files are binary artifacts intended to execute something, so their origin and contents need to be treated as security concerns.

### Security controls should fit the developer workflow
[20:11](https://www.youtube.com/watch?v=hUsuSst_K_M&t=1211s)
Sean acknowledges that security checks can slow experimental work and frustrate data scientists who are trying to test business value. He recommends meeting developers in the middle with controls that require little or no code change. Examples include vetted artifact stores, approved model collections, and proxies between developers and Hugging Face. These controls can scan or enforce policy without making every developer perform the same manual process. Sean says the right owner may come from machine learning or security, but the organization needs one person or team responsible for the process.

### Exposed infrastructure can turn a convenience feature into an attack path
[29:15](https://www.youtube.com/watch?v=hUsuSst_K_M&t=1755s)
Sean describes Ray's user interface as an example of a risk that teams may overlook. The interface has no authentication by default and can be easy to expose beyond the local environment. Attackers have scanned the internet for these interfaces and used reported vulnerabilities for privilege escalation. The issue became known as Shadow Ray. Sean says Ray's documentation explains how to restrict the interface to localhost, but users do not always read or follow that guidance. The example shows why teams must inspect the full behavior of a framework, including web and user interfaces.

### Patching and automated scanning are basic operating requirements
[34:00](https://www.youtube.com/watch?v=hUsuSst_K_M&t=2040s)
Sean says teams need frequent or continuous patching because public CVEs make vulnerable servers visible to attackers. He also recommends scanning containers with open-source tools such as Trivy, and scanning model artifacts with PickleScan and ModelScan. Giskard and LLM Guard can check trained models for vulnerabilities and other effects, including possible impacts from poisoned training data. For Hugging Face, he recommends reading the scan banners and results on the model page rather than copying an artifact without checking it. Clear ownership and automated scans in model registration and deployment pipelines reduce the chance that everyone assumes someone else handled security.

## Notable quotes
- Sean Morgan: "The core principle behind MLSecOps is that there's no one fix to this problem." (08:48)
- Sean Morgan: "These are binary artifacts that are meant to execute something." (16:40)
- Sean Morgan: "It's not just the security team's job to do that, it's throughout the life cycle of being cognizant of that." (18:16)
- Sean Morgan: "You have to get ahead of security, you don't want to wait until something bad happens." (41:49)

## Tools & references mentioned
- Protect AI
- MLOps Community
- MLSecOps Community
- Hugging Face
- Ray
- Shadow Ray
- MLflow
- PyTorch
- TensorFlow
- PickleScan
- ModelScan
- Giskard
- LLM Guard
- Trivy
- DARPA
- Open Source Security Foundation
- Linux Foundation
- Meta
- SpaceX
- 23andMe

## Who should watch
- You operate machine learning infrastructure and need to understand the security risks in model registries, inference servers, containers, and model repositories.
- Your team is training models on large external data sets and needs a practical way to preserve lineage and respond when a source is later found to be unsafe.
- Security and machine learning teams are struggling to work together without slowing experimentation, and you want examples of controls that can run behind existing workflows.

## Related talks

- [Enterprise Security and Governance MLOps](https://mlopstalks.com/talks/enterprise-security-and-governance-mlops) (Diego Oppenheimer, Algorithmia, 53:24)
- [ML Security: Why should you care?](https://mlopstalks.com/talks/ml-security-why-should-you-care) (Sahbi Chaieb, SAS, 52:20)
- [Enterprises Using MLOps, the Changing LLM Landscape, MLOps Pipelines](https://mlopstalks.com/talks/enterprises-using-mlops-the-changing-llm-landscape-mlops-pipelines) (Chris Van Pelt, Weights & Biases, 47:51)
- [DevOps, Security, and Observability in ML](https://mlopstalks.com/talks/devops-security-and-observability-in-ml) (Luke Marsden, MLOps Consulting, 32:46)
- [Engineering MLOps](https://mlopstalks.com/talks/engineering-mlops) (Emmanuel Raj, TietoEvry, 51:55)