# Enterprise Security and Governance MLOps

Diego Oppenheimer, Algorithmia | MLOps Coffee Sessions | Episode 45 | 53:24
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=JNZk8diyIuE
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/enterprise-security-and-governance-mlops
Published: 2021-07-02
Tags: governance, reliability, security

## TL;DR
- Operational machine learning systems face more security and governance scrutiny than development environments, especially in regulated industries.
- ML engineers should involve DevSecOps, IT, and security teams early because machine learning adds risks around data, models, and probabilistic behavior.
- Organizations have to balance operational and brand risk against strategic risk, since locking systems down too much can prevent valuable models from reaching production.

## Summary
Diego Oppenheimer explains why security and governance are often the longest part of putting machine learning into production. Operational ML systems run continuously inside an organization's business processes, so they inherit IT controls and face additional scrutiny in financial services, life sciences, and defense. He recommends bringing DevSecOps and security teams into planning early, then building secure defaults into the platform so data scientists retain flexibility without repeating the same controls for every project. The discussion covers data confidentiality, system manipulation, adversarial examples, model and data poisoning, dependency risks, and exposure through external endpoints. Diego frames security decisions as a trade-off between operational risk, brand risk, and the strategic risk of failing to deploy useful systems. He also predicts that ML reliability engineers will emerge from SRE teams that learn how models behave in production, rather than from ML engineers taking on traditional pager duty.

## Key ideas
### Operational ML inherits the highest level of enterprise scrutiny
[03:31](https://www.youtube.com/watch?v=JNZk8diyIuE&t=211s)
Diego separates ML development from ML production, while acknowledging that they remain connected through retraining and automation. Algorithmia focuses on operational systems that run continuously inside an organization's business processes. Those systems must meet the organization's IT, security, and governance standards. Financial services, life sciences, and defense receive especially high scrutiny. Diego says the delay in production is often not building an API around a model. In a bank, security approval alone can take six months, which creates the long tail of many ML projects.

### DevSecOps teams should join ML planning early
[09:07](https://www.youtube.com/watch?v=JNZk8diyIuE&t=547s)
Diego recommends involving an enterprise's DevSecOps team as soon as a team starts planning an operational ML system. Working step by step without the end state in mind can expose new approval gates late in the project. The partnership also works in both directions. Security and operations teams need to understand probabilistic code, where the code may stay the same while the data changes the result. ML engineers need to learn the organization's controls and production constraints. Diego calls this a dual education process rather than an expectation that one ML engineer should know every library, data process, deployment practice, and internal software rule.

### Production platforms should provide flexibility inside secure guardrails
[19:52](https://www.youtube.com/watch?v=JNZk8diyIuE&t=1192s)
Diego describes ML production as a subsystem provided by IT and operations as a service. A mature platform can let data scientists use models and dependencies while controlling where packages come from, checking security, handling authentication, and scanning the operational system. He compares this to a bowling alley with guardrails: users can throw the ball hard, but the platform prevents it from going into the gutter. Earlier workflows often handed an R or MATLAB model to a software engineer for a secure rewrite in C. That rewrite is too slow for frequent retraining and deployment, so teams need pre-hardened environments instead.

### Security automation prevents each new use case from becoming a custom project
[23:07](https://www.youtube.com/watch?v=JNZk8diyIuE&t=1387s)
Diego says automation should include authentication, security, and governance rather than treating them as separate manual tasks. Rebuilding those controls for every model or environment produces uncontrollable technical debt. He gives the example of container images that must be scanned, patched, and replaced regularly. Some banks take systems down weekly to re-pave them, which can require rebuilding the entire Kubernetes environment with zero downtime. A model may be ready after two months, then need another six months of work to automate secure rehydration. Early coordination exposes these requirements before the team reaches the production gate.

### ML security adds risks around data and model behavior
[28:58](https://www.youtube.com/watch?v=JNZk8diyIuE&t=1738s)
Diego groups ML-specific risks around leaking sensitive data and manipulating a system into doing something it should not. Data confidentiality includes PII and sensitive behavioral information, with access controls similar to those used in traditional analytics. System manipulation can affect recommenders, exposed endpoints, and online-learning systems. Attackers may try to influence a recommender for discounts or feed bad examples to shift a model. Diego also discusses adversarial examples, model reverse engineering, and data poisoning. He uses Microsoft's Tay as an example of brand risk after users manipulated a language bot into producing racist content.

### Security investment should follow the actual risk and value
[35:09](https://www.youtube.com/watch?v=JNZk8diyIuE&t=2109s)
Diego does not advise every organization to apply the same security effort on day one. He says teams should make a conscious risk-reward decision based on what the system does and who can reach it. Operational risk can cause financial loss, while brand risk can damage the organization. Locking everything down can create strategic risk by preventing valuable systems from reaching production. An internally used recommender with no external exposure may have a lower downside than a fraud system or a customer-facing model. The important failure is making no decision because the team does not understand what could go wrong.

### Threat modeling should become part of ML system design
[40:05](https://www.youtube.com/watch?v=JNZk8diyIuE&t=2405s)
Diego recommends starting with the business outcome, then working backwards through the workflow to ask how the system could fail and who could be affected. A fraud system has a different risk profile from an internal recommendation tool. This resembles the threat modeling already used for conventional software. Diego describes his experience at Microsoft, where software could not ship without a threat model covering the attack surface and potential attack paths. He expects ML systems to receive the same treatment, with security teams and technical product or engineering staff asking questions together about data, models, access, and exposure.

### ML reliability work will grow out of SRE practice
[46:19](https://www.youtube.com/watch?v=JNZk8diyIuE&t=2779s)
Diego expects SRE teams to become more skilled in machine learning as real-time ML systems become more important. Today, an SRE may wake up for a service outage, discover that the issue is a model or feature problem, and then call a data scientist. An ML reliability engineer would own the operational application and understand enough about models, features, data problems, and automated publishing to respond quickly. Diego thinks this role is more likely to come from SREs learning ML than from ML engineers taking on traditional pager duty. The goal is faster diagnosis and rollback when continuously updated models behave badly.

## Notable quotes
- Diego Oppenheimer: "It's actually getting through the security and governance requirements of an organization before you can certify a system is operational." (07:01)
- Diego Oppenheimer: "There's no such thing as bulletproof in software." (24:47)
- Diego Oppenheimer: "The problem is when you don't make a conscious decision." (35:30)
- Diego Oppenheimer: "You always start with the end result." (38:25)
- Diego Oppenheimer: "I think it's actually SREs becoming more ML conscious." (48:48)

## Tools & references mentioned
- Algorithmia
- MLOps Community
- Kubernetes
- PyPI
- Flask
- Microsoft Tay
- Microsoft
- R
- MATLAB
- C
- AWS
- Infrastructure as Code
- IAM
- SRE
- ML reliability engineer
- Continuous Delivery for Machine Learning
- Google Cloud maturity model for MLOps
- Spotify

## Who should watch
- You are an ML engineer moving a model into an enterprise production environment and need to understand why security approval can dominate the schedule.
- Your DevSecOps or IT team supports ML systems but needs a practical way to divide platform controls from model-specific risks.
- You operate customer-facing or continuously retrained models and want to think through threat modeling, rollback, and incident response.

## Related talks

- [Designing ML Infra for ML & LLM Use Cases](https://mlopstalks.com/talks/designing-ml-infra-for-ml-llm-use-cases) (Amritha Arun Babu, Klaviyo & Abhik Choudhury, IBM, 1:00:18)
- [MLSecOps is Fundamental to Robust AI Security Posture Management](https://mlopstalks.com/talks/mlsecops-is-fundamental-to-robust-ai-security-posture-management) (Sean Morgan, Protect AI, 42:36)
- [Enterprises Using MLOps, the Changing LLM Landscape, MLOps Pipelines](https://mlopstalks.com/talks/enterprises-using-mlops-the-changing-llm-landscape-mlops-pipelines) (Chris Van Pelt, Weights & Biases, 47:51)
- [DevOps, Security, and Observability in ML](https://mlopstalks.com/talks/devops-security-and-observability-in-ml) (Luke Marsden, MLOps Consulting, 32:46)
- [Engineering MLOps](https://mlopstalks.com/talks/engineering-mlops) (Emmanuel Raj, TietoEvry, 51:55)