# AI in Production 2025 | Keynote

 | AI in Production 2025 | 2:07:08
Hosted by Demetrios Brinkmann

Source: https://www.youtube.com/watch?v=ioCIpbAKl_M
Channel: MLOps Community, now AAIF Live (https://www.youtube.com/@AAIFLive-x1r). Summarised by MLOps Talks.
Page: https://mlopstalks.com/talks/ai-in-production-2025-keynote
Published: 2025-03-13
Tags: evals, guardrails, latency

## TL;DR
- Guardrails are machine learning systems, so teams should measure their accuracy, latency, and coverage instead of treating them as fixed safety features.
- A guardrail can use rules, small fine-tuned models, or a second LLM call, with simpler checks placed before slower and more flexible checks.
- Guardrails should be applied at the AI Gateway for organization-wide protections and at the application level for risks specific to a product or industry.

## Summary
The keynote explains how guardrails fit into AI applications and why they need the same testing discipline as other machine learning systems. Guardrails can inspect inputs, validate actions, check generated responses, or sit around a shared AI Gateway. Their implementations range from regular expressions and keyword filters to small fine-tuned models and secondary LLM calls. The speaker presents benchmark results for personally identifiable information detection and jailbreak prevention. Microsoft Presidio is fast, with latency around 15 milliseconds, but its reported F1 score for the tested PII task was low. More sophisticated systems improved that score while remaining within a latency range the speaker considered barely perceptible to users. For jailbreak detection, different systems made different trade-offs between true positives, false positives, latency, and cost. The practical advice is to benchmark every guardrail model against the risks that matter for a particular application, then choose the appropriate combination of checks.

## Key ideas
### Guardrails can be placed throughout an AI application
[00:00](https://www.youtube.com/watch?v=ioCIpbAKl_M&t=0s)
The speaker describes guardrails as checks around the input and output sides of an AI application. In a customer-support agent, an incoming query can pass through an input classification guardrail, an intended action can pass through an action validation guardrail, and the generated response can pass through a response validation guardrail. This pattern applies to co-programming agents, multi-turn chatbots, and copilots. As applications become more sophisticated, teams tend to add guardrails at more points in the workflow rather than relying on one check at the end.

### A shared AI Gateway can apply common protections across an organization
[01:14](https://www.youtube.com/watch?v=ioCIpbAKl_M&t=74s)
Organizations that build a central AI platform can put a guardrail layer around the AI Gateway. Applications then use a guarded endpoint instead of calling raw LLM endpoints directly. The same layer can protect chatbots, retrieval-augmented applications, and agent workflows. The speaker later recommends adding broad protections such as content safety, jailbreak detection, and PII detection at this shared level, while keeping application-specific checks closer to the application.

### Guardrails use different techniques because policy checks vary in difficulty
[02:28](https://www.youtube.com/watch?v=ioCIpbAKl_M&t=148s)
Simple risks can be handled with rules, regular expressions, pattern matching, or keyword filters. The speaker gives phone numbers and other PII as examples. Small fine-tuned models can handle factuality, toxicity, language, topic detection, and named-entity detection. More complicated checks, such as whether a response is coherent across several sentences or has the right tone, may need a secondary LLM call. A practical guardrail system combines these approaches in a waterfall, using fast and cheap checks first and slower, more flexible checks only when needed.

### The benchmark measures both guardrail quality and operational cost
[05:35](https://www.youtube.com/watch?v=ioCIpbAKl_M&t=335s)
The speaker says guardrails are often discussed without reporting how accurate they are, how much latency they add, or how many risks they catch. The benchmark covers six common guardrail tasks and compares open-source models, provider endpoints, and prompting approaches. The keynote focuses on PII detection and jailbreak detection. The full report includes the data sets, model choices, and results so that teams can independently verify the measurements and compare the trade-offs for their own systems.

### PII detection has uneven performance across different entity types
[07:30](https://www.youtube.com/watch?v=ioCIpbAKl_M&t=450s)
The PII benchmark compares Microsoft Presidio, a transformer-based open-source model, and a guardrail ensemble. Presidio is easy to use, inexpensive, and fast, with latency around 15 milliseconds in the speaker's test. Its F1 score for the tested task was about 0.37, while more sophisticated systems improved that result by 75 percent. GPU-based guardrails had latency around 64 milliseconds, which the speaker said was still unlikely to be perceptible to a person. Performance also varied by entity type. Names were difficult, while email addresses and phone numbers had much higher scores.

### Jailbreak detectors make different trade-offs between catching attacks and raising false alarms
[12:40](https://www.youtube.com/watch?v=ioCIpbAKl_M&t=760s)
The jailbreak benchmark compares Meta's Llama Prompt Guard, an Anthropic prompting approach, Microsoft Shield Prompt, and a transformer-based model built by the speaker's team. Llama Prompt Guard had the lowest tested GPU latency, around 52 milliseconds, but it was heavily skewed toward predicting a positive label. Anthropic's prompted model had a strong true-positive rate, while also having much higher latency and cost. The speaker's model performed best on the reported balance metrics, including area under the curve, F1, and accuracy. The comparison shows why a single accuracy number is not enough.

### Benchmark data must cover more than the easiest jailbreak category
[15:38](https://www.youtube.com/watch?v=ioCIpbAKl_M&t=938s)
The speaker found that open-source jailbreak data sets over-represented roleplay, pretend, and hypothetical attacks, including DAN-style prompts. The benchmark team balanced the data to include additional jailbreak categories. This matters because a detector can look effective when it is tested mostly on one familiar attack pattern and then fail on other types. The report shares the data and curation details so teams can inspect what their own benchmark actually measures.

### Guardrails should be selected at both enterprise and application levels
[25:46](https://www.youtube.com/watch?v=ioCIpbAKl_M&t=1546s)
In the question session, the speaker recommends putting common protections in an organization-wide guardrail proxy, especially when every model request passes through a central Gateway. Application teams still need their own checks before production because customer-facing, internal, regulated, and creative applications have different risks. A hallucination may be a liability in a customer-facing product but acceptable in a creative application. The speaker argues that a single universal guardrail cannot express all of these different requirements.

## Notable quotes
- Shrea: "Guardrails and any AI reliability framework you'll use under the hood are basically machine learning models and that you should benchmark every single ML model that you should use." (16:26)
- Shrea: "Even with Microsoft Presidio's adoption, nobody really benchmarks how effective Microsoft Presidio is." (08:53)
- Shrea: "There is no one universally applicable answer for that question about trade-off between cost and latency for safety, because every safety means different things to different people." (21:10)
- Shrea: "You still will end up needing some specification per application and per industry." (27:20)

## Tools & references mentioned
- Guardrails Hub
- Microsoft Presidio
- GLiNER
- Meta's Llama Prompt Guard
- Anthropic
- Claude
- Microsoft Shield Prompt
- AI for Privacy
- PII Asking 300K

## Who should watch
- You are building an AI Gateway and need to decide which protections belong in a shared organization-wide layer.
- Your application handles personal data, regulated workflows, or customer-facing responses and you need evidence about detector quality before deployment.
- You are comparing guardrail models and need to weigh false positives, latency, cost, and coverage instead of choosing the most familiar model.

## Related talks

- [Guardrails for LLMs: A Practical Approach](https://mlopstalks.com/talks/guardrails-for-llms-a-practical-approach) (Shreya Rajpal, Guardrails AI, 11:51)
- [Building the Next Generation of Reliable AI](https://mlopstalks.com/talks/building-the-next-generation-of-reliable-ai) (Shreya Rajpal, Guardrails AI, 30:27)
- [Transforming AI Safety & Security](https://mlopstalks.com/talks/transforming-ai-safety-security) (Manojkumar Parmar, AIShield, a Corporate Startup of Bosch, 23:33)
- [Harnessing AI APIs for Safer, Accurate, & Reliable Applications](https://mlopstalks.com/talks/harnessing-ai-apis-for-safer-accurate-reliable-applications) (Ron Heichman, SentinelOne, 1:08:14)
- [GenAI: An Unreliable Information Store](https://mlopstalks.com/talks/genai-an-unreliable-information-store) (Noble Ackerson, Venta, 27:55)