MeetupThe Motivation for MLOpsGuardrails, governance and security in 2023
38 sessions
Large Language Models in Production Round-table ConversationClaimHannes Hapke says that Digits prefers hosting open-source models in-house because its financial data must remain secure and under its control.18:19
PodcastMultilingual Programming and a Project Structure to Enable ItClaimA Python script can search repositories with regular expressions, and a multi-language workflow can clean repositories and generate security reports.55:49
Challenges and Opportunities in Building Data Science Solutions with LLMsClaimThe speakers want to discuss the challenges and opportunities of building data science solutions with large language models, especially risk, compliance, and guardrails for deployment.0:31
Age of Industrialized AIClaimDan Jeffries says prompt padding, constrained prompts and watcher models can provide short-term guardrails, but rule-based heuristics will not be sufficient on their own.13:02
What is the role of ML Engineers in the time of GPT4 and BARD?ClaimHannes Hapke identifies proprietary data, strict security or privacy requirements, low-latency needs, and strategically important intellectual property as reasons to build machine-learning systems in-house.9:46
Data Privacy and SecurityClaimDiego Oppenheimer frames the panel around data privacy, security, trust, hallucinations, and the difference between low- and high-affordability use cases for large language models.1:27
PodcastCohere Large Language Model at ScaleClaimNils Reimers moved from web development and IT security into AI and machine learning because he wanted to build tangible systems whose value users could directly experience.8:59
Building LLM Applications for ProductionClaimChip Huyen says privacy is challenging both when building an LLM application and when relying on an external provider.8:06
LLMs as Intelligent AssistantsClaimSarah Aerni says Salesforce's AI Cloud is a unified architecture built on secure Hyperforce infrastructure, with a platform, large language models, builders, and Salesforce applications.8:00
PodcastOpen Source and Fast Decision MakingClaimGiving narrowly focused services access to entire repositories, drives, or communication systems creates a serious security risk.11:26
Building Production CopilotsClaimTristan Zajonc describes a production copilot system as having user experiences, a router, agents, plugins, and observability and guardrails.9:00
Wardley Mapping Prompt EngineeringClaimPrompt-engineering systems may include conversation history, prompting techniques, guardrails, chains, agents, privacy-enhancing technologies, DataOps, MLOps, and FinOps.4:05
Building ProductsClaimSahar Mor says open-source language models are the future because they can run at the edge, support privacy-sensitive use cases, work without an internet connection, and be fine-tuned for specific purposes.2:58
Challenges in Providing LLMs as a ServiceClaimCohere supports customer data privacy by allowing customers to deploy its models in a self-managed container through SageMaker.9:24
Transforming AI Safety & SecurityClaimManojkumar Parmar says AIShield is a Corporate Center of Bosch and that his talk is about transforming AI safety and security.0:30
Incorporating LLMs in High-stake Use CasesClaimResults reported in public headlines may not transfer to a team's own high-stakes task when the tasks, domains, privacy requirements, or robustness constraints differ.9:29
The Confidence Checklist for LLMs in ProductionClaimRohit Agarwal recommends privacy policies, personally identifiable information masking, and privacy-focused tools for protecting data sent to model providers.26:48
Guardrails for LLMs: A Practical ApproachClaimShreya Rajpal says Guardrails AI is focused on AI safety and reliability for large language model applications.1:11
Evolving AI Governance for an LLM WorldClaimAI governance exists to establish accountability for AI actions, protect personal data, and provide safeguards for robustness, safety, and risk assessment.3:12
Enabling Defense Missions with Local LLMsClaimLeapfrogAI is designed for egress-limited, ingress-limited, sensitive, secure, and air-gapped environments where users retain complete ownership of their data.12:43
PodcastHarnessing MLOps in FinancePushed backMichelle Marie Conway rejects the idea that banking's security controls are merely inconvenient, arguing that strict controls are necessary to protect customers and their finances.29:40
LLM SecurityPushed backWhether current LLM engineering projects are ready for production use was challenged by the speakers' shared view that most are still demos and need stronger security guardrails.52:04
PodcastThe Centralization of Power in AIClaimClem Delangue argues that concentration of power is the number one danger in AI because centralization makes companies, nonprofits, and governments dependent on a single failure point.19:19
Product Strategy for LLM Features When LLMs Aren't Your ProductPushed backFor non-user-facing cybersecurity use cases, LLM risks can be reduced with additional controls and testing, rather than treating the LLM as an unguarded production component.12:27
GenAI: An Unreliable Information StoreClaimNoble Ackerson says enterprise deployments need guardrails, tests, and humans in or on the loop to provide feedback.9:33
PodcastImpact of LLMs on the Tech Stack and Product DevelopmentClaimBito does not currently use a dedicated guardrail tool; it reviews user feedback and failures, adjusts prompts and rules, and periodically runs tests against the models.30:30
Evaluating LLMs for AI RiskClaimTesting should continue after production, particularly when a company or use case is exposed to security risks and adversarial attacks.2:23
PodcastDesigning for Forward Compatibility in Gen AIPushed backRohit Agarwal says current vector databases do not sufficiently enforce permissions, creating a risk that prompt engineering could retrieve unauthorized data.42:32
PodcastGuarding LLM and NLP APIs: A Trailblazing Odyssey for Enhanced SecurityClaimSupply-chain vulnerabilities are a distinct and important security area for machine-learning systems.3:59
PodcastEnterprises Using MLOps, the Changing LLM Landscape, MLOps PipelinesPushed backChris Van Pelt says model-generated SQL or code must be constrained by the permissions and sandbox available to the executing user or process.28:20
PodcastModel Management in a Regulated EnvironmentPushed backThe organizational approach for coordinating data science with legal, compliance, and finance depends on company size and structure; centralized governance is not the only valid model.52:50