Teams can prototype AI systems quickly, but moving from roughly 80% complete to production quality requires evaluation, iteration, expert review, and user feedback.
2
Open-source models and self-managed infrastructure give teams more control over model versions, data location, token costs, and output behaviour.
3
Agent systems need governance, observability, constrained actions, and ordinary software engineering practices because sandboxes alone cannot guarantee safety.
Summary
The panel discusses how AI platforms are changing as teams adopt agents, open-source models, and more distributed systems. Frederick describes a mental-health hackathon run with Bell Canada and Kids Help Phone, where the evaluation pipeline ran on Kubernetes in the BuzzHPC environment and handled more than 1,000 submissions. The speakers compare managed APIs with self-hosted models, covering cost, data residency, model version control, GPU selection, and scaling to zero. They are direct about the gap between a fast prototype and a production system. Evaluation needs manual review and user feedback, since an LLM judging another LLM can simply move the uncertainty elsewhere. The discussion also covers agent stacks, structured generation, governance, telemetry, sandboxes, and QA agents. Their advice is practical: use platform capabilities where they remove operational work, preserve control where model behaviour and data matter, and apply established software development practices to agent-generated code.
The hackathon used self-service infrastructure to evaluate sensitive conversational agents
Frederick describes a hackathon organised with Bell Canada and Kids Help Phone around mental health conversations. Teams built models or prompts that decided when a conversation should move to a human, including cases involving suicide ideation. His team built an evaluation pipeline on Kubernetes in the BuzzHPC environment, using hosted LLMs, GPUs, and CPUs. Teams could trigger jobs, inspect their results, and see a leaderboard. The system handled 1,000 submissions during the week. A new hidden dataset was introduced during the event, which changed the rankings and forced teams to test whether their systems generalised.
A platform should remove operational work while leaving teams control over important choices
Alan presents BuzzHPC as a sovereign AI cloud with access to GPU infrastructure while keeping workloads within Canadian jurisdiction. In the hackathon, Buzz provided compute, model inference services, and Jupyter notebook infrastructure. Shu says platform users need reliability, rapid scaling, and responsive support. Built-in governance features could also reduce the work enterprise developers must do themselves. The panel frames the boundary as a practical choice: teams should use the platform for infrastructure capabilities that are hard to operate, while retaining control over model selection, data handling, and application behaviour when those decisions affect their product.
Self-hosted open-source models become more useful as token costs and data concerns appear
Frederick says many teams begin with an API key from OpenAI or Anthropic because it is easy to start. Later, they may need to host a model locally because they want to control data location, manage costs, or avoid provider limits. Alan adds that open-source models let teams run evaluation and synthetic-data workloads without the same token-usage limits imposed by frontier APIs. Self-hosting requires skills in model serving and GPU management, which is where a provider such as BuzzHPC can help. The speakers also discuss how model and GPU size affect the cost and quality of different tasks.
GPU choice depends on memory, generation, workload, and model size
Alan distinguishes GPU generation from GPU memory. Newer Blackwell GPUs can improve inference efficiency for highly quantized models, but an older A100 may still be necessary when a model needs more VRAM. He also argues that the best model depends on the task. Difficult work may need a large model, while routine work can use a smaller one. Agent-to-agent communication remains less explored, so he recommends testing larger open-source models where possible before accepting the limitations of smaller local models. He warns that hardware pricing changes as supply becomes constrained, so tokenomics calculations can become outdated.
The difficult part of an AI product is the work after the first prototype
Frederick says GenAI prototypes often reach about 80% of the desired result quickly. The remaining work to reach production quality is much harder. Teams need proper evaluation, repeated changes, user feedback, and a willingness to investigate failures. He warns against relying too heavily on an LLM as a judge because that raises the question of who evaluates the judge. Manual evaluation by experts is still needed. The panel also connects this work to ordinary product development: teams must understand business requirements, define the product clearly, and check whether it solves a real operational need.
Agent stacks are changing too quickly for teams to become attached to one framework
Frederick describes the progression from prompting an LLM to adding retrieval, tools, memory, context, MCP servers, and agent harnesses. Because the technology and practices change quickly, a system built today may be built differently six months later. He advises teams to choose what fits the product now while accepting that the stack will evolve. Shu describes three enterprise paths: a common platform for simple agents and retrieval, AI features embedded in existing enterprise software, and custom production systems using frontier models, open-source models, and internal capabilities. The enterprise is doing less experimentation with new SaaS because existing products already add AI features.
Structured generation can make agent workflows more predictable
Alan says his agent stack includes tools such as AutoGen or AG2, Hugging Face small agents, CrewAI, and coding tools such as Claude Code, Codex, Open Code, Cursor, and Windsurf. His main frustration is limited control over those coding tools. Structured generation can constrain the model's output vocabulary so it produces a required format, such as a number between zero and ten. The same approach can enforce tool calls or an order of operations. Alan connects this to deep research workflows, where deterministic steps and model-generated work are combined. Pydantic can generate schemas for constrained-generation systems such as XGrammar, Outlines, and Guidance.
Production agents need controls before they need detailed failure analysis
Shu says enterprise agent systems need governance and observability because an agent can take harmful actions, such as deleting a database or repeatedly charging a service. Controls should limit which actions an agent can perform and prevent irreversible changes where possible. The panel discusses telemetry for tool calls and MCP servers, with Alan naming AgentOps and Arize Phoenix as tools that use OpenTelemetry-style tracing. Useful observability should make it possible to filter large traces and identify likely causes without requiring an expensive analysis of every token. Sandboxes help, but Alan warns that software sandboxes can be brittle when agents control a command line.
Agent-generated software still needs requirements, tests, reviews, and visual checks
Frederick says building software with agents does not remove ordinary engineering practices. Teams should define requirements, choose an architecture, avoid duplicated or tangled code, create tests, and ask an agent to review its own work. The panel discusses QA agents that take screenshots, click through interfaces, and report problems before a pull request reaches production. Sandboxes and Playwright-based tools can let agents test code in simulated environments. Frederick adds that language models are good at code because code is text, but they may miss how CSS and other code affect the actual user interface. A visual feedback loop is needed.
"The first thing I want to point out is that there's model size and then there's model generator or sorry, GPU size and then there's GPU generation."Alan17:38
Who should watch
You are deciding whether to use managed model APIs or host open-source models and need a practical way to think about cost, control, data location, and GPU capacity.
Your team has an impressive AI demo and needs to understand the evaluation, product, governance, and operational work required before production.
You are building coding agents or enterprise agents and want concrete guidance on structured outputs, testing, telemetry, action limits, and sandbox risks.
