Spencer Reagan sees the best near-term agent use cases in turning messy, unstructured information into consistent summaries, profiles, and follow-up answers.
2
Agents become difficult to deploy when they can call tools, because non-deterministic behavior, excessive data retrieval, API access, and security risks are hard to constrain.
3
Safe deployment requires ongoing oversight, spending limits, adversarial testing, identity-aware authentication, and rules that can change during execution.
Summary
Spencer Reagan describes where AI agents are producing practical value and where deployment becomes difficult. He points to spreadsheet-heavy work, customer-profile creation, summarization, document review, coding, and personalized marketing. A hotel group, for example, can use an LLM to normalize dining preferences spread across many properties. Reagan advises teams to start with tasks they could explain clearly to a capable intern, then use deterministic software where it fits and LLMs for semantic work. The harder problems involve browser interaction, tool calls, security, and the cost of supervising many agents. He argues that governance must constrain non-deterministic systems without removing their usefulness. Airia provides orchestration, evaluations, red teaming, guardrails, monitoring, and controls around tool use. Reagan also argues that agents need identities and user-based permissions instead of shared API keys. Dynamic rules should prevent actions such as sending company data outside the organization after an agent has accessed sensitive information.
LLMs are already useful for normalizing messy business information
Reagan says current business value often comes from summarizing and expanding information in spreadsheets, records, emails, contracts, and other written material. LLMs can reduce varied descriptions to a shared set of criteria, then produce a consistent output for a sales lead sheet, an RFP, or a customer profile. He gives the example of a hotel group where each property stores dining preferences differently. An LLM can extract restaurants, food types, preferred times, and family details into one profile across the properties. This makes information usable without requiring someone to inspect every local spreadsheet or CRM record.
A clear description of an intern's task is a good test for an agent project
Reagan's test for an agent idea is whether a company could explain the task in natural language to an intelligent intern who is new to the business. If the task is to inspect customer records across 15 hotels and extract dining habits into a defined profile, the work is specific enough to automate. If the request is only that one data set should somehow talk to another, the project probably lacks a usable description. He connects this test to return on investment. Repetitive work that people avoid because it is tedious or involves a large amount of data is often a good place to look.
Teams should divide deterministic software from LLM-based work
Reagan refers to Andrej Karpathy's distinction between software 2.0 and software 3.0. Deterministic components remain useful for tasks with fixed behavior, while LLMs help with summarization, semantic understanding, and tool calls. He warns against applying AI to an entire problem without deciding which parts need it. This division also makes the value easier to estimate. Tasks with heavy repetition and a clear natural-language description are more promising than elaborate multi-agent designs built because they are interesting. The host observes that many agent workflows are overengineered, and Reagan agrees that teams should identify the specific work where AI adds value.
Browser automation exposes how poorly existing interfaces fit agents
Reagan says many newer automation requests describe a person logging into a website, checking information, clicking buttons, and uploading files. Browsers were built around human vision and interaction, so agents must be fitted onto interfaces designed for people. He compares this with other technology that has been added to existing human infrastructure instead of replacing it. The same constraint applies to how people communicate with models. Humans type or speak and read a screen, which creates a narrow channel for systems that can process far more information. Reagan expects chat to remain useful, but he also sees purpose-built interfaces with summaries, buttons, and controls that bring AI into the applications people already use.
Tool access creates both operational and security risks
Reagan says tool calls and the Model Context Protocol are moving agents beyond simple question answering and retrieval. An agent can decide whether to look up more information, read the result, and take an action. That creates two problems. The agent must know exactly when and why to use a tool, since an API may return far more data than its context window can handle. It also needs security constraints because an agent with broad permissions can cause damage. Reagan compares this to giving sharp scissors to a toddler. Governance must define what the agent may do while accepting that its behavior is non-deterministic.
Supervision and budgets are needed before agents can be trusted
Reagan compares a new agent with an intern who needs frequent check-ins until they earn trust. He describes a practice where an error is explained to the agent while a separate process examines how to avoid the same error in the future. He admits that this does not scale if thousands of agents require constant human attention. His proposed direction is a hierarchy of agents with different levels of capability, cost, autonomy, and trust. Customers also ask for spending controls from the start. Reagan mentions limits by user, project, task, day, or rolling time window, with investigation into the API calls that caused a budget spike.
Red teaming needs synthetic conversations and human spot checks
Airia treats agent red teaming as an adversarial testing problem because ordinary integration tests do not cover conversational behavior well. Reagan describes generating conversations with different personalities and situations, including angry users, people who are not fluent in English, and overly verbose users. The resulting conversations can be evaluated for accuracy and for how closely the agent follows the expected behavior. He supports combining an LLM judge with human spot checks. LLM evaluation provides scale, while people can catch failures that an automated judge misses. Reagan also warns that models used as judges tend to prefer their own answers, so evaluations need a diverse set of judging models.
Agents need identities and dynamic permissions to interact with business systems safely
Reagan says the current habit of downloading MCP servers from GitHub and passing around API keys repeats decades of security mistakes. An agent that can access Atlassian or Salesforce may lack a clear identity, making data governance and access control difficult. He argues that agents should use the identity of the person who initiated the task, or receive their own managed identities through mechanisms such as OAuth, user-based authentication, and dynamic client registration. Permissions should also change during execution. If an agent reads sensitive information from a Jira issue, a rule can block it from sending a resulting email outside the company. These rules provide a fixed boundary around otherwise flexible agent behavior.
"If you can articulate how you would describe it to just an intern that you hire off the street, chances are it's pretty easy to build an agent out of it."Spencer Reagan10:32
Who should watch
You are choosing agent projects and need a practical test for whether the work is specific enough to automate.
Your agents can call APIs or business tools, and you need controls for permissions, spending, data exposure, and failures.
You are building an evaluation or red-team process and want to combine adversarial conversations with automated judges and human review.
