Medical residents provide a useful model for deploying AI agents with supervision, escalation paths, and ongoing evaluation.
2
Healthcare agents must recognize when to ask for help, respond within the time available, and communicate information in a format that fits clinical work.
3
Safe deployment requires local testing, workflow measurement, post-deployment monitoring, and human review of overrides and near misses.
Summary
Dr. Sarah Gebauer compares AI agents in healthcare with medical residents. Residents make decisions with partial training, supervision, and a duty to ask for help. She argues that agents need the same structure. Their evaluation should include whether they recognize critical moments, escalate appropriately, communicate in a clinically useful format, and act within the time available. Healthcare also creates problems with incomplete inputs, incomplete outputs, disagreements between agents and clinicians, bias, legal responsibility, and differences between local patient populations. Gebauer says products must be tested on local data and measured in real workflows through user satisfaction and time studies. After deployment, teams should track performance, review near misses, study cases where humans overrode an agent, and capture feedback. In the discussion, she says clinical use remains limited, while sepsis prediction and ambient scribes are more established. She advises engineers to involve clinicians and present tools as assistants rather than replacements for diagnosis.
Medical residents offer a working model for supervised agents
Gebauer uses medical residents as an analogy for AI agents because residents begin work with partial training and make decisions under supervision. They may score well on exams yet struggle at the bedside. The medical system has developed safeguards around this situation over a long period. Those safeguards include supervision, continued learning, and expectations that trainees will ask for help. Gebauer argues that healthcare AI can borrow this structure instead of treating agents as either fully autonomous systems or unusable risks.
A useful agent must recognize critical junctures and contact the right person at the right time. Gebauer compares this with residents, where a strong trainee knows when to ask for help. Failure to escalate can matter when a patient is beginning to deteriorate. The agent also needs to match its response to the time available. A stroke alert may call for a short answer because treatment is urgent, while a rare-disease discussion in a neurology clinic can support a longer explanation with references.
Gebauer says communication is a major source of errors and patient harm in healthcare, so AI integration does not remove that risk. Agents need to provide the right information to the right person at the right time. She gives the example of an emergency patient with heart failure symptoms after a salt-heavy meal. When the facts arrive in an expected format, a physician can quickly form a diagnosis and treatment plan. A long, poorly ordered story slows the clinician down and makes the tool frustrating to use.
Healthcare agents face missing information and disagreement
Clinicians gather many signals from a patient, including observations made while simply walking past a room. AI agents may not have comparable access to context, so a small missing detail can change a diagnosis or treatment decision. Agents can also omit information in their output, much as residents sometimes do. The supervising clinician must recognize when important facts are missing. Gebauer also raises unresolved questions about disagreements between agents, disagreements with clinicians, and the liability attached to accepting or rejecting an agent's recommendation.
Local testing and workflow fit determine readiness
An agent should be tested on the data, patients, and settings where it will operate. Small differences between institutions can change which illnesses are common and how patients present. Gebauer says workflow integration will determine whether doctors, nurses, and other healthcare workers accept a product. Teams should track user satisfaction and conduct time studies, while checking whether the agent reaches the right person with the right information. Testing should also cover edge cases, failure modes, training, user acceptance, reliability, and robustness.
Post-deployment monitoring can borrow from medical training
Healthcare needs stronger monitoring after deployment. Gebauer compares agent performance tracking with resident evaluations and case-log reviews. Near misses can be examined in a way similar to morbidity and mortality conferences. Teams can review cases where a human overrode the agent and capture feedback about what the agent failed to consider. This creates a way to find problems after launch instead of relying only on pre-deployment testing.
In the discussion, Gebauer says she does not currently have AI tools in her own day-to-day clinical work. Her experience includes health technology consulting and AI model evaluation at RAND, while healthcare adoption has moved slowly. Current use cases are concentrated in administrative work such as billing and revenue cycle management. In clinical settings, she mentions predictive models for identifying patients who may develop sepsis and ambient scribes that turn doctor-patient conversations into notes.
Clinicians should be treated as partners, not replaced
Gebauer advises engineers to seek clinical input and work on problems that affect real people. She says doctors often dislike tools that suggest diagnoses because figuring out a diagnosis is a valued part of the job. A product may be easier to adopt when it is presented as an assistant that supports clinical reasoning. She contrasts that with claiming that the tool will diagnose everyone, which can make clinicians feel that the most satisfying part of their work is being taken away.
