Eric Landry says healthcare AI needs tighter controls because errors can affect human lives, so he is testing hybrid systems that combine intent-based responses with LLMs.
2
Zeteo Health is building a RAG chatbot that uses a curated clinical knowledge base to engage underserved patients with short, useful conversations about their health.
3
Landry evaluates models and retrieval settings against a ground-truth dataset, tracks experiments with MLflow, and plans to monitor a small pilot before expanding access.
Summary
Eric Landry describes the practical limits of building AI for healthcare. He compares his early NLP work in 2005, when computation and memory made experiments run overnight, with current systems that can test many models quickly. At Babylon Health, his teams built cloud-agnostic ML infrastructure and intent-based chatbots that could hand users to live agents. His current work at Zeteo Health focuses on patient engagement in underserved communities, where reminders, trusted information, and follow-up conversations may help people take medication and attend checkups. The chatbot uses curated clinical content rather than the model's general knowledge. Landry is also designing the conversation flow, response tone, bias checks, hallucination tests, and escalation paths. He uses Ragas and MLflow to compare models and retrieval settings, and NeMo Guardrails to handle prompt injection, security violations, and predictable intent-based responses. The planned pilot will begin with a small group of users and expand only after observing how the system behaves.
Early NLP work was limited by compute, memory, and slow iteration
Eric Landry entered machine learning through a 2005 University of Texas thesis on clustering scientific genetics articles. He tested different algorithms and wrote them in Java, spending substantial time reading papers, translating proofs, cleaning data, and optimizing data structures. His home computer often ran experiments overnight and sometimes crashed with a stack overflow. At Sun Microsystems, he could use newer servers, reducing some runs to a couple of hours over lunch. He says the biggest change since then is how easily engineers can now iterate through algorithms with tools such as PyTorch.
Babylon Health combined model infrastructure with chat and human handoff
At Babylon Health, Landry's AI platform team built cloud-agnostic infrastructure that could deploy models across Azure, AWS, and other environments. It supported streaming inference on a Kafka stream, inference as a service, batch processing, and model training across NLP, structured data, and medical data use cases. His conversation platform used intent-based chatbots, live chat, automated responses, and a handoff to a live agent when the user was dissatisfied. The chatbot launched shortly before Babylon closed its US operation, but early results suggested that it deflected up to 40% of users from live agents.
Healthcare requires hybrid chatbot designs with controlled responses
Landry says an LLM chatbot cannot be released without appropriate controls, especially when the system handles health questions. He is experimenting with a hybrid design that combines intent-based or rules-based behavior with an LLM. Some responses need to be predictable because an unsafe or inappropriate answer can damage trust and put people at risk. He contrasts healthcare with his former travel work at Expedia, where a broken system might ruin a vacation but would not normally threaten a person's life. The goal is to use generative responses where they help while retaining control over sensitive interactions.
Keeping medical data in its region requires different models or federated learning
Landry explains that healthcare data often cannot move between regions, such as from the UK to the US, for training or inference. Teams therefore need strategies that keep data in place. Sometimes that means using separate models because the data, rules, and distributions differ by region. His teams also experimented with federated learning, where regional agents train locally and transfer model weights rather than the underlying data. He tested the idea on mobile devices with skin-cancer images, allowing devices to contribute to training without sending the images away. The experiment did not become an enterprise priority, although he considered it technically successful.
Zeteo Health focuses on engagement before more ambitious medical use cases
At Zeteo Health, Landry is working on healthcare access for underserved communities, including Black and Hispanic communities in the US that he says often distrust the healthcare system. He connects that distrust with lower engagement in care. The product focuses on practical behavior, such as taking medication, attending checkups, testing blood glucose, and following a monitoring plan. Its conversational AI framework uses a RAG chatbot to learn from patient conversations and detect trends across groups. Landry gives the example of coughs and fevers becoming more common in Austin, which could support a notification encouraging people to get a flu shot during flu season.
The chatbot should guide a conversation in small pieces
Landry does not want the model to answer health questions with a long block of generic medical knowledge. Zeteo Health plans to use a curated knowledge base reviewed by healthcare professionals, so the model retrieves approved information instead of relying on its own learned knowledge. The response should begin with the basic information, then let the patient ask for more detail. For example, a person asking about prostate cancer testing might first hear that there are three types of tests, then receive more information about a PSA test after asking a follow-up question. Landry says the design problem is conversation flow, not only question-and-answer quality.
Landry built an evaluation framework based on Ragas and connected it to MLflow to track experiments. He measures faithfulness, hallucinations, and bias against a ground-truth dataset. When he tested a Hugging Face dataset labeled for cultural bias against an LLM and a bias detector, the detector missed statements he considered clearly harmful. He therefore wrote additional checks for specific cultures and plans to run the tests before deployment. He also distinguishes subjective cultural sensitivity from objective medical bias. For prostate cancer, general guidance may begin testing at 50, while guidance for Black men can recommend starting between 40 and 45 because incidence is higher.
Model selection and guardrails should be based on the application's own tests
Landry started with OpenAI GPT-3.5 and created a baseline using Ragas. He then tested models available through Amazon Bedrock, paying particular attention to faithfulness and hallucination metrics. Anthropic Claude 3.5 Sonnet performed best in his tests at that point, but he tells teams to repeat the work on their own data because model behavior depends on the dataset. He also tested embedding models and retrieval parameters such as K values. NeMo Guardrails is being explored for prompt injection, security violations, and cases where the product needs a known response for a detected intent. These tools complement the evaluation baseline rather than replacing it.
The pilot will start small because real users expose failures
Landry expects the system to behave differently once real people use it. Before a planned pilot with Mount Sinai Medical Center, the team intends to begin with about 50 users, observe their reactions, and increase exposure in stages. Production monitoring should catch users becoming angry, unsafe responses, and other failures that pre-deployment tests miss. He compares this with Expedia systems that handled up to 2,000 requests per second. Zeteo Health will have far less traffic, which makes close observation possible during the early stages. Landry is honest that users may dislike the product and that the team may need to change direction.
