AI agents automate complex, ambiguous work by looping through model reasoning, tool calls, observations, and further actions.
2
LangGraph gives 11X more control and reliability than a simple agent loop, but its structure can become obsolete when better models arrive.
3
Hosted agent platforms reduce infrastructure work while creating limits around scaling, deployment pipelines, observability, and vendor lock-in.
Summary
Sherwood Callaway explains how 11X builds Alice, an AI sales development representative, and Julian, a voice agent for inbound sales. He compares text and voice agents, arguing that voice systems are harder because they must interpret tone, parallel cues, and latency-sensitive interaction. Alice uses agents for audience sourcing, lead research, outreach sequencing, and message writing. Sherwood describes the tradeoff between giving users control and letting the agent operate on autopilot. He then explains why 11X chose LangGraph and LangGraph Cloud, including the benefits of hosted queues and scaling infrastructure, along with problems involving deployment limits, traffic spikes, observability, and platform coupling. He expects agents to move toward simpler structures as models improve, although current model limits still make graphs useful. The discussion ends with monitoring and evaluation. 11X uses metrics, logs, traces, production evaluations, and Arize to check whether generated messages contain unsupported claims.
Agents automate work that was previously too ambiguous for software
Sherwood defines an agent as software that can automate complex and ambiguous tasks that could not previously be automated. Agents use tools, sometimes memory, and usually an agent framework. At the center is a loop: a language model plans what to do, calls a tool, observes the result, and decides whether to continue or finish. In Alice, that loop can source prospects, research individual leads, choose an outreach sequence, and write each message. The system decides when to contact someone and what product claims or personal details to include.
Voice agents have extra difficulty because meaning depends on tone and timing
Sherwood contrasts his work on Alice with his earlier company, Opkit, which built phone-calling software for healthcare providers. Healthcare operations often required calls to insurance companies for eligibility checks, prior authorizations, and claims follow-up. Voice agents have to interpret inflections and parallel cues that are absent from typed text. They also have to do this in real time, with latency constraints. Sherwood says the engineering problems at 11X resemble those Opkit faced, although 11X can build on more mature technology and more sophisticated systems.
Alice balances user control against letting the agent make better decisions
Alice creates campaigns by sourcing an audience, researching leads, choosing a message sequence, and generating the content of each message. Sales users may want to impose a particular structure, case study, or target segment, even when 11X believes Alice would perform better with more freedom. Sherwood describes the product problem as a balance between control and results. The company believes Alice does its best work when users put it on autopilot, while still allowing customers to guide brand voice and other parts of the output.
Graphs provide reliability while models still have limits
Sherwood says LangGraph's graph structure is flexible enough to get an agent into production. A simpler agent can use one node and repeatedly follow a reason, act, and observe loop. That structure becomes more attractive as models improve. Current models still make mistakes, cost money, and produce inconsistent results, so 11X adds graph structure around steps that need predictable performance. A graph can make a campaign stage work consistently, which matters when the product promises customers that the system will perform reliably.
Agent architecture can become obsolete before its extra structure pays off
Sherwood worries about premature optimization in agent graphs. Teams can add nodes and loops to improve one part of a workflow, then repeat that process as they add capabilities. A later model may achieve the same result with a much simpler design. The accumulated graph becomes difficult to change and can prevent the agent from doing other things. 11X therefore has to decide whether an investment in scaffolding will pay off before a new model makes it unnecessary. Sherwood is candid that this decision is still based partly on practical experimentation and intuition.
Hosted agent infrastructure trades speed for control
11X uses LangGraph Cloud to run its agents as Node processes in containers behind an API and queue. A request creates an agent run, the queue holds it, and a worker starts the process with the initial state. This prevents requests from being lost during peak traffic, although they may take longer to process. The hosted service saved 11X from building and operating substantial infrastructure while the company was rebuilding Alice under time pressure. The costs included deployment complexity, limits on individual deployments, connection timeouts during spikes, and less control over supporting processes.
Observability is split across overlapping tools and walled gardens
11X sends general logs, traces, and metrics to Datadog, while LangSmith provides visibility into agents running on LangGraph Cloud. Sherwood prefers telemetry in one place because developers should not have to search several systems during an incident. Moving agent data from LangSmith into Datadog has been difficult. A hosted cloud also limits options such as running a sidecar for metrics and traces, which would be easier on AWS, GCP, or Azure. Sherwood sees a growing overlap among agent hosting, observability, and cloud infrastructure products.
Production evals check whether generated messages stay grounded in research
11X evaluates a sample of message-writer traces with an LLM judge. The evaluation compares the research report and demographic information used as inputs with the generated email. It checks whether the message contains claims that do not match the research, including hallucinated facts about a prospect or company. Sherwood says the team samples about 1% because running an LLM evaluation across all traces would be expensive. The resulting score acts as a signal for whether a prompt change has made messages more or less reliable. Arize is the current tool used for this evaluation experiment.
"I think that agents the characteristics that they have are they can automate complex and ambiguous tasks, tasks that really were not you could not automate previously."Sherwood Callaway05:09
Who should watch
You are building a text or voice agent and need to decide how much workflow structure to add before models improve.
Your agent runs in production, but its logs, traces, evaluations, and application telemetry are spread across different systems.
You are choosing between hosting agent infrastructure yourself and using a managed platform with queues, scaling, and deployment support.
