ThousandEyes can use an AI agent to combine application, network, and routing telemetry into a fault-domain assessment.
2
The agent uses Semantic Kernel, an LLM, preprocessing, and stored chat history to turn a dashboard investigation into an explanation with follow-up prompts.
3
Shraddha Yeole evaluates the agent with subject-matter experts, known outage data, human review, and metrics such as hallucination rate, answer relevance, and prompt alignment.
Summary
Shraddha Yeole describes how Cisco ThousandEyes is moving from dashboards that present telemetry to an AI assistant that interprets it. She uses a Spotify availability problem during a Google Cloud outage as an example. An operator would usually inspect HTTP, network, and BGP routing views, then combine evidence across many metrics. The assistant instead correlates those layers and produces a fault-domain assessment, with prompts for drilling into a specific layer. The production design routes requests through Semantic Kernel to a ThousandEyes agent, which preprocesses API data before calling an LLM and stores chat history. Yeole explains the prompting work behind the system, including role prompts, step-by-step reasoning, examples, XML tags, and JSON schemas. She is direct about the hard parts: telemetry is large and complex, raw API responses create excessive token use, and the data needs baselines and enrichment. Evaluation combines expert review, outage ground truth, and LLM-based checks.
ThousandEyes traces a digital experience across systems outside the customer network
Yeole frames troubleshooting around a Spotify stream that buffers or fails to load. The cause could be local Wi-Fi, an internet service provider, a cloud provider, or an intermediate transit network. ThousandEyes provides path traces across the application server, provider, cloud platform, and internet backbone. That visibility lets an assistant investigate environments the operator does not own as well as the customer's network.
Dashboards leave operators to assemble the fault domain by hand
Yeole describes a Google Cloud outage affecting Spotify availability. A ThousandEyes user would click through HTTP, network, and BGP routing layers and inspect many metrics to decide whether the application or network caused the problem. She identifies cognitive overload, lost context during investigation, and difficulty interpreting the views for people without deep network operations experience.
The assistant correlates layered telemetry into an explanation
Clicking an explain button starts an investigation across the available layers. In the example, the assistant identifies an HTTP-level problem, lower availability than the baseline, and impact across multiple agent locations. It also connects the affected phase with different error codes. Users can open follow-up prompts to ask about a particular layer. The intended change is from presenting data to interpreting it and isolating a fault faster.
The production agent preprocesses data before calling the language model
The request from the AI assistant passes through Semantic Kernel, an open-source framework from Microsoft, and then reaches the ThousandEyes agent. The agent calls API endpoints, performs preprocessing before sending information to the LLM, receives the response, and displays it. Chat history is stored to support faster explanations. The architecture keeps the model call behind an application layer that prepares the telemetry.
Prompt design combines network expertise with controlled output structure
The team uses role prompting that asks the model to act as a network engineer. Yeole says they worked with solution analysts and subject-matter experts to develop step-by-step reasoning prompts. Few-shot examples show the expected input and output, including links that users can open quickly. XML tags for instructions, examples, and output format, together with a JSON schema in the system prompt, give the response a structure the product can consume.
Evaluation uses outage evidence and several response checks
The team began with manual evaluation by subject-matter experts. For the Spotify case, a known outage and its published analysis provided reference material for comparing agent responses. The team also uses an LLM to assess hallucination rate, answer relevance, prompt alignment, and whether the supplied data is appropriate. Human review remains part of the evaluation loop rather than treating model scoring as the only check.
Telemetry needs baselines and enrichment before it is useful to an LLM
Application and network telemetry creates both volume and complexity. Yeole says the team preprocesses and enriches the data, including HTTP timing and latency measures, and uses baselining or predefined machine-learning strategies before sending it to the model. Passing raw API data directly produced high token counts and poor correlations. The model needs a more focused representation of the underlying measurements.
The next version may learn from network-specific outage data
Yeole says the team is adding continuous learning and a human feedback loop built around outage ground truth and subject-matter review. The current system uses an Anthropic model. Future work includes fine-tuning a model on network data, adding more training examples, and hosting that model within the company's environment.
