An AI SRE needs a model of the production system because production has little shared ground truth and changes quickly.
2
Cleric uses layered knowledge graphs, system memory, confidence scores, and read-only tools to narrow the search for root causes.
3
Fully autonomous remediation is still years away, so Cleric starts with search-space reduction and collaboration with engineers.
Summary
Willem Pienaar explains why an autonomous AI SRE is harder to build than a general coding agent. Production systems are sprawling, dynamic, and full of company-specific context. Engineers have tests and ground truth during development, while production rarely has a complete record of past failures and their fixes. Cleric builds layered knowledge graphs from infrastructure, deployments, Slack discussions, run books, and past incidents. Its diagnostic agent uses those graphs and existing tools to narrow the area where an engineer should investigate. Pienaar describes three forms of memory, budget limits for investigations, confidence scoring, and evaluation environments that reproduce failures with the same tools used in production. The system is read-only today and focuses on low-risk, collaborative diagnosis. Pienaar is clear that changing production automatically is a longer-term goal because teams need evidence, guard rails, and trust before allowing an agent to act on critical systems.
Production gives agents far less ground truth than development does
Willem Pienaar says development has tests, IDEs, feedback cycles, and public examples of issues with their solutions. Enterprise production environments have no comparable dataset containing every failure and fix. Systems are also sprawling and dynamic, with cloud projects, networks, Kubernetes clusters, nodes, pods, containers, processes, code, and cross-system references. AI-generated code adds more components whose internals teams may not fully understand. The result is an unsupervised diagnostic problem where velocity increases while system understanding declines.
A knowledge graph lets the agent reduce the search space
Cleric builds a graph from infrastructure and unstructured sources such as Kubernetes objects, conversations, deployments, and GitHub issues. Willem describes a small OpenTelemetry reference stack with roughly 12 or 13 services, where the displayed graph showed only about 10 percent of the relationships and covered only the infrastructure layer. The graph helps the diagnostic agent walk causal relationships toward a root cause instead of exploring production from first principles. It also lets background scans find risks such as a publicly exposed bucket or a pod whose memory growth may cause a crash.
Cleric uses budgets and human steering to contain investigations
Background graph building uses cheaper models and a daily budget. An individual investigation also receives a spending cap, such as 10 cents or a dollar, and the agent uses its tools until it reaches that limit. Engineers can tell it to continue or stop. Pienaar says the system aims first at search-space reduction, such as identifying the likely service or cluster, rather than changing code or infrastructure. The agent reports what it checked and what it did not check, so the engineer can steer the next step.
Agents need stopping signals because long investigations often go wrong
Pienaar says successful investigations tend to finish quickly, while failed ones often continue slowly for several minutes. Cleric uses confidence scores and a final critic to avoid sending noisy findings to engineers. Before presenting a diagnosis, the agent enriches the issue, checks for similar historical incidents, and looks at whether earlier solutions received positive feedback. The agent can still chase a recursive loop or take the wrong path. Knowing where it performs well helps the product decide when to stay quiet.
System, procedural, and episodic memory cover different kinds of context
Willem separates memory into the knowledge graph, procedural memory, and episodic memory. The graph captures system state and relationships. Procedural memory records how a team performs a task, similar to a run book. Episodic memory records a specific incident and what the team did. Cleric uses post-change system health, code changes, engineer interactions, and evaluation data as feedback. A rejected finding is negative evidence. An approved recommendation or generated pull request is positive evidence, although the production feedback remains sparse.
Cleric keeps generalized agent logic and evaluation work in its product while injecting customer-specific procedures, memories, and terminology for each environment. Willem describes this as a layering problem. The base agent should gain improvements that apply across customers, while each deployment needs its own processes and history. The team first gives the agent enough raw information to produce useful results, then works on cost and latency. Pienaar compares this to training a new engineer who earns more scope after demonstrating value.
Confidence controls whether an asynchronous alert reaches an engineer
Cleric starts investigations asynchronously when an alert arrives. If the agent has enough confidence, it returns findings. If not, it stays quiet. Engineers can set the confidence threshold and control how concise or detailed the response should be. A synchronous conversation works differently because the engineer is actively refining the request, so the agent generally responds to each question. The initial request still needs enough specificity. If the agent confuses a production issue with a development task, it can waste time and budget, so it asks clarifying questions when the target is unclear.
Evaluation must reproduce the production interface, including its tools
Willem says agents need the same grounding tools that engineers use, including observability systems, logs, traces, and configuration. Language models handle code, configuration, and logs better than metrics and time series. Cleric therefore builds an evaluation environment with matching tool interfaces, then introduces chaos and checks whether the agent identifies the known cause. This preserves transfer between evaluation and production. Pienaar favors trace-based systems with rich events because they give an agent more detail than a simple CPU time series.
Automatic remediation will arrive gradually through risk-controlled access
Cleric is read-only today and does not allow its agent to change production. Pienaar says teams may accept suggestions that enter existing processes, such as pull requests with guard rails. The company starts in lower-risk areas, such as internal Airflow or CI/CD systems, before touching critical customer-facing systems. Engineers resist automatic action mainly because of uncertainty. Trust grows when the agent makes useful diagnoses and safe changes, but Pienaar estimates that full, end-to-end resolution is still several years away.
