Podcast

Cracking the Black Box: Real-Time Neuron Monitoring & Causality Traces

Mike Oaten, TIKOSEpisode 358 · 47:26 · Feb 2026 · 76 viewsHosted by Demetrios Brinkmann
Thumbnail for Cracking the Black Box: Real-Time Neuron Monitoring & Causality Traces Watch on YouTube
TL;DR
  1. 1

    Mike Oaten argues that the EU AI Act's harmonized standards give engineering teams a practical route to compliance through defined testing, evaluation, and monitoring requirements.

  2. 2

    TIKOS uses an SDK to capture inference data from open-weights deep learning models, transform it into causality traces, and compare live outputs with expected profiles.

  3. 3

    High-risk AI projects need regulatory scoping before implementation, with trace-level records that support audits, investigations, contestability, and explanations for affected users.

Summary

Mike Oaten explains how AI teams can turn the EU AI Act from a legal concern into an engineering plan. He focuses on the harmonized standards beneath the Act, which describe the practices needed for areas such as fairness, safety, and transparency. TIKOS builds assurance tooling around mechanistic interpretability. Its SDK connects to an open-weights model, captures inference data, reduces it into causality traces, and checks those traces against expected behavior. The system can flag out-of-profile behavior, such as a suspected prompt injection, and can support live gating or later investigation. Oaten also describes why financial services and defense teams are considering open-weights models, since commercial model APIs hide internal behavior. He gives an autonomous vessel as an example where teams must define the operating domain and map each automated capability to applicable rules before building. The conversation also covers the organizational gap between business, technical, and risk teams, and the records needed for audits and user appeals.

Key ideas
00:45

The EU AI Act becomes practical through its harmonized standards

Mike Oaten says TIKOS has focused on the EU AI Act and the harmonized standards written underneath it. The Act sets broad expectations, while those standards describe how teams can meet them in practice. They cover the work needed to show that a system is unbiased, fair, safe, or otherwise fit for its risk class. Following the relevant standards can provide a "presumption of conformity" with the Act. Oaten says teams still need to interpret the standards for their own system, but the remaining gap is small enough to turn into concrete tests, evaluations, and monitoring requirements. He also notes that many standards are licensed by organizations such as ISO and the British Standards Institution.

05:41

Regulatory observability starts with the risks of the system

Demetrios Brinkmann asks how regulatory observability differs from ordinary engineering observability. Oaten says engineers usually watch whether systems work and patch known failure modes. Regulatory work begins with an inventory of AI systems and a risk assessment. Teams need to identify prohibited systems, high-risk systems, and systems with fewer obligations. High-risk examples include loan decisions that could discriminate against people, and AI inside products such as cars, lifts, toys, or machinery that already require safety evidence. Monitoring and evaluation must then target the specific harms the system could cause. A model that must prove it is not biased needs more detailed evidence than a system that only needs to demonstrate ordinary operational performance.

08:27

Causality traces expose behavior hidden by input-output testing

TIKOS's approach uses mechanistic interpretability during inference. Oaten says the company gets inside a running model and extracts causal chains behind decisions such as approving or refusing a loan. These raw traces become the basis for testing and observation. Oaten contrasts this with the model cards and input-output tests available from commercial providers. Those tests can be sophisticated, but they do not reveal what happens inside the model. For high-risk uses, TIKOS therefore expects customers to consider open-weights models, which provide more access for inspection, fine-tuning, and retrieval-based systems. Oaten describes a trade-off between the performance of a closed model and the control needed to meet demanding compliance requirements.

23:23

High-risk projects must scope regulations before building

Oaten describes an autonomous surface vessel project in the defense sector. The first work is to define the vessel's domain of operation, its reporting behavior, how it will alert other vessels, and which sensors and data sources it will use. The team then breaks the system into automated components and maps each component to the relevant maritime, coast guard, defense, and rules-of-the-sea requirements. The vessel remains subject to applicable safety expectations even though there are no people aboard. Oaten says this changes the order of work. In a high-risk project, the first third may involve scoping and compliance checks rather than implementation. Building first and discovering regulatory constraints later can lead to expensive redesign or a reduced project scope.

29:04

The product turns inference logs into profiles and alerts

TIKOS integrates through an SDK connected to an open-weights deep learning model. Each inference produces raw vector-style data, which the system transforms and stores. An information-minimization step removes noise and retains the signal needed to create a causality trace for that model run. Teams can execute many test cases, establish a golden set for normal behavior, and analyze traces against measures such as accuracy, robustness, bias, and transparency. The resulting in-profile and out-profile behavior can support monitoring during deployment. Oaten gives prompt injection as an example of an out-of-profile event in an open-weights language model. The system could flag and gate the output in real time, while a dashboard stores source logs and displays results.

36:00

Trace retention supports audits and individual appeals

Oaten says customers can use the system as a record of every model run without enabling all analysis in real time. A live setup can monitor sensitive areas and gate outputs, while a risk-based setup can store traces and analyze them when a complaint or regulatory request arrives. In financial services, internal auditors, external regulators, and affected users may need access to the evidence behind a decision. A person denied a loan cannot reasonably receive raw vectors and matrices as an explanation. The bank needs to translate the recorded data into the features and characteristics that drove that particular result. Oaten connects this requirement to contestability and redress under the EU AI Act, including the need to explain an individual decision rather than only report aggregate model metrics.

18:48

Technical and risk teams need shared evidence

Oaten describes a recurring gap between business teams, technical teams, and risk or governance teams. A business may fund a system, engineers may build it, and the risk team may arrive later with additional tests that shrink the project or create technical debt. TIKOS tries to give engineers precise material about what a system will need to meet the relevant rules, while giving risk teams enough technical evidence to understand how those requirements are implemented. This creates common ground before the teams make decisions. Oaten connects the process to the rate at which prototypes reach production. Projects with valid business cases can still become waste if late compliance work forces them to be abandoned or rescoped.

41:47

Oaten treats regulation as a condition for wider adoption

Oaten says the EU AI Act reflects expectations that users should not face opaque or unfair systems and should be able to ask why a system produced an answer. He accepts that AI is probabilistic and difficult to make as deterministic as ordinary software, but says teams should still work toward greater transparency and explainability. In his view, trust is part of the path from investment to deployment. Without evidence that systems can be understood and challenged, organizations may remain cautious even when the technology could help with areas such as medical research. He is careful to say that TIKOS addresses only one part of this problem, alongside AI literacy and other policies.

"So you got this phrase called presumption of conformity in this world which is if you do if you have actually if you can demonstrate that you followed the standards then you will automatically be compliant with the act."Mike Oaten03:21
Who should watch
  • You are building AI systems for financial services, defense, healthcare, or another setting where a normal input-output evaluation may not satisfy risk teams or regulators.
  • Your team needs to decide whether the additional control of an open-weights model is worth the performance trade-off of using a closed commercial model.
  • Business, engineering, and governance groups keep discovering compliance requirements late in delivery and need a shared way to scope them earlier.