Observation vs. explanation
The split between recording what happened in an AI incident (observation) and the current theory of why (explanation), so early facts are preserved while explanations are revised.
Reporting and explaining are different jobs. An observation record captures what happened: the runtime conditions, the authority the system had, the external effect, how it was detected, immediate containment, and what is known, suspected and unknown. An explanation record holds the current hypotheses, supporting evidence, confidence, root cause and corrective action.
Keeping them apart protects the evidence. The observation should be difficult to improve after the fact: corrections are appended as facts, not rewritten around a later hypothesis. The explanation is expected to change, so it is versioned as understanding grows, including the root cause.
Why it matters
When the first report must contain a finished explanation, early anomalies get delayed, sanitised or lost. Separating the two lets an organization preserve the strange thing before making it make sense, which is the heart of AI incident reporting. Pair it with a separate severity and uncertainty rating.
Read more in The first AI incident report should be incomplete.
Related terms
AI incident reporting
Recording anomalous AI behaviour as evidence at the moment it is observed, before root cause is known, keeping what was seen separate from the later explanation of why.
Severity and uncertainty
Two separate axes for rating an AI incident: how serious the effect is, and how well it is understood. Something can be minor but unexplained, or serious and well understood.
NIST AI RMF
The NIST AI Risk Management Framework: a voluntary, use-case-agnostic US framework for managing AI risks. NIST's related work also stresses monitoring deployed AI systems after launch.