Explainability

The ability to give people a meaningful account of why an AI system produced a particular output or decision.

Definition

NIST's four principles of explainable AI ask that systems provide explanations, that the explanations are meaningful to their audience, that they accurately reflect the system's process, and that systems operate within their knowledge limits.

For generative AI, explainability often relies on showing sources, the steps an agent took and the rules it applied, since the inner workings of large models are hard to interpret. Regulated decisions, such as credit or hiring, may require specific explanations to the people affected.

Sources

  1. NIST IR 8312, Four Principles of Explainable Artificial Intelligence (2021)