Abstract: The use of Artificial Intelligence (AI) is becoming established as a component of safety-critical engineering systems such as the autonomous transportation systems, energy infrastructure, industrial automation, and structural monitoring. As much as AI models and especially deep neural networks prove to be more accurate in prediction, their black box nature create serious issues when it comes to interpretability and safety guarantees. In safety critical areas explainability is not only a good idea, but a precursor to regulatory compliance, accountability and mitigation of hazards. This paper creates a comprehensive mathematical conceptualization of using interpretability metrics and formal safety verification protocols of AI-based engineering systems. Based on the recent developments in explainable artificial intelligence (XAI), mechanistic interpretability, and formal verification theory, we operationalize interpretability as a measurable functional characterization of model structures and safety verification as meet-in-the-middle of reachable state spaces. Our abstraction of causal circuits and verification of time logics constraints using Shapley-based attribution functions make our integrated optimisation model offer explanation fidelity and safety invariants. The framework illustrates how the measures of interpretability can be integrated in the form of the side-constraints in formal verification pipelines, such that the model decision-making can be transparent and safely proven. Findings suggest that the metrics of coupling explanation coupled with reachability analysis can minimize unsafe decision regions and enhance calibration of the trust. The paper adds a mathematically based architecture that can be able to fill in the gap between heuristic XAI methods and serious engineering safety requirements. It is discussed in implications to aerospace, autonomous systems and industrial control environment as well as computational trade-offs and scalability issues. The model suggested creates a direction towards the certifiable AI systems in which interpretability and safety are not set at cross purposes.

Keywords: AI safety verification, Explainable artificial intelligence, Formal mathematical modeling, Mechanistic interpretability

Pages: 87-100

WSEAS Transactions on