WHEN AI GETS IT WRONG: DETECTING, EXPLAINING, AND MITIGATING FAILURE PATTERNS IN INTELLIGENT COMPUTING SYSTEMS

Abstract

Despite the widespread use of artificial intelligence (AI) in high stakes areas, intelligent systems have not yet made the leap to producing inaccurate, biased, or unusual results. One of the primary obstacles to creating reliable AI is the fact that standard performance metrics (like accuracy and F1-score) measure overall performance and can mask consistent failure modes that are specific to particular contexts. In addition, failure detection, explainability and mitigation are often investigated separately as single technical problems instead of as a cohesive process. To meet this demand, the authors propose a framework that quantitatively relates failure detection, explanation and mitigation through a continuous process, by systematically connecting these three elements. To overcome this lack, this paper suggests a framework connecting the three elements (failure detection, explanation and mitigation) in a continuous process from a quantitative perspective.  The methodology considers out-of-distribution detection, confidence calibration, and uncertainty estimation as additional metrics to consider when assessing a machine-learning system beyond the standard metrics for the task. To understand how the input features, contribute to the model's results, and to identify patterns of error that could stem from data faults, distribution changes or lack of confidence in the model, Explainable AI techniques (XAI) are then applied. Guided by these insights, specific mitigation measures are taken on the data level, model level, and decision level such as balancing datasets, retraining models, or applying fallback thresholds. The pre- and post-mitigation performance is carefully assessed on a variety of criteria including predictive accuracy, calibration error, and robustness for the noisy and shifted data distributions. The framework's ability to incorporate detection, explanation, and mitigation, meanwhile, shows that systematic failure patterns can greatly improve the overall reliability, transparency, and robustness of intelligent computing systems in a dynamic real-world environment by implementing a continuous feedback loop.