Finding the Right XAI Method—A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Science

🎙️ Episode 20: Finding the Right XAI Method—Evaluating Explainable AI in Climate Science🔗 DOI: https://doi.org/10.48550/arXiv.2303.00652🧩 AbstractExplainable AI (XAI) methods are increasingly used in climate science, but the lack of ground truth explanations makes it difficult to evaluate and compare them effectively. This episode dives into a new framework for systematically evaluating XAI methods based on key properties tailored to climate research needs.📌 Bullet points summaryIntroduces XAI evaluation for climate science, offering a structured approach to assess and compare explanation methods using key desirable properties.Identifies five critical properties for XAI in this context: robustness, faithfulness, randomization, complexity, and localization.Evaluation shows that different XAI methods perform differently across these properties, with performance also depending on model architecture.Salience methods often score well on faithfulness and complexity but lower on randomization.Sensitivity methods typically do better on randomization but at the expense of other properties.Proposes a framework to guide method selection: assess the importance of each property for the research task, compute skill scores for available methods, and rank or combine methods accordingly.Highlights the role of benchmark datasets and evaluation metrics in supporting transparent and context-specific XAI adoption in climate science.💡 The Big IdeaThis work empowers climate researchers to make informed, task-specific choices in explainable AI, turning a fragmented XAI landscape into a guided, comparative process rooted in scientific needs.📖 CitationBommer, Philine Lou, et al. "Finding the right XAI method—A guide for the evaluation and ranking of explainable AI methods in climate science." Artificial Intelligence for the Earth Systems 3.3 (2024): e230074.