EXPLAINABLE AI : foundations, methodologies and applications

This book presents an overview and several applications of explainable artificial intelligence (XAI). It covers different aspects related to explainable artificial intelligence, such as the need to make the AI models interpretable, how black box machine/deep learning models can be understood using various XAI methods, different evaluation metrics for XAI, human-centered explainable AI, and applications of explainable AI in health care, security surveillance, transportation, among other areas. The book is suitable for students and academics aiming to build up their background on explainable AI and can guide them in making machine/deep learning models more transparent. The book can be used as a reference book for teaching a graduate course on artificial intelligence, applied machine learning, or neural networks. Researchers working in the area of AI can use this book to discover the recent developments in XAI. Besides its use in academia, this book could be used by practitioners in AI industries, healthcare industries, medicine, autonomous vehicles, and security surveillance, who would like to develop AI techniques and applications with explanations. Preface 6 Contents 8 Contributors 15 Abbreviations 18 1 Black Box Models for eXplainable Artificial Intelligence 22 1.1 Introduction to Machine Learning 23 1.1.1 Motivation 24 1.1.2 Scope of the Paper 24 1.2 Importance of Cyber Security in eXplainable Artificial Intelligence 25 1.2.1 Importance of Trustworthiness 26 1.3 Deep Learning (DL) Methods Contribute to XAI 28 1.4 Intrusion Detection System 29 1.4.1 Classification of Intrusion Detection System 31 1.5 Applications of Cyber Security and XAI 32 1.6 Comparison of XAI Using Black Box Methods 38 1.7 Conclusion 40 References 41 2 Fundamental Fallacies in Definitions of Explainable AI: Explainable to Whom and Why? 46 2.1 Introduction 46 2.1.1 A Short History of Explainable AI 46 2.1.2 Diversity of Motives for Creating Explainable AI 48 2.1.3 Internal Inconsistency of Motives for Creating XAI 49 2.1.4 The Contradiction Between the Motives for Creating Explainable AI 50 2.1.5 Paradigm Shift of Explainable Artificial Intelligence 51 2.2 Proposed AI Model 52 2.2.1 The Best Way to Optimize the Interaction Between Human and AI 52 2.2.2 Forecasts Are not Necessarily Useful Information 53 2.2.3 Criteria for Evaluating Explanations 54 2.2.4 Explainable to Whom and Why? 56 2.3 Proposed Architecture 57 2.3.1 Fitness Function for Explainable AI 57 2.3.2 Deep Neural Network is Great for Explainable AI 58 2.3.3 The More Multitasking the Better 58 2.3.4 How to Collect Multitasking Datasets 59 2.3.5 Proposed Neural Network Architecture 59 2.4 Conclusions 62 References 62 3 An Overview of Explainable AI Methods, Forms and Frameworks 64 3.1 Introduction 64 3.2 XAI Methods and Their Classifications 66 3.2.1 Based on the Scope of Explainability 66 3.2.2 Based on Implementation 67 3.2.3 Based on Applicability 68 3.2.4 Based on Explanation Level 69 3.3 Forms of Explanation 70 3.3.1 Analytical Explanation 70 3.3.2 Visual Explanation 71 3.3.3 Rule-Based Explanation 72 3.3.4 Textual Explanation 73 3.4 Frameworks for Model Interpretability and Explanation 74 3.4.1 Explain like I'm 5 75 3.4.2 Skater 75 3.4.3 Local Interpretable Model-Agnostic Explanations 75 3.4.4 Shapley Additive Explanations 75 3.4.5 Anchors 76 3.4.6 Deep Learning Important Features 76 3.5 Conclusion and Future Directions 77 References 78 4 Methods and Metrics for Explaining Artificial Intelligence Models: A Review 81 4.1 Introduction 81 4.1.1 Bringing Explainability to AI Decision—Need for Explainable AI 83 4.2 Taxonomy of Explaining AI Decisions 84 4.3 Methods of Explainable Artificial Intelligence 87 4.3.1 Techniques of Explainable AI 89 4.3.2 Stages of AI Explainability 90 4.3.3 Types of Post-model Explaination Methods 94 4.4 Metrics for Explainable Artificial Intelligence 99 4.4.1 Evaluation Metrics for Explaining AI Decisions 100 4.5 Use-Case: Explaining Deep Learning Models Using Grad-CAM 101 4.6 Challenges and Future Directions 102 4.7 Conclusion 105 References 105 5 Evaluation Measures and Applications for Explainable AI 109 5.1 Introduction 109 5.2 Literature Review 110 5.3 Basics Related to XAI 111 5.3.1 Understanding 111 5.3.2 Explicability 111 5.3.3 Explainability 111 5.3.4 Transparency 112 5.3.5 Explaining 112 5.3.6 Interpretability 112 5.3.7 Correctability 112 5.3.8 Interactivity 112 5.3.9 Comprehensibility 112 5.4 What is Explainable AI? 113 5.4.1 Fairness 113 5.4.2 Causality 113 5.4.3 Safety 113 5.4.4 Bias 113 5.4.5 Transparency 113 5.5 Need for Transparency and Trust in AI 114 5.6 The Black Box Deep Learning Models 114 5.7 Classification of XAI Methods 115 5.7.1 Global Methods Versus Local Methods 116 5.7.2 Surrogate Methods Versus Visualization Methods 116 5.7.3 Model Specific Versus Model Agnostic 116 5.7.4 Pre-Model Versus In-Model Versus Post-Model 116 5.8 XAI’s Evaluation Methods 117 5.8.1 Mental Model 117 5.8.2 Explanation Usefulness and Satisfaction 117 5.8.3 User Trust and Reliance 117 5.8.4 Human-AI Task Performance 118 5.8.5 Computational Measures 118 5.9 XAI’s Explanation Methods 118 5.9.1 Lime 118 5.9.2 Sp-Lime 119 5.9.3 DeepLIFT 119 5.9.4 Layer-Wise Relevance Propagation 119 5.9.5 Characteristic Value Evaluation 119 5.9.6 Reasoning from Examples 120 5.9.7 Latent Space Traversal 120 5.10 Explainable AI Stakeholders 120 5.10.1 Developers 120 5.10.2 Theorists 120 5.10.3 Ethicists 121 5.10.4 Users 121 5.11 Applications 121 5.11.1 XAI for Training and Tutoring 121 5.11.2 XAI for 6G 122 5.11.3 XAI for Network Intrusion Detection 122 5.11.4 XAI Planning as a Service 123 5.11.5 XAI for Prediction of Non-Communicable Diseases 123 5.11.6 XAI for Scanning Patients for COVID-19 Signs 124 5.12 Possible Research Ideology and Discussions 127 5.13 Conclusion 128 References 128 6 Explainable AI and Its Applications in Healthcare 131 6.1 Introduction 131 6.2 The Multidisciplinary Nature of Explainable AI in Healthcare 133 6.2.1 Technological Outlook 133 6.2.2 Legal Outlook 134 6.2.3 Medical Outlook 135 6.2.4 Ethical Outlook 135 6.2.5 Patient Outlook 136 6.3 Different XAI Techniques Used in Healthcare 136 6.3.1 Methods to Explain Deep Learning Models 137 6.3.2 Explainability by Using White-Box Models 139 6.3.3 Explainability Methods to Increase Fairness in Machine Learning Models 140 6.3.4 Explainability Methods to Analyze Sensitivity of a Model 141 6.4 Application of XAI in Healthcare 142 6.4.1 Medical Diagnostics 142 6.4.2 Medical Imaging 143 6.4.3 Surgery 146 6.4.4 Detection of COVID-19 146 6.5 Conclusion 147 References 148 7 Explainable AI Driven Applications for Patient Care and Treatment 154 7.1 General 154 7.2 Benefits of Technology and AI in Healthcare Sector 156 7.3 Most Common AI-Based Healthcare Applications 158 7.4 Issues/Concerns of Using AI in Health Care 160 7.5 Why Explainable AI? 161 7.6 History of XAI 165 7.7 Explainable AI’s Benefits in Healthcare 166 7.8 XAI Has Proposed Applications for Patient Treatment and Care 169 7.9 Future Prospects of XAI in Medical Care 170 7.10 Case Study on Explainable AI 171 7.11 Framework for Explainable AI 172 7.12 Conclusion 173 References 173 8 Explainable Machine Learning for Autonomous Vehicle Positioning Using SHAP 176 8.1 Introduction 177 8.1.1 Global Navigation Satellite System (GNSS) and Autonomous Vehicles 178 8.1.2 Navigation Using Inertial Measurement Sensors 179 8.1.3 Inertial Positioning Using Wheel Encoder Sensors 179 8.1.4 Motivation for Explainability in AV Positioning 180 8.2 eXplainable Artificial Intelligence (XAI): Background and Current Challenges 180 8.2.1 Why XAI in Autonomous Driving? 180 8.2.2 What is XAI? 182 8.2.3 Types of XAI 183 8.3 XAI in Autonomous Vehicle and Localisation 185 8.4 Methodology 186 8.4.1 Dataset: IO-VNBD (Inertial and Odometry Vehicle Navigation Benchmark Dataset) 187 8.4.2 Mathematical Formulation of the Learning Problem 187 8.4.3 WhONet’s Learning Scheme 189 8.4.4 Performance Evaluation Metrics 189 8.4.5 Training of the WhONet Models 190 8.4.6 WhONet’s Evaluation 191 8.4.7 SHapley Additive exPlanations (SHAP) Method 191 8.5 Results and Discussions 191 8.6 Conclusions 194 References 197 9 A Smart System for the Assessment of Genuineness or Trustworthiness of the Tip-Off Using Audio Signals: An Explainable AI Approach 203 9.1 Introduction 204 9.2 Background 205 9.3 Proposed Methodology 206 9.3.1 Dataset Used 206 9.3.2 Pre-processing 209 9.3.3 Feature Extracted 209 9.3.4 Feature Selected 209 9.3.5 Machine Learning in SER 210 9.3.6 Performance Index 210 9.4 Results and Discussion 211 9.5 Conclusion 219 References 226 10 Face Mask Detection Based Entry Control Using XAI and IoT 228 10.1 Introduction 229 10.2 Literature Review 230 10.3 Methodology 231 10.3.1 Web Application Execution 231 10.3.2 Implementation 232 10.3.3 Activation Functions 234 10.3.4 Raspberry Pi Webserver 235 10.4 Results 236 10.4.1 Dataset 236 10.4.2 Model Summary 236 10.4.3 Model Evaluation 237 10.5 Conclusion 238 References 240 11 Human-AI Interfaces are a Central Component of Trustworthy AI 242 11.1 Introduction 242 11.2 Regulatory Requirements for Trustworthy AI 244 11.3 Explicability—An Ethical Principle for Trustworthy AI 246 11.4 User-Centered Approach to Trustworthy AI 247 11.4.1 Stakeholder Analysis and Personas for AI 247 11.4.2 User-Testing for AI 251 11.5 An Example Use Case: Computational Pathology 252 11.5.1 AI in Computational Pathology 252 11.5.2 Stakeholder Analysis for Computational Pathology 253 11.5.3 Human-AI Interface in Computational Pathology 259 11.6 Conclusion 264 11.7 List of Abbreviations 265 References 269