Explainable, Transparent Autonomous Agents and Multi-Agent Systems : Second International Workshop, EXTRAAMAS 2020, Auckland, New Zealand, May 9–13, 2020, Revised Selected Papers

This book constitutes the proceedings of the Second International Workshop on Explainable, Transparent Autonomous Agents and Multi-Agent Systems, EXTRAAMAS 2020, which was due to be held in Auckland, New Zealand, in May 2020. The conference was held virtually due to the COVID-19 pandemic. The 8 revised and extended papers were carefully selected from 20 submissions and are presented here with one demo paper. The papers are organized in the following topical sections: explainable agents; cross disciplinary XAI; explainable machine learning; demos. Preface 6 Organization 8 Contents 10 Explainable Agents 12 Agent-Based Explanations in AI: Towards an Abstract Framework 13 1 Introduction 13 2 Background 15 2.1 Related Work 16 3 Explanation vs. Interpretation 18 3.1 A Conceptual Framework for XAI 20 3.2 Discussion 22 3.3 Practical Remarks 23 4 Assessment of the Framework 24 4.1 Model Explanation 24 4.2 Outcome Explanation 25 4.3 Model Inspection 25 4.4 Transparent Box Design 26 5 Towards the Social Dimension of Explainability 26 6 Conclusion 28 References 29 Agent EXPRI: Licence to Explain 31 1 ``EXPRI, Agent EXPRI'': Introduction 31 2 ``For Your Eyes only'': Related Work in Privacy 32 3 ``A View to an EXPRI'': The Agent Architecture 33 3.1 Utility-Driven Component of EXPRI 33 3.2 Value-Aligned Component of EXPRI 35 3.3 Resolution of MPCs 36 4 ``From Practical Reasoning with Love'': Design of the Cognitive Process 37 4.1 Problem Formulation 38 4.2 Epistemic Stage 39 4.3 Choice of Action 40 5 ``The Cognitive Process Is Not Enough'': Challenges for Designing the Social Process 43 6 ``Tomorrow Never Dies'': Discussion and Future Work 46 References 46 In-Time Explainability in Multi-Agent Systems: Challenges, Opportunities, and Roadmap 49 1 Introduction 49 2 Background 51 3 Challenges 53 4 Opportunities 56 5 Application Scenarios 57 6 Road Map 59 7 Conclusion 60 References 61 Cross Disciplinary XAI 64 Decision Theory Meets Explainable AI 65 1 Introduction 65 2 Background 66 2.1 Decision Theory 67 2.2 Explainable Artificial Intelligence 68 3 Contextual Importance and Utility (CIU) 69 3.1 CIU of One Input 71 3.2 CIU of More Than One Inputs 74 4 Conclusions 80 References 80 Towards the Role of Theory of Mind in Explanation 83 1 Introduction 83 2 Desiderata for Theory of Mind in Explanation 85 3 A Belief-Based Account of Explanation 86 3.1 Mental States as Epistemic States 86 3.2 Characterizing Explanations 88 3.3 Explanations Involving Multiple Agents 90 4 ``Best'' Explanations for Whom? 91 5 Explainer-Explainee Discrepancies 93 6 The (In)Adequacy of the Explainer's Beliefs 94 6.1 Sources of (In)Adequacy 95 7 Related Work 96 8 Concluding Remarks 97 References 99 A Situation Awareness-Based Framework for Design and Evaluation of Explainable AI 102 1 Introduction 102 2 Situation Awareness in the Human Factors Literature 104 3 Situation Awareness-Based Levels of XAI Framework 105 3.1 Level 1: XAI for Perception 106 3.2 Level 2: XAI for Comprehension 106 3.3 Level 3: XAI for Projection 107 4 Example Approaches Achieving the Levels of XAI from the XAI Literature 108 4.1 Example Approaches Achieving Level 1 XAI 108 4.2 Example Approaches Achieving Level 2 XAI 109 4.3 Example Approaches Achieving Level 3 XAI 109 5 Determining Human Informational Needs 110 6 Evaluating Explanation Quality: A Method for Situation Awareness-Based XAI Assessment 111 6.1 Existing Level 1 XAI Methods and Metrics 111 6.2 Existing Level 2 XAI Methods and Metrics 112 6.3 Existing Level 3 XAI Methods and Metrics 112 6.4 The SAGAT Test and Its Applicability for Assessment of XAI 112 7 Example Application of the Framework 113 8 Future Directions 114 9 Conclusion 115 References 115 Explainable Machine Learning 119 Towards Demystifying Subliminal Persuasiveness: Using XAI-Techniques to Highlight Persuasive Markers of Public Speeches 120 1 Introduction 120 2 Related Work 121 2.1 The Effect of Non-verbal Cues in Persuasive Messages 121 2.2 Explainable Artificial Intelligence 122 3 Data Annotations and Model 124 3.1 Corpus and Annotation Process 124 3.2 Model Architecture and Training 125 4 Highlighting the Cues: Visualising the Network's Eyes 127 4.1 Grad-CAM 127 4.2 Layer-Wise Relevance Propagation 129 5 Discussion and Limitations 130 6 Conclusion 132 References 133 Explainable Agents for Less Bias in Human-Agent Decision Making 136 1 Introduction 136 2 Background 137 3 Explainable Artificial Intelligence 138 3.1 Challenges of Explainable Machine Learning 139 4 Human-Agent Interaction Method 141 4.1 Generate Test Data 141 4.2 Training of Model 142 4.3 Explanation Types 142 5 Empirical Assessment 143 5.1 Study Description 144 5.2 Data Collection 145 5.3 Result Analysis 147 6 Discussion and Perspectives 150 6.1 Limitations 151 6.2 Future Work 152 7 Conclusion 152 References 152 Demos 154 Explainable Agents as Static Web Pages: UAV Simulation Example 155 1 Introduction 155 2 Explainable Agents as Static Web Pages 156 2.1 Motivation 156 2.2 Architecture 157 3 UAV Simulation Example 157 4 Conclusion 159 References 160 Author Index 161 Front Matter ....Pages i-x Front Matter ....Pages 1-1 Agent-Based Explanations in AI: Towards an Abstract Framework (Giovanni Ciatto, Michael I. Schumacher, Andrea Omicini, Davide Calvaresi)....Pages 3-20 Agent EXPRI: Licence to Explain (Francesca Mosca, Ştefan Sarkadi, Jose M. Such, Peter McBurney)....Pages 21-38 In-Time Explainability in Multi-Agent Systems: Challenges, Opportunities, and Roadmap (Francesco Alzetta, Paolo Giorgini, Amro Najjar, Michael I. Schumacher, Davide Calvaresi)....Pages 39-53 Front Matter ....Pages 55-55 Decision Theory Meets Explainable AI (Kary Främling)....Pages 57-74 Towards the Role of Theory of Mind in Explanation (Maayan Shvo, Toryn Q. Klassen, Sheila A. McIlraith)....Pages 75-93 A Situation Awareness-Based Framework for Design and Evaluation of Explainable AI (Lindsay Sanneman, Julie A. Shah)....Pages 94-110 Front Matter ....Pages 111-111 Towards Demystifying Subliminal Persuasiveness: Using XAI-Techniques to Highlight Persuasive Markers of Public Speeches (Klaus Weber, Lukas Tinnes, Tobias Huber, Alexander Heimerl, Marc-Leon Reinecker, Eva Pohlen et al.)....Pages 113-128 Explainable Agents for Less Bias in Human-Agent Decision Making (Avleen Malhi, Samanta Knapic, Kary Främling)....Pages 129-146 Front Matter ....Pages 147-147 Explainable Agents as Static Web Pages: UAV Simulation Example (Yazan Mualla, Timotheus Kampik, Igor H. Tchappi, Amro Najjar, Stéphane Galland, Christophe Nicolle)....Pages 149-154 Back Matter ....Pages 155-155