Multimodal Agents for Ageing and Multicultural Societies : Communications of NII Shonan Meetings

This book aims to explore and discuss theories and technologies for the development of socially competent and culture-aware embodied conversational agents for elderly care. To tackle the challenges in ageing societies, this book was written by experts who have a background in assistive technologies for elderly care, culture-aware computing, multimodal dialogue, social robotics and synthetic agents. Chapter 1 presents a vision of an intelligent agent to illustrate the current challenges for the design and development of adaptive systems. Chapter 2 examines how notions of trust and empathy may be applied to human–robot interaction and how it can be used to create the next generation of emphatic agents, which address some of the pressing issues in multicultural aging societies. Chapter 3 discusses multimodal machine learning as an approach to enable more effective and robust modelling technologies and to develop socially competent and culture-aware embodied conversational agents for elderly care. Chapter 4 explores the challenges associated with real-world field tests and deployments. Chapter 5 gives a short introduction to socio-cognitive language processing that describes the idea of coping with everyday language, irony, sarcasm, humor, paralinguistic information such as the physical and mental state and traits of the dialogue partner, and social aspects. This book grew out of the Shonan Meeting seminar entitled “Multimodal Agents for Ageing and Multicultural Societies” held in 2018 in Japan. Researchers and practitioners will be helped to understand the emerging field and the identification of promising approaches from a variety of disciplines such as human–computer interaction, artificial intelligence, and modelling and learning. Preface Contents 1 Adaptive Systems for Multicultural and Ageing Societies 1.1 Vision/Use Case 1.1.1 Example 1. Same Appointment, Different Contexts 1.1.2 Example 2. Synchronizing Agendas 1.1.3 Example 3. Implicit Appointments 1.2 Overview 1.3 What to Adapt 1.3.1 Features of the Context (What to Adapt to) 1.3.2 Features of the Companion System 1.4 When to Adapt 1.5 How to Adapt 1.5.1 Adaptivity in Dialogue 1.5.2 Adaptivity in Multimodal Interaction 1.6 Conclusions References 2 Modeling Trust and Empathy for Socially Interactive Robots 2.1 Introduction 2.1.1 Motivating Distinctions 2.1.2 Applications Domains for Elderly People 2.2 Background 2.3 Modeling Trust 2.3.1 Overtrust 2.4 Modeling Empathy 2.4.1 Overview, Challenges, and Definitions 2.4.2 Computational Models of Emotions and Empathic Behavior 2.4.3 Empathic Values, Culture, and Ethics 2.4.4 Emotions, Empathic Mentalization 2.4.5 Empathic Behavior and Culture 2.5 Research Agenda 2.5.1 Problems Caused by Recording 2.5.2 Empathic Behavior of Robots 2.5.3 Explainability 2.5.4 Conversation Between Robots and Aged People 2.6 Evaluation Methods 2.6.1 Measuring Empathy 2.6.2 Measuring Trust 2.7 Conclusion References 3 Multimodal Machine Learning for Social Interaction with Ageing Individuals 3.1 Introduction 3.2 Multimodal 3.3 High Variability 3.4 Sparse and Noisy Resources 3.5 Concept Drift 3.6 Conclusion References 4 Multimodal and Multicultural Field Agents: Considerations for ``outside-the-lab'' Studies 4.1 Introduction 4.2 Lessons to Be Learned from HCI Field Studies 4.2.1 External Factors 4.2.2 Experimenter's Influence 4.2.3 Controlled Versus Uncontrolled Field Studies 4.2.4 Role Fluidity During Field Studies 4.3 Methods for Field Studies 4.4 Ethical Concerns for Field Studies 4.5 Conclusion or Is It Worth the Effort? References 5 Socio-Cognitive Language Processing for Special User Groups 5.1 Introduction 5.2 Spoken Language Understanding 5.2.1 Rule-Based Approaches 5.2.2 Statistical Approaches 5.2.3 Deep Learning Approaches 5.2.4 Implications for Socio-Cognitive Language Understanding 5.3 The Sociocognitive View 5.4 Conclusion References This book aims to explore and discuss theories and technologies for the development of socially competent and culture-aware embodied conversational agents for elderly care. To tackle the challenges in ageing societies, this book was written by experts who have a background in assistive technologies for elderly care, culture-aware computing, multimodal dialogue, social robotics and synthetic agents. Chapter 1 presents a vision of an intelligent agent to illustrate the current challenges for the design and development of adaptive systems. Chapter 2 examines how notions of trust and empathy may be applied to human–robot interaction and how it can be used to create the next generation of emphatic agents, which address some of the pressing issues in multicultural ageing societies. Chapter 3 discusses multimodal machine learning as an approach to enable more effective and robust modelling technologies and to develop socially competent and culture-aware embodied conversational agents for elderly care. Chapter 4 explores the challenges associated with real-world field tests and deployments. Chapter 5 gives a short introduction to socio-cognitive language processing that describes the idea of coping with everyday language, irony, sarcasm, humor, paralinguistic information such as the physical and mental state and traits of the dialogue partner, and social aspects. This book grew out of the Shonan Meeting seminar entitled “Multimodal Agents for Ageing and Multicultural Societies” held in 2018 in Japan. Researchers and practitioners will be helped to understand the emerging field and the identification of promising approaches from a variety of disciplines such as human–computer interaction, artificial intelligence, modelling, and learning.