Human-computer interaction : thematic area, HCI 2022, held as part of the 24th HCI international conference, HCII 2022, virtual event, June 26 - July 1, 2022 : proceedings Part 1, Theoretical approaches and design methods

The three-volume set LNCS 13302, 13303 and 13304 constitutes the refereed proceedings of the Human Computer Interaction thematic area of the 24th International Conference on Human-Computer Interaction, HCII 2022, which took place virtually in June-July 2022. The 132 papers included in this HCI 2022 proceedings were organized in topical sections as follows: Part I: Theoretical and Multidisciplinary Approaches in HCI; Design and Evaluation Methods, Techniques and Tools; Emotions and Design; and Children-Computer Interaction, Part II: Novel Interaction Devices, Methods and Techniques; Text, Speech and Image Processing in HCI; Emotion and Physiological Reactions Recognition; and Human-Robot Interaction, Part III: Design and User Experience Case Studies, Persuasive Design and Behavioral Change; and Interacting with Chatbots and Virtual Agents. Foreword HCI International 2022 Thematic Areas and Affiliated Conferences List of Conference Proceedings Volumes Appearing Before the Conference Preface Human-Computer Interaction Thematic Area (HCI 2022) HCI International 2023 Contents – Part I Contents – Part II Contents – Part III Theoretical and Multidisciplinary Approaches in HCI Non-destructive Interaction. Embracing Mistakes as an Activity Theoretical UX-Design Approach 1 Introduction 1.1 Motivation 1.2 Research Question 1.3 Paper Structure 2 Literature: Activity Theory and UX 2.1 Basic Principles 2.2 Activity Theory and Human-Computer Interaction 2.3 UX 3 Method 3.1 Research Context 4 Using the ORCA Software 4.1 Simulation and Probing 4.2 Simulation as Individual Expansive Learning 4.3 Mediating Practice 5 Design Decisions 5.1 Hierarchical Structure of Iterative Interaction 5.2 Activities as Stored Explorative Simulations 5.3 User Experience as a Continuous Personal Development Process 6 Concluding Remarks 6.1 Limitations 6.2 Contributions 6.3 Further Work and Direction References Immersive Sims: A New Paradigm or a New Game Genre? 1 Introduction 1.1 Paradigms in Human Computer Interaction 2 Characteristics of the Third HCI Paradigm 2.1 Embodied Interaction 2.2 Plurality of Perspectives and Multiple Interpretations 2.3 Context-Driven Design Space 2.4 Meaning Making and Meaningful Engagement 3 Analysis of Immersive Sims 3.1 High Level Characteristics of Immersive Reality Sims 3.2 Low-Level Characteristics of Immersive Sims 3.3 Embodied Interaction with its Material, Affective and Social Dimensions 4 Immersive Sim Prototype 4.1 Usability Testing 4.2 Test Results 5 Evaluation of Immersive Sims as a New Paradigm 6 Conclusion 6.1 Future Work References Hybrid Spaces, Hybrid Methodologies: Finding Ways of Working with Social Sciences and Humanities in Human-Computer Interaction 1 Introduction 2 Third Spaces and Hybridity 2.1 Theoretical Background of Hybridity: Bhabha, Anzalduá, Haraway 2.2 Third Space and Hybridity in PD and HCI 2.3 Potential of Hybrid Methodologies in HCI 3 Exploring Hybridity 3.1 Experimental Spaces Through Artistic Research and Research Through Design 3.2 Intersectional Critical and Speculative Methodologies in Machine Learning Systems Design 4 Towards Conceptualizing Hybrid Methodologies References A Survey on Automated Machine Learning: Problems, Methods and Frameworks 1 Introduction 2 Problem Statement 2.1 Hyperparameter Optimization (HPO) Problem 2.2 Combined Algorithm Selection and Hyperparameter Optimization (CASH) Problem 2.3 Machine Learning Pipeline Creation (MLPC) Problem 3 Methodology 3.1 Search Space 3.2 Search Algorithm 3.3 Cost Efficiency 4 Frameworks 5 Future Direction References Blending Learning Practice and Research for STEAM Education 1 Introduction 2 The Necessity of the Application of Blending Learning in STEAM Education 2.1 Present Situation and Characteristics of STEAM Education 2.2 The Advantages of Blending Learning 3 The Design of Blending STEAM Education Mode 3.1 Set Curriculum Objectives 3.2 Design of Blending Learning Mode 4 Teaching Practice 4.1 Set Curriculum Objectives 4.2 The Content and Effect of Offline Learning 5 Conclusion References The Status of Human-Machine Communication Research: A Decade of Publication Trends Across Top-Ranking Journals 1 Introduction 2 Literature Review 2.1 HMC: Technology as Communicators in Communication 2.2 The Present Study 3 Data and Methods 4 Analytical Strategy 4.1 LDA Topic Modeling 5 Results 5.1 Research Topic Composition and Distribution 5.2 Associations Between Theory, Methods, Affiliations, and Technologies Among Journals 5.3 Trends Associated with First Country Affiliations 5.4 Trends Associated with Funding 6 Discussion 7 Limitations and Future Directions 8 Concluding Remarks References A Descriptive Model of Passive and Natural Passive Human-Computer Interaction 1 Introduction 2 Related Work 3 Natural Passive Interaction 3.1 Definition 3.2 System Response Time 3.3 Type of Response 4 Sample Cases 4.1 Earbuds with Spatial Audio that Maintain the Ideal 3D Audio Image Regardless of Head Rotation 4.2 Display System that Adjusts Naturally to User Position 4.3 Exoskeleton with Adaptive Assistance Power 5 Discussion, Conclusions and Future Work References Engagements2as ‘HCI Material’: Propagating Community Agency, Through Embedded Technologies 1 Research Objective(s) 1.1 Introduction 1.2 Material Engagements as Political Acts 1.3 Defining Active Engagement 1.4 Design-Led Frame 1.5 Design-Led Field Exemplars 1.6 Art-Led Frame 1.7 Art-Led Field Exemplars 1.8 Engagement2(Common Frame) 1.9 Engagement Making 2 Method 3 Results 4 Discussion 4.1 Value to HCI Practice 4.2 Summary 5 Conclusion References Multimodal Semantics for Affordances and Actions 1 Introduction 2 Modeling Human Object Interactions 3 Modeling Habitats and Affordances 4 VoxML 5 Reasoning with Affordances 6 Conclusion and Future Directions References Interaction Designers of the Future: Shedding Light on Students Entering the Industry 1 Introduction 2 Background 3 Method 3.1 Survey Design 3.2 Data Collection 3.3 The Respondents 3.4 Data Analysis 3.5 Limitations 4 Findings 4.1 Quantitative Findings 4.2 Qualitative Findings 5 Discussion 6 Conclusion References Conversations Across Disciplines: Interdisciplinary Experiences from Research in Sociology, Human-Computer Interaction, and Pharmacy 1 Introduction 2 Interdisciplinarity 3 Quantitative-Qualitative Debate 4 Challenging Experiences from Conversations Across Disciplines 5 Advantages of Interdisciplinary Research Combining Qualitative and Quantitative Methods 6 Conclusion References Design and Evaluation Methods, Techniques and Tools A Coding Framework for Usability Evaluation of Digital Health Technologies 1 Background 2 Current Challenges 3 Proposed Coding Framework 4 Codebook Development 4.1 Usability Principles: Which Usability Principle is Targeted by the Feedback? 4.2 Design Components: Which Design Function and Element Are Targeted by the Feedback? 4.3 Clinical Role: Which Clinical Role Provided the Feedback? 4.4 Data-Driven Design Modifications 5 Use Case 5.1 Data Collection 5.2 Feedback Coding 5.3 Results 6 Limitations and Future Directions 7 Conclusion References Designer Interface for a Class of Wearable Computer Applications 1 Introduction 2 Overview of Our Prior Work 3 Motivation for New Features 4 New Features for Embedded System Designer Interfaces 4.1 Aggregation of Sensor Input Over Time 4.2 Access to Information Outside the Wearable Device 4.3 Advanced Actuator Control 4.4 Advanced Reporting 4.5 Automatic Generation of Computer Code 5 Closing Remarks References An Analysis of Communication Balance in Online Consensus-Building Game 1 Introduction 2 Method 2.1 NASA Game 2.2 Experimental Procedure 2.3 Number of Groups 3 A Hypothesis on Communication Balance 3.1 Score and Amount of Remarks 3.2 Analysis of Communication Balance 4 Conclusion References A Study on Consensus Building Mechanism Based on Kansei ~Consideration of Experimental Tasks that Cause Conflicts~ 1 Introduction 2 Related Studies and Purpose 2.1 Related Studies 2.2 Review of Previous Studies 2.3 Purpose 3 Experimental Task Design 3.1 Requirements for the Experimental Task 3.2 Contents of the Task 4 Experiment 4.1 Experimental Environment 4.2 Participants 4.3 Procedure 5 Results and Discussions 6 Conclusions and Future Works References Comparison of Moderated and Unmoderated Remote Usability Sessions for Web-Based Simulation Software: A Randomized Controlled Trial 1 Introduction 1.1 Research Questions and Contributions 2 Related Work 2.1 Moderated and Unmoderated Usability Testing 2.2 Usability Testing Methods 2.3 Careless Responding 2.4 Survey Invitation Elements 2.5 Usability Questionnaires 3 Methodology 3.1 Preparation Components 3.2 HOMVIZ Software and Test Tasks 3.3 Respondent Selection 3.4 Apparatus 3.5 Survey Modes 3.6 Questionnaires 3.7 Data Analysis 4 Results 4.1 RQ1: Comparing the Outcomes of the Moderated and Unmoderated Sessions 4.2 RQ2: Ensuring Data Reliability in Unmoderated Usability Sessions 4.3 Limitations and Challenges 5 Conclusion References A Comparison of Laboratory and Synchronous Remote Usability Testing Methods Using AR 1 Introduction 1.1 AR in Usability Testing 1.2 Remote Usability Testing 1.3 Immersive User Experience Using AR 2 Methodology 2.1 Perceived Product Attractiveness 2.2 User Performance 2.3 User Study Design 2.4 Participants 2.5 Prototype 2.6 Tasks 3 Data Analysis 4 Limitation References Scenario-Based Methods for Hard-to-Reach Populations in Healthcare 1 Introduction 1.1 Scenario-Based Design 1.2 Research Background 1.3 Paper Objectives 2 Methods 2.1 Scenario-Based Design Sessions 3 Results 3.1 Haptic Wearable Device 3.2 Color-Based Visual System on Computer Screens 3.3 Post-visit Summary 4 Discussion 4.1 Informing Scenarios with Clinician Input from a Design Workshop 4.2 Recruiting Designers as Participants 4.3 Providing Contextual Information to Participants for the Design Sessions 5 Conclusions References Tell Me What that Means to You: Small-Story Narratives in Technology Adoption 1 Introduction 2 Background 3 Method 3.1 Study 1: Workshop 3.2 Study 2: Technology Validation 3.3 Study 3: Protocol Validation 4 Results 4.1 Study 1: Workshop 4.2 Study 2: Technology Validation 4.3 Study 3: Protocol Validation 5 Discussion 6 Limitations and Future Work 7 Conclusion References Uncertainty of Information (UoI) Taxonomy Assessment Based on Experimental User Study Results 1 Introduction 2 Uncertainty of Information (UoI) 3 Explanation of the Taxonomies 4 User Study 5 Results 5.1 Scenario One 5.2 Scenario Two 5.3 Summary 6 Conclusion References Case Studies of Motion Capture as a Tool for Human-Computer Interaction Research in the Areas of Design and Animation 1 Introduction 2 Motion Capture 3 Discussion 4 Conclusion References Investigation on Research Model of Product Design Integrated with Big Data Technology 1 Introduction 2 Literature Review 2.1 Relationship Between Design Research and Product Design 2.2 Relationship Between Design Research and Product Design 2.3 Research on Big Data Technology 2.4 Research on Big Data Technology 3 Methodology 4 Results 4.1 Classification of Research Demand Information 4.2 Product Design Research in the Era of Big Data 5 Conclusions and Suggestions References Exploring Mobile Co-design in the Context of Use Continuous Elicitation and Evaluation of Design Suggestions 1 Introduction 2 Distributed Co-design 2.1 Participatory Design 2.2 Interactive Prototyping 3 Method 3.1 Background 3.2 Wizard of Oz and Interactive Prototyping 3.3 The Co-design Sessions in Detail 3.4 Participants (n = 36) 3.5 Analysis 4 Findings 4.1 Findings Related to the Co-design Method 4.2 Findings Related to Elicitation and Evaluation of Design Suggestions 5 Discussion 5.1 A Co-design Method or a Participatory Design Method? 5.2 Suggestions for Future Research 5.3 The Value of This Study 5.4 Conclusion References A Co-creation Interaction Framework and Its Application for Intelligent Design System 1 Introduction 2 Related Work 2.1 HCI in Intelligent Design 2.2 Co-creation and Intelligent Design 3 Co-creation Interaction Framework 3.1 User-Centered 3.2 Collaborative Interaction in Design Process 3.3 Diverse Human-Computer Relationships 4 Case Study: A Co-creative Drawing System 4.1 Problem Definition 4.2 System Implementation 4.3 Discussion 5 Conclusion References Towards Establishing Consistent Proposal Binning Methods for Unimodal and Multimodal Interaction Elicitation Studies 1 Introduction 2 Related Work 2.1 Binning Tools 2.2 Binning Criteria 3 Data Collection 4 Video Annotation Procedure 4.1 Stroke Segmentation 4.2 Gesture Information Recording 4.3 Stroke Coding 5 Interaction Proposal Binning 5.1 Hand Pose 5.2 Movement Path 6 More than Mid-Air Gestures 7 Limitation and Future Work 8 Conclusion References Build User Experience Evaluation System——Selection and Integration of Indicators of Automotive Products 1 Introduction 1.1 Background of the Experience Economy 1.2 Review of the User Experience 2 Overview of the Measurement Dimensions 2.1 Behavioral Performance 2.2 Sensory Perception 2.3 Psychological Measurement 3 User Experience Model Based on AHP 3.1 Analytic Hierarchy Process (AHP) 3.2 Empirical Analysis 4 Conclusion References Emotions and Design Cross-Cultural Design and Evaluation of Student Companion Robots with Varied Kawaii (Cute) Attributes 1 Introduction and Motivation 1.1 Kawaii 1.2 Prior Work 2 Design of Companion Robots for University Students 2.1 Overview 2.2 Team One’s Persona, Scenario and Companion Robot Designs 2.3 Team Two’s Persona, Scenario and Companion Robot Designs 3 Evaluation 3.1 Purpose of the Evaluation 3.2 Participant Demographics 3.3 Study Procedure 3.4 Comparison of Kawaii/Cute Within Each Cluster of Videos 3.5 Differences Between Round-Colorful and Angular-Greyscale Robots 3.6 Selecting Sample Robots for the Second-Phase Biosensor Study 4 Discussion 5 Future Work References Technology in Automotive Brands: Function or Fashion? 1 Introduction 2 Fashion Technology 3 Methodology: Image Coding and Categorizations 4 Results and Analysis 4.1 The Disappeared Layers 4.2 Display and Benz’s Horizon 4.3 The Pawing Between Symbol and Fashion 5 Discussion and Future Works References A Framework to Semi-automated Usability Evaluations Processing Considering Users' Emotional Aspects 1 Introduction 2 Related Work 3 Usability Smells 4 Proposed Framework - EmotiUsing 4.1 Implementation 5 Experiments: Validating Framework 5.1 Dataset 5.2 ML Model 5.3 Results 6 Conclusions References Research on Speech Interaction Design Based on Emotion 1 Preface 2 Current Research Status of Voice Interaction 3 Voice Interaction Based on Emotion 4 The Influence of Personification Degree on User Satisfaction 4.1 Experimental Purpose 4.2 Participating Users 4.3 Experiment 1: Test of Speech Speed Comfort of Voice Interactive Feedback 4.4 Experiment 2: The Influence of the Personification of Virtual Character Model on User Satisfaction 4.5 Experimental Results and Data Analysis 5 Discussion 6 Conclusion References A Proposal of Classification Model for Kawaii Fashion Styles in Japan Using Deep Learning 1 Introduction 2 Model Construction 2.1 Dataset Preparation 2.2 Model Construction 3 Model Evaluation 3.1 Evaluation Method 3.2 Classification Distribution 3.3 Classification Probability 3.4 Classification Performance Metrics 4 Model Improvement 4.1 Improvement on Model Construction 4.2 Improved Model Performance 5 Discussion 6 Conclusion References You Keep Me Hangin’ On: An Analysis of Motivational Mechanisms in Nutrition Apps 1 Introduction 2 Background 2.1 Motivation Theories and Mechanisms 2.2 Gamification 2.3 Nutrition Apps 3 Technical Artifact 3.1 Goal Setting 3.2 Chatbot 3.3 Leaderboard 3.4 Rewards 3.5 Feed 4 User Study 4.1 Methodology 4.2 Results 5 Discussion 6 Conclusion and Future Work References Affective Evaluation of Virtual Kawaii Robotic Gadgets Using Biological Signals in a Remote Collaboration of American and Japanese Students 1 Introduction 2 Design of Collaborative Activities 3 Implementation of Collaborative Practice 3.1 Introduction 3.2 Design of Persona and Scenario 3.3 Design of Four Companion Robots 3.4 Development of Four Companion Robots 3.5 Final Presentation and Evaluation 3.6 Special Lectures 3.7 Farewell Meeting 4 Discussion 5 Conclusion References Possibilities and Research Issues for Measuring Human Emotions in Real Life 1 Introduction 2 Current Status and Issues on Low Burden Measurement 2.1 Wearable Devices 2.2 Environment-Embedded Devices 2.3 Camera-Video-Based Systems 3 Technical and Research Issues 3.1 Measurement Failure 3.2 Reliability of Measurement 3.3 Data Collection Using a Small Device 4 Field Trials Using M5Stack or M5StickC 4.1 Baby Robot that Makes Newborn’s Presence Felt by Distant Father 4.2 Early Detection of Emotional Changes in Children with Autism Spectrum Disorder 4.3 Assessment of Tension and Self-control Support for Student Athletes 5 Discussions 6 Conclusion References Model Construction of “Kawaii Characters” Using Deep Learning 1 Introduction 2 Method 2.1 Constructing “Kawaii Character” Dataset 2.2 Constructing “Kawaii Character” Model 3 Classification Results 4 Evaluating Parameter Settings for Transfer Learning 4.1 Accuracy Evaluation by Changing Learning Rate 5 Conclusion References Interface Visual Design Based on User Emotional Experience – Taking Pregnancy and Parenting Applet as an Example 1 Introduction 1.1 Background of the Study 1.2 Current Status of Research 1.3 The Problem to be Solved 2 Analysis of Female User Groups in Pregnancy and Parenting Applet 2.1 Characteristics of Female User Groups in Pregnancy and Parenting Applet 2.2 User Role Models of Female User Groups in Pregnancy and Parenting Applet 2.3 Exploring the Real Needs of Users and the Functional Architecture of Interface Design 3 Study of Interaction Interface Design Principles 3.1 Eye Movement Experimental 3.2 Data Analysis 3.3 A Study of Visual Design Paradigms for Interfaces Based on Affective Design Theory 4 The Design of the “Lamb Pregnancy” Applet Based on the User’s Emotional Experience 4.1 Product Positioning and Objectives 4.2 Product Benefit Points 4.3 The Design Framework of the “Lamb Nurture” Applet 5 Design Assessment 5.1 EEG Experimental Evaluation 5.2 Data Analysis 5.3 Conclusion References Improving Emotional Intelligence in the New Normal Using Metaverse Applications for Digital Native 1 Introduction 2 Literature Review 3 Research Methodology 4 Experimental Research and Analysis Study 4.1 Focus Interview Study Result 4.2 User Analysis 5 Research Evaluation and Analysis 5.1 PESTLE Analysis Result 5.2 Emotional Intelligence Analysis Result 5.3 Opportunity Gap Analysis Result 5.4 Quality Function Deployment on Features Development Analytical Result 5.5 Kano Model Analysis 6 Discussion and Conclusion References Children-Computer Interaction A Systematic Review of Multimodal Interaction in Artificial Intelligent System Supporting Children to Learn Music 1 Introduction 2 Literature Review 3 Findings: The Synthesis of the Literature 3.1 Multimodal Systems 3.2 Robot System 4 Discussion 5 Conclusion References Research on the Design of Children’s Breath Training Products from the Perspective of Motivation Theory 1 Introduction 2 Motivation Theory and Breath Training 2.1 Motivation Theory 2.2 Current Status and Deficiencies of Breathing Training 3 Comprehensive Evaluation of Children’s Breathing Training Products 3.1 Comprehensive Evaluation Process of Children’s Breathing Training Products 3.2 Construction of Usability Evaluation Model of Children’s Breath Training Products 3.3 Hierarchical Single Sort 3.4 Consistency Check 3.5 Total Ranking of Weights 4 Evaluation Results and Design Strategy 4.1 Strengthen the Safety of Medical Products and Exclude Factors that Reduce Incentives 4.2 Naturalized Interactive Behavior to Maintain the Motivational Process 4.3 Hierarchical Rehabilitation Model to Guide Rehabilitation Behavior 5 Summary References Eating Experiences with Interactive Tableware for Improving Eating Behavior in Children 1 Introduction 2 Related Work 3 Concept Design 4 System Overview 5 User Study 6 Results 7 Discussion 8 Conclusion and Future Work References Sundial: Assist Children Learning Time with a Shining Ball Shaped Device 1 Introduction 2 Related Work 2.1 Development of the Concept of Time 2.2 Children’s Time Teaching 2.3 Tangible User Interface 3 Sundial—The Interactive Device Assists Children Understand Time 3.1 Overview 3.2 Design Process and Rationale 3.3 Use Case 3.4 The Prototype 4 User Study Setup 4.1 Study Design 4.2 Participants 4.3 Procedure 5 Discussion 5.1 The Use of Sundial 5.2 Sundial and the Periphery of the Attention 5.3 Findings 6 Conclusion 6.1 Limitations 6.2 Conclusion and Future Work References Scan Buddy: A Gamified App to Prepare Children for an MRI Scan 1 Introduction 2 Stakeholder Interviews and Parent Questionnaire 2.1 Methods 2.2 Results 3 Design of the Scan Buddy App 3.1 Learning Goals 3.2 Design Framework 3.3 Scan Buddy App Modules 4 Feasibility Study 4.1 Method 4.2 Results 4.3 Implications for Final App Design 5 Discussion 6 Conclusion References A Learning Support System of Acquiring Four Skills to Succeed a Fundamental Trick of Kendama Named as ``Around Japan'' 1 Introduction 2 Related Works 3 Four Fundamental Skills 4 Proposed System 4.1 System Configuration 4.2 Sequence of Learning Skills 5 Experiment 5.1 Purpose 5.2 Participants 5.3 Procedure 6 Result and Consideration 6.1 Hypothesis 1 6.2 Hypothesis 2 7 Conclusion References The Color Analysis of “Lolita” Fashion Style in Japan 1 Introduction 2 Research Purpose 3 Proposed Method 3.1 Overview 3.2 Colorfulness 3.3 Calculation of Dominant Colors 4 Result and Discussion 4.1 Colorfulness Comparison 4.2 Saturation and Brightness 4.3 Further Analysis 5 Conclusion References Author Index