Human-Computer Interaction – INTERACT 2021: 18th IFIP TC 13 International Conference, Bari, Italy, August 30 – September 3, 2021, Proceedings, Part III (Lecture Notes in Computer Science)

The five-volume set LNCS 12932-12936 constitutes the proceedings of the 18th IFIP TC 13 International Conference on Human-Computer Interaction, INTERACT 2021, held in Bari, Italy, in August/September 2021. The total of 105 full papers presented together with 72 short papers and 70 other papers in these books was carefully reviewed and selected from 680 submissions. The contributions are organized in topical sections named: Part I: affective computing; assistive technology for cognition and neurodevelopment disorders; assistive technology for mobility and rehabilitation; assistive technology for visually impaired; augmented reality; computer supported cooperative work. Part II: COVID-19 & HCI; croudsourcing methods in HCI; design for automotive interfaces; design methods; designing for smart devices & IoT; designing for the elderly and accessibility; education and HCI; experiencing sound and music technologies; explainable AI. Part III: games and gamification; gesture interaction; human-centered AI; human-centered development of sustainable technology; human-robot interaction; information visualization; interactive design and cultural development. Part IV: interaction techniques; interaction with conversational agents; interaction with mobile devices; methods for user studies; personalization and recommender systems; social networks and social media; tangible interaction; usable security. Part V: user studies; virtual reality; courses; industrial experiences; interactive demos; panels; posters; workshops. The chapter ‘Stress Out: Translating Real-World Stressors into Audio-Visual Stress Cues in VR for Police Training’ is open access under a CC BY 4.0 license at link.springer.com. The chapter ‘WhatsApp in Politics?! Collaborative Tools Shifting Boundaries’ is open access under a CC BY 4.0 license at link.springer.com. Welcome Preface IFIP TC13 -- http://ifip-tc13.org/ IFIP TC13 Members Conference Organizing Committee Contents – Part III Games and Gamification Diegetic and Non-diegetic Health Interfaces in VR Shooter Games 1 Introduction 2 Background 3 Study 3.1 VR Shooter Game 3.2 Health Interfaces 3.3 Study Design 3.4 Recruitment and Participants 3.5 Procedure 4 Results and Discussion 4.1 Overlay 4.2 Wristwatch 4.3 Physical 5 Conclusion and Future Work References Encouraging Chemistry Learning Through an Augmented Reality Magic Game 1 Introduction 2 Related Work 3 Magic Elements 3.1 Game Description 4 Evaluation 4.1 Participants 4.2 Results 5 Conclusions References Engagement and Discrete Emotions in Game Scenario: Is There a Relation Among Them? 1 Introduction 2 Emotion Recognition 3 Related Works 4 Methods 4.1 Study 1: Emotions and Engagement Analyzed by Software 4.2 Study 2: Emotions Analyzed by Software, and Engagement Self-assessed by Users 5 Results and Discussion 5.1 Study 1: Emotions and Engagement Analyzed by Software 5.2 Study 2: Emotions Measured by Software and Engagement Measured by Self-assessment 5.3 Overall Analysis 6 Final Remarks References Explore Data, Enjoy Yourself - KUbism, A Playful Approach to Data Exploration 1 Introduction 2 Related Work 2.1 Data Exploration and Fluid Interaction 2.2 Exploration in Videogames 2.3 Minecraft in Research 3 KUbism 3.1 Implementation and Architecture 3.2 Data Source Format 3.3 A Gameplay-Based Interface to Data Exploration 4 Methods 4.1 Experiment Data 4.2 Experiment World 4.3 Study Design 4.4 Subjective Experience Evaluation 5 Results 5.1 Demographics 5.2 Task Time 5.3 Task Activity 5.4 Subjective Experience Evaluation 6 Discussion 7 Conclusions and Future Work References Goalkeeper: A Zero-Sum Exergame for Motivating Physical Activity 1 Introduction 2 Related Work 2.1 Gamifying Physical Activity: The Case of Pokémon GO 2.2 Deposit Contracts and Physical Activity Motivation 2.3 Is Goalkeeper Betting? 3 The Goalkeeper App 4 Study 4.1 Participants and Groups 4.2 Procedure and Measures 5 Results 5.1 Deposit Contracts and Physical Exercise Motivation (RQ1) 5.2 Group Dynamics and Peer Competition (RQ2) 5.3 User Profile Effects (RQ3) 6 Discussion 6.1 Embrace Risk and Design for the Long-Run 6.2 Design for Peer Competition in Zero-Sum Exergame Settings 6.3 Design for the Group and the Individual 6.4 Limitations 7 Conclusion and Future Work References Graph-Based Method for the Interpretation of User Activities in Serious Games 1 Introduction: Player Evaluation in Serious Games 2 Bayesian Knowledge Tracing 3 Layered Graphs in the Description of Gameplay 3.1 User Decision as Graph Production 3.2 Gameplay Graph 4 Applying Graph-Based Method for the Interpretation of User Activities in Serious Games 5 Conclusion References Improving Student Experience and Learning Performance with Traditional Instructional Methods and New Digital Media 1 Introduction 2 Related Work 2.1 Instructional Media 2.2 Instructional Methods 3 Design of the Experiment 3.1 Software Development 3.2 The Learning Activity 4 Methodology 4.1 Hypotheses and Instruments 4.2 Procedure and Participants 5 Results 5.1 Demographic Data 5.2 Learning Performance 5.3 User Experience 6 Discussion 7 Conclusion References JoyFlick: Japanese Text Entry Using Dual Joysticks for Flick Input Users 1 Introduction 2 Japanese Writing System 3 Related Work 3.1 Phonetics-Based Kana Input 3.2 Text Entry with a Game Controller 3.3 Phonetics-Based Kana Input using a Game Controller 4 JoyFlick 4.1 Key Layout 4.2 Entry Method 4.3 Design Features 5 User Study 1 5.1 Participants 5.2 Apparatus 5.3 Task 5.4 Procedure 5.5 Results and Analysis 5.6 Discussion 6 User Study 2 6.1 Participants 6.2 Apparatus, Task, and Procedure 6.3 Results and Analysis 6.4 Discussion 7 Discussion and Future Work 7.1 Input Method Editor 7.2 User-Defined Key 7.3 Other Contexts 7.4 Combination of JoyFlick and Kana Syllabary Keyboard 8 Conclusion References MuseFlow: Facilitating Mind-Wandering Through Video Games 1 Introduction and Motivation 2 Related Work 2.1 Mind-Wandering 2.2 Incubation Effect 3 MuseFlow Game Design 4 Study Method and Design 4.1 Dependent Variables and Measures 4.2 Procedure 5 Results and Discussion 6 Limitations 7 Conclusion and Future Work References Gesture Interaction Fingerprint Scroll: Comparison of Touchless and Touch-Based Scroll Navigation Methods 1 Introduction 1.1 Input Styles for Mobile Devices 2 Related Work 3 Methodology 3.1 Participants 3.2 Apparatus 3.3 Procedure 3.4 Design 4 Results and Discussion 4.1 Completion Time 4.2 Error Percentage 4.3 Participant Feedback 5 Conclusion References Gesture Interaction in Virtual Reality 1 Introduction 2 Methods 3 Results 4 Discussion and Conclusions References Sticking Out Like a Non-dominant Thumb 1 Introduction 2 Background 2.1 Hand Usage in Touch Input 2.2 Fitts’ Law and Throughput 2.3 Laterality and Handedness in HCI 3 Study 3.1 Experiment 3.2 Results 4 Discussion 4.1 Thumb Performance of Left-Handed Participants 5 Conclusion References The Effect of Rhythm in Mid-air Gestures on the User Experience in Virtual Reality 1 Introduction 2 Experiment 3 Results 3.1 User Performances 3.2 Questionnaire 3.3 Discussion of Experiment Results 4 Conclusion References User-defined Bend Gesture Completion Strategies for Discrete and Continuous Inputs 1 Introduction and Background 2 Study on User-Defined Gesture Completion Strategies 2.1 Participants 2.2 Apparatus 2.3 Task 2.4 Procedure 3 Results and Discussion 3.1 Gesture Completion Strategies for Discrete Input 3.2 Gesture Completion Strategies for Continuous Input 4 Implications and Recommendations 5 Conclusion References Human-Centered AI Consumer Needs and Design Practices for Trusted Social Commerce Platforms 1 Introduction 2 Background 3 Methods 3.1 Study One 3.2 Study Two 4 Findings 4.1 News Feeds and Browsing (Social and Commercial Features and Functions – Ease and Difficulty) 4.2 Searching and Exploring 4.3 Sellers’ Practices 4.4 Product Information 4.5 Reviews and Feedback 4.6 Writing Reviews’ Issues and Motivations 4.7 Family and Friends 5 Discussion 5.1 Mixing and Simplifying Social and Commercial Features and Functions 5.2 Trust Activities and Mechanisms 6 Conclusion References DeepVANet: A Deep End-to-End Network for Multi-modal Emotion Recognition 1 Introduction 2 Multi-modal Emotion Recognition 2.1 Architecture 2.2 Training and Inference 3 Experiment 3.1 Quantitative Results 3.2 Qualitative Visualization 4 Conclusion References Detection of Subtle Stress Episodes During UX Evaluation: Assessing the Performance of the WESAD Bio-Signals Dataset 1 Introduction 2 Description of Datasets 2.1 WESAD Dataset 2.2 Ground Truth Dataset 3 Classifiers Creation: Training Process and Results 4 Evaluation of Classifiers 5 Conclusions, Limitations and Future Work References Emotion Elicitation with Stimuli Datasets in Automatic Affect Recognition Studies – Umbrella Review 1 Introduction 2 Affective Experiments Outline 2.1 Emotion Theories 2.2 Emotion Elicitation 2.3 Emotional Datasets 2.4 Emotion Elicitation Effectiveness 3 Reviews 4 Materials and Methods 4.1 Protocol, Search Strategies, Inclusion Criteria 4.2 Screening, Data Extraction, Quality Assessment 4.3 Analysis 5 Results 5.1 Study Selection 5.2 Characteristics of Included Studies 5.3 Quality of Included Studies 6 Discussion 6.1 Summary, Quality and Potential Biases in Results 6.2 Suggestions 6.3 Limitations and Strengths 7 Conclusions and Future Work References Detecting Emotions Through Machine Learning for Automatic UX Evaluation 1 Introduction 2 Background and Related Work 3 Construction of in-the-Wild Dataset 4 Comparison of Learning Algorithms for User’s Emotions Classification 5 Conclusion and Future Works References Helping Professionals Select Persona Interview Questions Using Natural Language Processing 1 Introduction 2 Related Literature 3 Methodology 3.1 Data Collection 3.2 Classifying the Questions 3.3 Clustering the Questions 4 Results 4.1 Use Case Definition 4.2 The Final Questions 4.3 Evaluation and Guidance for Application 5 Discussion 5.1 Research Contribution 5.2 Limitations, Future Research, and Theorization 6 Conclusion References ML Classification of Car Parking with Implicit Interaction on the Driver's Smartphone 1 Introduction 2 Related Work 3 User Interaction 4 Architecture and ML Model 4.1 Trimming 4.2 Heading Normalization 4.3 Acceleration and Rotation Rate 4.4 Model Creation 5 Results and Limitations 6 Conclusions and Ongoing Work References ObjectivAIze: Measuring Performance and Biases in Augmented Business Decision Systems 1 Introduction 2 Augmented Business Decision Making 3 Related Work 3.1 Decision Theory 3.2 Decision Support Systems 3.3 Impact of Algorithmic Decision Aids on Work Practices, and Ethical Considerations 4 A Performance Model for Decision-Making 4.1 Decision Aid Effectiveness and Collaboration 4.2 Automation Bias and Resistance 5 Study 5.1 Choice of a Decision Task to Evaluate 5.2 Stimulus Presentation 5.3 Second Run: Presentation Variants 5.4 Subjects Recruitment and Filtering 6 Results 6.1 Decision Aid Effectiveness 6.2 Quantification of the Automation Bias 6.3 Comparison Between Presentation Variants 6.4 Qualitative Feedback 7 Discussion 7.1 Limitations 7.2 Towards a Methodology and Embedded Measures of Performance in Augmented Decision Making 8 Conclusion References Service-Oriented Justification of Recommender System Suggestions 1 Introduction 2 Background and Related Work 2.1 Service Blueprints 2.2 Explaining and Justifying Recommendations 3 Methodology 3.1 Service-Based Representation of Airbnb Experiences 3.2 Analysis of Reviews 3.3 Visual Summary of User Feedback 3.4 Preliminary Test with Users in the Home Booking Domain 4 Conclusions and Future Work References Using the Design of Adversarial Chatbots as a Means to Expose Computer Science Students to the Importance of Ethics and Responsible Design of AI Technologies 1 Introduction 2 ``Responsible Design of AI'' Exemplified by Chatbots 2.1 Pedagogical Goals 2.2 Student Project 3 Outcome 3.1 Student Projects 3.2 Guidelines on Ethical Chatbots 4 Student Feedback 5 Conclusion References Human-Centred Development of Sustainable Technology DIY Homes: Placemaking in Rural Eco-Homes 1 Introduction 2 Related Work 2.1 Sustainable Placemaking 2.2 DIY in HCI 3 Methodology 3.1 Research Settings 3.2 Methods and Participants 4 Findings 4.1 Material-First Approach 4.2 Working with Nature 4.3 Sociality and Everydayness of Making 5 Discussion 5.1 DIY Home as Placemaking 5.2 Co-design with Non-humans 6 Conclusion References E-Scooter Sustainability – A Clash of Needs, Perspectives, and Experiences 1 Introduction 2 Related Work 2.1 Urban Sustainable Infrastructure and Micro-mobility 2.2 Mobility as a Service and Ownership 2.3 Hybrid Mobility 3 Study 3.1 Individual Interviews 3.2 In Situ Observations and Interviews 3.3 Social Media Comments 3.4 Data Analysis 4 Results 4.1 Mobility Needs 4.2 Environmental Motivations and Perceptions 4.3 Digital and Physical: Hybrid Mobility Experiences 4.4 Non-user Perceptions 5 Discussion 5.1 Green Mobility/Environmental Focus 5.2 Urban Infrastructure and Safety 5.3 Social and Economic Sustainability 5.4 Utilizing Hybridity to Regulate Riding 6 Conclusion References Objective Evaluation of Subjective Metrics for Interactive Decision-Making Tasks by Non-experts 1 Introduction 2 Related Work 2.1 Visualization Techniques for Optimization Problems 2.2 Evaluation of Tools for Multi-objective Choice Tasks 3 Mapping Subjective Metrics with Objective Measures Using Pragmatism: Comparing TOP-Slider (S4DM) with PSS (P4DM) as a Case Study 3.1 A 3-step Pragmatic Approach 3.2 Step 1: Profiling with a Preliminary Study 3.3 Step2: Metrics for Comparison 3.4 Step 3: Comparing Sliders4DM with P4DM 4 Experimental User Study 4.1 Apparatus 4.2 Participants 4.3 Decision Task 4.4 Procedure 5 Results 6 Discussion 6.1 Analysis 6.2 Summary of the Findings 6.3 Limitations and Caveats 7 Conclusion and Take-Away Message References Human-Robot Interaction Design Guidelines for Collaborative Industrial Robot User Interfaces 1 Introduction 2 Related Work 2.1 Interaction with Cobot Systems 2.2 Evaluating Cobots: Metrics, Measures, Methodologies 2.3 Design Guidelines and Heuristic Evaluation 3 Development of the Cobot UI Design Guidelines 3.1 Procedure for Establishing Cobot UI Guidelines 3.2 Literature Review with Database Search 3.3 Clustering of Guidelines Based on Affinity Diagramming 3.4 Revision Based on Modified Heuristic Evaluation 4 Evaluation Study Based on Modified Heuristic Evaluation 4.1 Procedure of the Evaluation Study 4.2 Participants 4.3 Evaluation Results 4.4 Revision of the Guidelines 5 UI/UX Design Expert Interviews 5.1 Participants 5.2 Procedure and Interview Protocol 5.3 Interview Themes 6 Proposed Set of Guidelines 6.1 Comparison with Existing Heuristics and Usability Guidelines 6.2 Validity and Generalizability of Guidelines 7 Conclusion References Intention Recognition in Human Robot Interaction Based on Eye Tracking 1 Introduction 2 State of the Art 3 Data Collection 4 Cascade Effect 5 Machine Learning 5.1 Accommodating the Data 5.2 Model Topology and Training Method 5.3 Testing the Model 6 Proposed Algorithm Combining both Methods 7 Conclusion References Interaction Initiation with a Museum Guide Robot—From the Lab into the Field 1 Introduction 2 Related Work 3 Lab Experiment 3.1 Results 3.2 Discussion 4 Field Study 4.1 Findings 4.2 Discussion 5 Conclusion References Recognition of a Robot's Affective Expressions Under Conditions with Limited Visibility 1 Introduction 2 Background and Related Work 3 Experiment 1 3.1 Methodology 4 Results - Experiment 1 5 Experiment 2 5.1 Methodology 6 Results - Experiment 2 7 Results - Combined 8 Discussion 9 Limitations and Future Work 10 Conclusion References ToolBot: Robotically Reproducing Handicraft 1 Introduction 2 Handicraft and Mass Fabrication 3 Related Work 3.1 Personal Fabrication 3.2 Assisted Handicraft 3.3 3D Photocopying 3.4 Painting Robots 3.5 Robot Programming by Demonstration 4 Methods 5 Implementation 6 The ToolBot System 6.1 Data Preprocessing 6.2 Data Editing 6.3 Data Validation 7 Scaling Tool or Workpiece 8 User Study 8.1 Procedure 8.2 Results 9 Discussion 10 Conclusion References Information Visualization An Immersive Approach Based on Two Levels of Interaction for Exploring Multiple Coordinated 3D Views 1 Introduction 2 Related Work 2.1 Multiple Views on Large Displays 2.2 Multiple Views in AR and VR Environments 3 The Spaces Approach 3.1 Spaces 3.2 Interaction Techniques 3.3 Implementation Details 4 Evaluation 4.1 Hypotheses 4.2 Use Case 4.3 Tasks 4.4 Training and Pilot Test 4.5 Experiment 5 Results 6 Discussion and Limitations 6.1 Findings 6.2 Limitations 7 Conclusions and Future Work References Directing and Combining Multiple Queries for Exploratory Search by Visual Interactive Intent Modeling 1 Introduction and Related Work 2 Directing and Combining Queries 3 Exploratory Search with Interactive Intent Modeling 3.1 Initializing the Search Stream 3.2 User Feedback 3.3 Intent Visualization 3.4 Document Retrieval 4 Operations on Exploratory Search Queries 4.1 Intersection Operation 4.2 Ranking for Intersecting Documents 4.3 Difference Operation 5 User Experiments 5.1 Task Design 5.2 Experiment Setup 5.3 Evaluation of Results 6 Conclusions and Discussion References Situated Visualization of Historical Timeline Data on Mobile Devices: Design Study for a Museum Application 1 Introduction 2 Background and Design Process 2.1 Method 2.2 Problem Analysis 3 Related Work 3.1 Mobile Devices in Museums 3.2 InfoVis on Mobile Devices and in Museums 4 Iteration 1: Visualization Concepts 4.1 Visualization Concepts 4.2 Evaluation 4.3 Results 5 Iteration 2: Linear Timelines 5.1 Visualization Concepts 5.2 Evaluation 5.3 Results 6 Reflections and Lessons Learned 7 Conclusion References Strategies for Detecting Difference in Map Line-Up Tasks 1 Introduction 2 Related Work 3 Study 3.1 Analytic Background and Study Aim 3.2 Study Conditions 3.3 Design and Tasks 3.4 Data Set 3.5 Participants and Procedure 4 Results 4.1 Participant Performance 4.2 Participant Preference 4.3 Participant Confidence 4.4 Strategies 5 Discussion 5.1 Strategies 5.2 Performance and Confidence 5.3 Map Design and Spatial Autocorrelation 5.4 Methodological Challenges 6 Limitations and Future Work 7 Conclusion References TEVisE: An Interactive Visual Analytics Tool to Explore Evolution of Keywords’ Relations in Tweet Data 1 Introduction 2 Related Work 3 TEVisE: Tweets Timeline Evolution Visual Explorer 3.1 The Enhanced Adjacency Matrix Diagram 3.2 The Timeline Evolution of Tweets 3.3 Interaction and Filtering Options 4 User Study 1: TEVisE vs. TExVis 4.1 Study Goals 4.2 Participants and Material 4.3 Tasks, Hypotheses, and Metrics 4.4 Usability Results 4.5 Cognitive Load Results 5 Study 2: TEVisE Exploratory Study Between the Summary View and the Timeline Evolution View 5.1 Results 6 Discussion 7 Conclusion and Future Work References Visualization of User’s Behavior in Indoor Virtual Environments Through Interactive Heatmaps 1 Introduction 2 Related Work 3 The Proposed System 4 Pilot Study 4.1 Objectives, Dataset and Participants 4.2 Procedure 4.3 Results 5 Discussion and Conclusions References VizInteract: Rapid Data Exploration Through Multi-touch Interaction with Multi-dimensional Visualizations 1 Introduction 2 Related Work 2.1 Ad-Hoc Visualization Construction 2.2 Touch-Based Interaction for Visualizations 2.3 Interaction with Multi-dimensional Data Visualizations 3 Design Goals 3.1 Rapid Multi-touch Composition and Editing of Visualizations 3.2 Easy-To-Learn Set of Interactions 3.3 Flexible yet Powerful Interface 4 VizInteract Interactions 4.1 Adding Dimensions 4.2 Changing the Orientation of a Histogram 4.3 Combining Views to Create Scatter Plots 4.4 Combining Views to Create Parallel Coordinate Plots 4.5 Combining Dimensions to Create Radar Charts 4.6 Filtering Operations 4.7 Combining Views to Create Scatter Plot Matrices 4.8 Canvas Operations 5 User Study I 5.1 Participants 5.2 Apparatus and Datasets 5.3 Study Design 5.4 Results 5.5 Discussion 6 User Study II 6.1 Apparatus and Participants 6.2 Study Design 6.3 Results 6.4 Discussion 6.5 Challenges 7 Conclusion and Future Work Appendix References Interaction Design and Cultural Development Blend in or Pop Out? Designing an Embedded Interface for a Historical Cemetery 1 Introduction 2 Related Work 2.1 Historical Landmarks and Technology 2.2 Interactivity at Cemeteries 2.3 Unobtrusive Design 3 Iterative Approach and Results 3.1 Brainstorming and Low-Fidelity Prototypes 3.2 Expert Interviews 3.3 In Situ Survey and Guerrilla Test 4 Discussion and Limitations 4.1 Limitations 4.2 Suitability of the Birdhouse-Inspired Prototype 4.3 Untouched Preservation Versus Encouraged Engagement 5 Summary, Conclusion and Next Steps References Cognitive Limitations of Older E-Commerce Customers in Product Comparison Tasks 1 Introduction 2 Related Work 2.1 Older Adults, E-Commerce, Multi-attribute Choice 2.2 Age-Related Cognitive and Motivational Changes 2.3 Strategies in Multi-attribute Choice Tasks 3 Experiment Design 3.1 Participants 3.2 Materials 3.3 Experiment Procedure 4 Results and Discussion 4.1 Age-Related Differences in Accuracy of Multi-attribute Choice Tasks 5 Conclusions References Multisensory Experiences for Art Appreciation 1 Introduction 1.1 Research Design 1.2 Designing Multisensory Experiences by Secondary Sense Isolation 2 Narrative Based Multisensory Experiences for Art Appreciation 3 Conclusion References Older Auctioneers: Performance of Older Users in On-Line Dutch Auctions 1 Introduction 2 Related Work 2.1 Older Adults, E-Commerce, and Dutch Auctions 2.2 Search Demands as Moderator of the Effect of Aging on Decision Making 2.3 Need for Cognitive Closure as Moderator of the Effect of Aging on Decision Making 3 Experiment Design 3.1 Participants 3.2 Used Questionnaires 3.3 Dutch Auction Task 3.4 Experiment Procedure 4 Results 5 Conclusions References Author Index