Affective Computing and Intelligent Interaction: Fourth International Conference, ACII 2011, Memphis, TN, USA, October 9-12, 2011, Proceedings, Part I (Lecture Notes in Computer Science, 6974)

This volume constitutes the refereed proceedings of the Fourth International Conference on Affective Computing and Intelligent Interaction, ACII 2011, held in Memphis, TN, USA, in October 2011. Title Preface Organization Table of Contents Invited Talks To Our Emotions, with Love: How Affective Should Affective Computing Be? Affect, Learning, and Delight Measuring Affect in the Wild Oral Presentations Affective Modeling from Multichannel Physiology: Analysis of Day Differences Introduction Day Differences and Non-stationarity of Affective Physiological Data Ensemble Approach for Classifying Physiological Data Measures, Methods, and Data Participants and Measures Procedure Feature Extraction and Classification Methods Results and Discussions Classification Results for Day Datasets Day Cross Validation Classification Results (Static Classifiers) Quantitative Analysis of Day Differences Winnow Algorithm Results Conclusions, Limitations and Future Work References The Dynamics between Student Affect and Behavior Occurring Outside of Educational Software Introduction Data Collection and Data Labeling Analysis and Results Overall Frequency of Behaviors and Affective States Transitions and Co-occurrences of Behaviors and Affective States Discussion and Conclusions References ikannotate – A Tool for Labelling, Transcription, and Annotation of Emotionally Coloured Speech Introduction Labelling Component Three Labelling Methods Temporal Characteristics of Emotion Labelling of Uncertainty Transcription and Annotation Transcription of Sentences Adding Annotation Features Output Formats XML File Format GAT Output Additional Tools PsychCreator ASRCreator Usability Tests Conclusion and Outlook References Being Happy, Healthy and Whole Watching Movies That Affect Our Emotions Introduction The Power of Emotions Emotional Impact of Videos and Movies Related Work The iFelt System User Study Conclusions and Perspectives References Investigating the Prosody and Voice Quality of Social Signals in Scenario Meetings Introduction Outline of the Methodology Acoustic Measures and Their Perceptual Correlates Perceptual Interpretation of Acoustic Measures Test-Running the Methodology on Controlled Data: NECA Vocal Effort Database Analysis of Meeting Data: AMI Meeting Corpus Dialogue Acts Analysis Speaker Roles Analysis Conclusions References Fast-FACS: A Computer-Assisted System to Increase Speed and Reliability of Manual FACS Coding Introduction Previous Work Facial Action Coding System (FACS) Automatic FACS Segmentation and Recognition from Video Fast-FACS Active Appearance Tracking Peak, Onset, and Offset Coding Learning a Metric for Onset/Offset Detection Graphical User Interface (GUI) Experimental Evaluations Inter-coder and Inter-system Agreement Accuracy of Estimated Onsets and Offsets Efficiency of Fast-FACS References A Computer Model of the Interpersonal Effect of Emotion Displayed in a Social Dilemma Introduction Background Models Evaluation Discussion References Agents with Emotional Intelligence for Storytelling Introduction Emotional Intelligence Related Work Conceptual Model Scenarios Summary and Future Work References “That’s Aggravating, Very Aggravating”: Is It Possible to Classify Behaviors in Couple Interactions Using Automatically Derived Lexical Features? Introduction The Corpus Methodology and Results Maximum Likelihood Classifier Classification on Reference Transcripts Audio Segmentation Classification on Audio Signal Lexical Significance Discussion References Predicting Facial Indicators of Confusion with Hidden Markov Models Introduction Related Work Corpus and Manual Annotations HMM Learning and Prediction of Student AU4 Model Learning Prediction of AU4 Discussion Hidden State Structure Limitations Conclusion References Recording Affect in the Field: Towards Methods and Metrics for Improving Ground Truth Labels Introduction Related Work Background Study Design Study Protocol Journaling Training Analysis Analysis of the Three Axis “In situ” Emotion Scores Improving Ground Truth through Triangulation of “In situ” and “End of Day” Ratings “In situ” and “End-of-Day” Combined Rating Analysis Extended Analysis: Assessing Evidence via Third Party Independent Raters Extended Analysis: Combining Third Party and Subject Ratings Conclusions References Using Individual Light Rigs to Control the Perception of a Virtual Character’s Personality Introduction Related Work Individual Lighting Rigs Our Approach towards Lighting User Study Materials and Method Results Discussion and Conclusion References Call Center Stress Recognition with Person-Specific Models Introduction Background and Previous Work Physiological Stress and Skin Conductance Automatic Stress Recognition Study Design Proposed Method Support Vector Machines Changing Class Priors Selecting Training Samples Experimental Setting Results Conclusions References Are You Friendly or Just Polite? – Analysis of Smiles in Spontaneous Face-to-Face Interactions Introduction Spontaneous Face-to-Face Banking Dataset Participants Experimental Setup Facial Expressions Coding Research Questions Experiments Smile Annotation and Segmentation Duration and Timing Smile Dynamics: Rise, Sustain and Decay Velocity of Rise, Sustain and Decay Discussion and Conclusions References Multiple Instance Learning for Classification of Human Behavior Observations Introduction Proposed Approach Corpus MIL Using Diverse Density SVMs Feature Extraction Experiments Conclusions and Future Work References Form as a Cue in the Automatic Recognition of Non-acted Affective Body Expressions Introduction Related Work The Affective Posture Recognition System Data Preparation Building the Training Set Building the Testing Set Low-Level Posture Description Testing Results Discussion and Conclusions References Design of a Virtual Reality Based Adaptive Response Technology for Children with Autism Spectrum Disorder Introduction System Design VR-Based Social Communication Task Module Real-Time Eye-Gaze Monitoring Module Individualized Adaptive Response Module Methods Participants Procedure Results Progression of VR-Based Social Communication Tasks Improvement in Behavioral Viewing Pattern Conclusion References Exploring the Relationship between Novice Programmer Confusion and Achievement Introduction Data Collection Data Labeling Model Construction and Data Relabeling Results and Discussion Conclusion References Semi-Coupled Hidden Markov Model with State-Based Alignment Strategy for Audio-Visual Emotion Recognition Introduction Feature Extraction Facial Feature Extraction Prosodic Feature Extraction Semi-Coupled Hidden Markov Model Model Formulation State-Based Bi-modal Alignment Strategy Experimental Result Conclusion References Associating Textual Features with Visual Ones to Improve Affective Image Classification Introduction Features for Emotional Semantics Textual Features Visual Features Experiments and Results Emotion Models Datasets Experimental Setup Results: Performance of Different Features Results: Performance of Different Combination Approaches on IAPS Results: Performance of Textual Features on Mirflickr2000 Conclusion References 3D Corpus of Spontaneous Complex Mental States Introduction Motivation Contributions Related Work Corpus Methodology Data Acquisition Segmentation and Annotation Data Analysis Validation Analysis of Hand-Over-Face Gestures Discussion References Evaluating the Communication of Emotion via Expressive Gesture Copying Behaviour in an Embodied Humanoid Agent Introduction Related Work System Overview Video Analysis Expressive Behaviour Generation Movement Expressivity Mapping Experiment Overview Materials Procedure Results Discussion Conclusion References Multi-score Learning for Affect Recognition: The Case of Body Postures Introduction Multi-score Learning and Its Measurements Learning Methods for Multi-score Learning K-Nearest Neighbour Regression Artificial Neural Networks Multi-task Learning Multi-score Learning on Affective Posture: Results Conclusions References Multi-modal Affect Induction for Affective Brain-Computer Interfaces Introduction Methods Participants Apparatus Stimuli Design and Procedure Data Processing and Analysis Results Subjective Ratings Heart Rate EEG Discussion Conclusion References Toward a Computational Framework of Suspense and Dramatic Arc Introduction Related Work Dramatis Framework Reading a Story Element Searching for Failure Computing Likelihood of Escape Updating Suspense Knowledge Structure Example Discussion and Future Work References A Generic Emotional Contagion Computational Model Introduction Background Emotional Contagion (EC) Emotional Contagion Biases Related Work A Model for Emotional Contagion Contagion Filter Mood Updater Emotional Expression Filter Model Simulations Conclusions and Future Work References Generic Physiological Features as Predictors of Player Experience Introduction Related Work Data Collection Games Participants Experimental Protocol Physiological Signals and Extracted Features Method Sequential Forward Feature Selection Neuroevolutionary Preference Learning Experiments Heart Rate Features Skin Conductance Features Conclusions References Guess What? A Game for Affective Annotation of Video Using Crowd Sourcing Introduction The Game Overview Game Play and Scoring Implementation Details Evaluation Data Labeling Pilot Main Experiment Results Discussion and Future Work References Modeling Learner Affect with Theoretically Grounded Dynamic Bayesian Networks Introduction Background Method Results Predictive Modeling Bayesian Networks Conclusion References Evaluations of Piezo Actuated Haptic Stimulations Introduction Methods Participants Apparatus Stimuli Procedure Data Analysis Results Discussion References The Relationship between Carelessness and Affect in a Cognitive Tutor Introduction Methods The Scatterplot Tutor Affect Observations Carelessness Detection Results Discussion and Conclusion References EmotionML – An Upcoming Standard for Representing Emotions and Related States Introduction Previous Work Syntax Design Principles: Self-contained Emotion Annotation Representations of Emotion Mechanism for Referring to an Emotion Vocabulary Meta-information References to the “Rest of the World” Time Representing Continuous Values and Dynamic Changes Vocabularies for EmotionML Conclusion and Future Work References Emotion-Based Intrinsic Motivation for Reinforcement Learning Agents Introduction Background and Related Work Intrinsically Motivated Learning Emotionally Motivated Learning Agents Affective Reward Features Experiments and Results Discussion References The Good, the Bad and the Neutral: Affective Profile in Dialog System-User Communication Introduction Relevant Work Dialog System Architecture Affective Profiles for a Dialog System Experimental Method Experimental Results Conclusions References Effect of Affective Profile on Communication Patterns and Affective Expressions in Interactions with a Dialog System Introduction Relevant Research Experimental Settings Overview of Dialog System Architecture Characteristic of the Participants Experimental Procedure Analysed Data-Set Applied Annotation Tools and Resources Effects of Affective Profile on Users' Communication Style Effect of Affective Profile on Users' Expression of Affective States Discussion and Outlook References Persuasive Language and Virality in Social Networks Introduction Virality as a Content Dependent Phenomenon Virality as a Multifaceted Phenomenon Dataset and Virality Metrics Experiments Discussion Dominant Word Classes in Viral Texts Related Work Conclusions References A Multimodal Database for Mimicry Analysis Introduction Mimicry Perception Conversation Recording Interaction Scenario Procedure Self-report of Participants Synchronized Multi-sensor Recording Setup Audio Channels Camera Views Audio/Video Synchronization Annotation Segmentation into Episodes of Interest Annotation within Segments Overview and Availability Conclusion and Future Work References Mood Recognition Based on Upper Body Posture and Movement Features Introduction Coding Body Posture and Movement Research Questions and Methodology Data Analysis Results Discussion References Emotional Aware Clustering on Micro-blogging Sources Introduction Related Work Problem Formulation The SentiTweetAlgo Experimentation Conclusions and Future Work References A Phonetic Analysis of Natural Laughter, for Use in Automatic Laughter Processing Systems Motivation and Related Work Annotation Scheme Laughter Phonetic Description Interpersonal Differences Phones Used Phone and Airflow Phases Duration Conclusion and Further Work References The Impact of Music on Affect during Anger Inducing Drives Introduction Method Participants Design Music Selection Anger Drive Dependent Variables Procedure Results Subjective Self-report Autonomic Variables Discussion and Conclusion References Unsupervised Temporal Segmentation of Talking Faces Using Visual Cues to Improve Emotion Recognition Introduction Proposed Method Histogram of Local Binary Patterns Integrated Gradient Histogram Time Series Data Segmentation Methods to Improve Emotion Recognition Experiments and Results Database Results Conclusions References The Affective Experience of Handling Digital Fabrics: Tactile and Visual Cross-Modal Effects Introduction Background: The Affective Experience of Handling Fabrics Experiment Protocol Analysis Results Conclusions References Ranking vs. Preference: A Comparative Study of Self-reporting Introduction Self-reporting Ranking Preference User Survey Case Studies System Instrumentation Self-reported Post-Experience The Playware Case Study The Maze-Ball Case Study Case Study Dissimilarities Results and Analysis H1 Test Statistic H2 Test Statistics Playware Maze-Ball Discussion References Poster Papers Towards a Generic Framework for Automatic Measurements of Web Usability Using Affective Computing Techniques Introduction Background Measuring Usability Proposed Framework Method and Data Primary Results and Discussion Conclusion and Future Work References Simulating Affective Behaviours : An Approach Based on the COR Theory Introduction Related Work Proposed Model Principle of the Model Definitions Architecture Needs and Behaviour Selection Computation of Behaviour's Payoff Individual Parameters Influencing Behaviours' Payoff Behaviour Selection and Need Value Update Directions for Evaluation Procedures Discussion and Future Work References Emotional Investment in Naturalistic Data Collection Introduction Simulated Car Driving Experimental Setup Subjects Procedure Discussion Remotely-Controlled Flying Experimental Setup Subjects Procedure Data Analysis Discussion Recommendations Location Scenario Secondary Task Data Collection and Analysis Conclusion References When Do We Smile? Analysis and Modeling of the Nonverbal Context of Listener Smiles in Conversation Introduction Nonverbal Context of Smiles in Human-Human Conversation Corpus Categories of Smiles The Nonverbal Context of the Smile Categories Discussion Predicting the Timing of Smiles in Human-Human Conversation Approach Results and Discussion Conclusions References Emotional Cognitive Architectures Introduction Cognitive Criteria Meet Emotions Emotions and Emotional Learning Mechanisms in CELTS CELTS' Long Route (LR) CELTS' Short Route (SR) Expectation Codelets Is CELTS an Emotional Cognitive Architecture? Conclusion References Kalman Filter-Based Facial Emotional Expression Recognition Introduction Facial Features for Emotion Recognition State-Space Model For Emotion Recognition Model Overview Emotion Recognition Model Parameters Estimation Experiment Results and Evaluation Material Dimensionality of the Facial Emotional Expression Hidden States Recognition Performance Conclusion and Discussion References SARA: Social Affective Relational Agent: A Study on the Role of Empathy in Artificial Social Agents Introduction Related Work A Model for Empathy Regarding Social Penetration Empathic Appraisal Empathic Response User-Agent Interaction Evaluation Test Scenarios Design and Procedure Results Conclusions and Future Work References Learning General Preference Models from Physiological Responses in Video Games: How Complex Is It? Introduction Case Studies MazeBall Dataset TORCS Dataset Physiological Features Cross Validation Affective Preference Model Linear Model LDA Non-linear Model Neuro Evolution Feature Selection Sequential Forward Selection Genetic Feature Selection Results and Conclusion References Towards Real-Time Affect Detection Based on Sample Entropy Analysis of Expressive Gesture Introduction Background Bounding Triangle Dynamic Expressive Behavior Analysis Entropy Measure Realtime Implementation Extracting Bounding Triangle Form Human Silhouette Extracting Dynamic Features: Smoothness and Symmetry Index Computing SampEn Output Conclusion References Predicting Learner Engagement during Well-Defined and Ill-Defined Computer-Based Intercultural Interactions Introduction The Link of Engagement to Learning Methodology Procedure Dependent Measures Results Discussion Conclusion and Recommended Future Research References Context-Independent Facial Action Unit Recognition Using Shape and Gabor Phase Information Introduction Feature Extraction Geometry-Based Features Appearance-Based Features Combination of Geometry-Based and Appearance-Based Features Feature Selection and Classification Experiments Conclusion References Conveying Emotion with Moving Images: Relationship between Movement and Emotion Introduction Related Work MPM Moving Effects of MPM Moving Effect Elements (Vertex Noise) Experiment Still Image Moving Effect Sound Data Moving Image Procedure Results Difference in Emotion Conveyed among Still Images Movement in Conveying Emotion Emotion Discussion Role of Elements Emotion Future Works References Hybrid Fusion Approach for Detecting Affects from Multichannel Physiology Introduction Background Multimodal Information Fusion Information Fusion in Affective Computing Data Collection: Participants, Sensors, Procedure Computational Model Feature Extraction, Selection and Classification Hybrid Information Fusion Results and Discussions Conclusion and Future Works References Investigating the Suitability of Social Robots for the Wellbeing of the Elderly Introduction Literature Review Focus Groups and Home Studies Results Conclusion References The Effects of Emotionally Worded Synthesized Speech on the Ratings of Emotions and Voice Quality Introduction Methods Participants Equipment and Physiological Measurement Stimuli Experimental Procedure Data Preprocessing and Analysis Results Ratings of Emotional Experiences Ratings of Voice Quality Electromyographic Responses Discussion References Evaluating a Cognitive-Based Affective Student Model Introduction Modelling User’s Emotions Emotions’ Inference Experiment Protocol Evaluation Results Discussion and Conclusions References Audio Visual Emotion Recognition Based on Triple-Stream Dynamic Bayesian Network Models Introduction Audio Visual Feature Extraction Audio Features Visual Features Triple Stream Asynchronous DBN (T_AsyDBN) Model Experiments and Results Conclusions and Future Work References Author Index Blank Page The two-volume set LNCS 6974 and LNCS 6975 constitutes the refereed proceedings of the Fourth International Conference on Affective Computing and Intelligent Interaction, ACII 2011, held in Memphis,TN, USA, in October 2011. The 135 papers in this two volume set presented together with 3 invited talks were carefully reviewed and selected from 196 submissions. The papers are organized in topical sections on recognition and synthesis of human affect, affect-sensitive applications, methodological issues in affective computing, affective and social robotics, affective and behavioral interfaces, relevant insights from psychology, affective databases, Evaluation and annotation tools. The two-volume set LNCS 6974 and LNCS 6975 constitutes the refereed proceedings of the Fourth International Conference on Affective Computing and Intelligent Interaction, ACII 2011, held in Memphis, TN, USA, in October 2011. The 135 papers in this two volume set presented together with 3 invited talks were carefully reviewed and selected from 196 submissions. The papers are organized in topical sections on recognition and synthesis of human affect, affect-sensitive applications, methodological issues in affective computing, affective and social robotics, affective and behavioral interfaces, relevant insights from psychology, affective databases, Evaluation and annotation tools Edited By Sidney D’mello, Arthur Graesser, Björn Schuller, Jean-claude Martin.