Modeling Human Behaviors in Psychology Using Engineering Methods (River Publishers Series in Information Science and Technology)

Information science and technology enables 21st century into an Internet and multimedia era. Multimedia means the theory and application of filtering, coding, estimating, analyzing, detecting and recognizing, synthesizing, classifying, recording, and reproducing signals by digital and/or analog devices or techniques, while the scope of "signal" includes audio, video, speech, image, musical, multimedia, data/content, geophysical, sonar/radar, bio/medical, sensation, etc. Networking suggests transportation of such multimedia contents among nodes in communication and/or computer networks, to facilitate the ultimate Internet. Theory, technologies, protocols and standards, applications/ services, practice and implementation of wired/wireless networking are all within the scope of this series. We further extend the scope for 21st century life through the knowledge in robotics, machine learning, cognitive science, pattern recognition, quantum/biological/molecular computation and information processing, and applications to health and society advance. Cover Half Title Series Title Copyright Contents I Modeling Human Behaviors: An Engineering Approach 1 Behavioral Signal Processing (BSP): Behavioral Informatics 1.1 BSP: Introduction 1.1.1 BSP: Technical Challenges and Complexities 1.2 BSP: Computational Methods for Dyadic Interaction Dynamics 1.2.1 BSP: Further Complexities in Modeling Interaction Dynamics 2 Applications in Modeling Human Behaviors Computationally 2.1 BSP Application Domains 2.2 Case Study I: Emotion Recognition from Speech 2.3 Case Study II: Quantifying Implicit Vocal Entrainment 2.4 Case Study III: Data-driven Perceptual Experiment II Affective Computing from Speech 3 Individual Utterance Emotion Recognition 3.1 Introduction 3.2 Emotion Databases and Acoustic Feature Extraction 3.2.1 The AIBO Database 3.2.2 The USC IEMOCAP Database 3.2.3 Acoustic Feature Extraction 3.2.4 Feature Selection and Normalization 3.3 Emotion Classification Framework 3.3.1 Building the Hierarchical Decision Tree 3.3.2 Building the Hierarchical Decision Tree for the AIBO Database and the USC IEMOCAP Database 3.3.3 Classifier for Binary Classification Tasks 3.4 Emotion Recognition Experiment Setup and Results 3.4.1 The AIBO Database 3.4.2 The USC IEMOCAP Database 3.5 Conclusions and Future Work 4 Dialog-based Emotion Recognition 4.1 Introduction 4.2 Emotion Database and Annotation 4.2.1 The USC IEMOCAP Database 4.2.2 Emotion Annotation 4.3 Dynamic Bayesian Network Model 4.4 Experimental Results and Discussion 4.4.1 Acoustic Feature Extraction 4.4.2 Experiment Setup 4.4.3 Experiment Results and Discussion 4.5 Conclusions and Future Work III Quantifying Human Behavior in Psychology 5 Implicit Vocal Synchrony Quantification 5.1 Introduction 5.2 BSP Database: The Couple Therapy Corpus 5.2.1 Pre-processing and Audio Feature Extraction 5.2.2 Behavioral Codes of Interest 5.3 Signal-derived Vocal Entrainment Quantification 5.3.1 PCA-based Similarity Measures 5.3.2 Representative Vocal Features 5.3.3 Vocal Entrainment Measures in Dialogs 5.4 Analysis of Vocal Entrainment Measures 5.4.1 Natural Cohesiveness of Dialogs 5.4.2 Entrainment in Affective Interactions 5.5 Affect Classification using Entrainment Measures 5.5.1 Classification Framework 5.5.2 Classification Setup 5.5.3 Classification Results and Discussions 5.6 Conclusions and Future Work 6 Analysis of Vocal Synchrony in Couples Therapy 6.1 Introduction 6.2 BSP Database: The Couple Therapy Corpus 6.3 PCA-based Vocal Entrainment Measures 6.3.1 Symmetric Entrainment Measures 6.3.2 Directional Entrainment Measures 6.3.3 Canonical Correlation Analysis 6.4 Analysis of Results and Discussions 6.4.1 Correlation Analysis: The Four Behavioral Dimensions 6.4.2 Canonical Correlation Analysis: Withdrawal 6.5 Lessons Learnt from Correlation Analysis 6.6 Vocal Entrainment and Demand-and-Withdraw in Couple Conflict 6.6.1 Demand and Withdraw 6.6.2 Behavioral Influence and Polarization of Demand and Withdraw 6.6.3 Data Analysis 6.6.4 Results and Discussions IV Data-driven Perceptual Experiment 7 Multiple Instance Learning Framework for Perceptual Experiment 7.1 Introduction 7.2 BSP Database: The Couple Therapy Corpus 7.3 Computational Framework 7.3.1 Multiple Instance Learning 7.3.2 Sequential Probability Ratio Test 7.4 Analysis Setup 7.4.1 Lexical Feature Extraction 7.4.2 Classification Setup 7.5 Detection Results and Discussions 7.6 Isolated-Saliency vs. Causal-Integration 7.7 Conclusions and Future Work V Outlook of BSP 8 Continuously Emerging Importance of Modeling Human Behavior Bibliography Author Biography The main purpose of the work is to showcase the interdisciplinary engineering approaches in modeling and understanding human behaviors during interpersonal interactions those that could be typical, distressed, or atypical. The ability to measure human behaviors quantitatively has been a core component and a major research direction in both fields of engineering and psychology – though often with distinct approaches designed for different targeted applications. Engineering methods often strive to achieve high predictive accuracies using behavioral informatics techniques; these techniques employ a combination of behavior measures derived using automated signal based descriptors, and of statistical frameworks modeled using machine learning techniques. These approaches are often distinct from the observational approaches the gold standard for the past three decades in the study of psychology, even in clinical settings. The observational approaches are largely based on human subjective judgments.