Personality in Speech: Assessment and Automatic Classification (T-Labs Series in Telecommunication Services)

This work combines interdisciplinary knowledge and experience from research fields of psychology, linguistics, audio-processing, machine learning, and computer science. The work systematically explores a novel research topic devoted to automated modeling of personality expression from speech. For this aim, it introduces a novel personality assessment questionnaire and presents the results of extensive labeling sessions to annotate the speech data with personality assessments. It provides estimates of the Big 5 personality traits, i.e. openness, conscientiousness, extroversion, agreeableness, and neuroticism. Based on a database built on the questionnaire, the book presents models to tell apart different personality types or classes from speech automatically. Preface 6 Contents 8 Introduction 11 1 Personality Assessment in Psychology 15 1.1 Definitions of Personality 15 1.2 Trait Theory of Personality 17 1.2.1 Allport's Trait Organization 17 1.2.2 Eysenck's P-E-N Theory of Super Factors 18 1.2.3 Cattell's 16 Personality Source Traits 19 1.2.4 Development of the ``Big 5'' Factor Inventories 22 1.2.5 Costa and McCrae and the ``NEO-'' Inventories 23 1.2.6 The Development of the German NEO-FFI 26 1.2.7 Super Short Forms of Five Factor Inventories 27 1.2.8 Trait Theories Comparison and Criticism 28 1.3 Summary 30 References 31 2 Speech-Based Personality Assessment 34 2.1 Contemporary Terminology in Speech Analysis 37 2.1.1 Prosodic Information and their Temporal Expansion 37 2.1.2 Extralinguistic and Paralinguistic Information 38 2.2 Vocal Cues of Personality Perception 38 2.2.1 Linking Perceptual Speech Properties and Personality 39 2.2.2 Correlation between Perceptual Personality and Acoustic Measurements 41 2.2.3 Work From Speech Synthesis 42 2.2.4 Signal-Based Automated Personality Modeling 44 2.2.5 Other Related Studies 46 2.2.6 Own Prior Work 47 2.3 Chapter Summary 48 References 50 3 Database and Labeling 55 3.1 Text-Dependent Data Recordings 56 3.2 Text-Independent Data Recordings 57 3.3 Multi-Speaker Data Recordings 60 3.4 Annotating the Recordings with Personality Ratings 61 3.5 Summary 63 References 65 4 Analysis of Human Personality Perception 66 4.1 Auditory Impressions and Personality Perception Hypotheses 67 4.2 Distributions and Correlation Analysis of Rating Responses 70 4.3 Factor Analyzing the NEO-FFI Item Responses 79 4.4 Analyses of Variance 82 4.5 Summary 89 References 90 5 Automatic Personality Estimation 92 5.1 Automatic Extraction of Personality Cues from Speech 93 5.1.1 Intensity 93 5.1.2 Pitch 94 5.1.3 Spectrals 96 5.1.4 Loudness 97 5.1.5 MFCC 97 5.1.6 Formants 98 5.1.7 Other Descriptors 99 5.2 Voice Activity Detection and Segmentation 99 5.3 Feature Definition 100 5.3.1 Intensity 101 5.3.2 Pitch 103 5.3.3 Spectrals 105 5.3.4 Loudness 106 5.3.5 MFCC 106 5.3.6 Formants 106 5.3.7 Other Features 106 5.4 Feature Selection 107 5.5 Normalization 110 5.6 Modeling for Personality Classification and Trait Score Prediction using SVM 111 5.6.1 Personality Classification Using SVM 112 5.6.2 Trait Score Prediction Using SVM 114 5.6.3 Non-Linear Mapping and Parameter Tuning 116 5.7 Evaluation 118 5.7.1 Evaluation Metrics 118 5.7.2 Evaluation Method 120 5.8 Results from Text-Dependent Data 121 5.8.1 Results from Automatic Classification 121 5.8.2 Results from Automatic Trait Score Prediction 125 5.9 Results from Text-Independent Data 131 5.9.1 Results from Automatic Classification 131 5.9.2 Results from Automatic Trait Score Prediction 137 5.10 Results from Multi-Speaker Data 139 5.10.1 Results from Automatic Trait Score Prediction 141 References 148 6 Discussion of the Results 150 6.1 Results from Classification of Personality Targets 150 6.2 Prediction of Individual Traits Scores from Speech 154 6.2.1 Prediction of Openness 155 6.2.2 Prediction of Conscientiousness 157 6.2.3 Prediction of Extroversion 158 6.2.4 Prediction of Agreeableness 160 6.2.5 Prediction of Neuroticism 162 6.3 Discussion of Influencing Factors 164 6.3.1 Design of Speech Database 164 6.3.2 Signal-Based Feature Extraction and Model Set-up 170 References 173 7 Conclusion and Outlook 174 7.1 Contributions and Conclusions 174 7.2 Outlook 179 References 183 Appendix A Label Distributions Text-DependentRecordings 184 Appendix B Label Distributions Text-IndependentRecordings 186 Front Matter....Pages i-xiv Personality Assessment in Psychology....Pages 1-19 Speech-Based Personality Assessment....Pages 21-41 Database and Labeling....Pages 43-53 Analysis of Human Personality Perception....Pages 55-80 Automatic Personality Estimation....Pages 81-138 Discussion of the Results....Pages 139-162 Conclusion and Outlook....Pages 163-172 Back Matter....Pages 173-176