Sequential Analysis and Observational Methods for the Behavioral Sciences

Behavioral scientists – including those in psychology, infant and child development, education, animal behavior, marketing, and usability studies – use many methods to measure behavior. Systematic observation is used to study relatively natural, spontaneous behavior as it unfolds sequentially in time. This book emphasizes digital means to record and code such behavior; while observational methods do not require them, they work better with them. Key topics include devising coding schemes, training observers, and assessing reliability, as well as recording, representing, and analyzing observational data. In clear and straightforward language, this book provides a thorough grounding in observational methods along with considerable practical advice. It describes standard conventions for sequential data and details how to perform sequential analysis with a computer program developed by the authors. The book is rich with examples of coding schemes and different approaches to sequential analysis, including both statistical and graphical means. CONTENTS 6 FIGURES 10 PREFACE 14 1 Introduction to Observational Methods 18 SYSTEMATIC QUANTITATIVE MEASUREMENT VERSUS QUALITATIVE NARRATIVE 18 CORRELATIONAL VERSUS EXPERIMENTAL DESIGNS 20 PREDICTOR VERSUS OUTCOME VARIABLES 21 VARIABLES, UNITS, AND SESSIONS 21 WHY USE OBSERVATIONAL METHODS? 23 SEQUENTIAL ANALYSIS OF BEHAVIOR 24 SUMMARY 28 2 Coding Schemes and Observational Measurement 30 WHERE DO CODING SCHEMES COME FROM? 30 MUST CODES BE MUTUALLY EXCLUSIVE AND EXHAUSTIVE? 31 GRANULARITY: MICRO TO MACRO 35 CONCRETENESS: PHYSICALLY TO SOCIALLY BASED CODES 36 CODES VERSUS RATING SCALES 38 THE CODING MANUAL 39 SUMMARY 41 3 Recording Observational Data 43 UNTIMED-EVENT RECORDING 45 TIMED-EVENT RECORDING 46 INTERVAL RECORDING 47 Partial-Interval or One-Zero Sampling 49 Momentary or Instantaneous Sampling 49 Whole-Interval Sampling 49 SELECTED-INTERVAL RECORDING 51 LIVE OBSERVATION VERSUS RECORDED BEHAVIOR 52 DIGITAL RECORDING AND COMPUTER-ASSISTED CODING 54 SUMMARY 57 4 Representing Observational Data 60 A SEQUENTIAL DATA INTERCHANGE STANDARD (SDIS) 60 REPRESENTING TIME 61 SINGLE-CODE EVENT SEQUENCES 63 TIMED-EVENT AND STATE SEQUENCES 65 INTERVAL AND MULTICODE EVENT SEQUENCES 67 A UNIVERSAL CODE-UNIT GRID 68 ALTERNATIVES: SPREADSHEET AND STATISTICAL PACKAGE GRIDS 70 DATA MANAGEMENT AND FILE FORMATS 71 SUMMARY 72 5 Observer Agreement and Cohen’s Kappa 74 POINT-BY-POINT VERSUS SUMMARY AGREEMENT 75 THE CLASSIC COHEN’S KAPPA 76 WHEN IS KAPPA BIG ENOUGH? 79 Is Statistical Significance Useful? 80 Observer Bias and Kappa Maximum 81 Observer Accuracy, Number of Codes, and Their Prevalence 82 Standards for Kappa (Number of Codes Matters) 83 COMPARING AN OBSERVER WITH A GOLD STANDARD 85 Agreement and Reliability 86 Errors of Commission and Omission 86 SUMMARY 87 6 Kappas for Point-by-Point Agreement 89 EVENT-BASED AGREEMENT: THE ALIGNMENT PROBLEM 89 TIME-BASED AGREEMENT: INFLATED COUNTS? 94 EVENT-BASED AGREEMENT FOR TIMED-EVENT SEQUENCES 95 INTERVAL-BASED AGREEMENT USING COHEN’S KAPPA 98 WEIGHTED KAPPA: WHEN DISAGREEMENTS DIFFER IN SEVERITY 98 ARE ALL KAPPAS OVERRATED? 100 SUMMARY 101 7 The Intraclass Correlation Coefficient (ICC) for Summary Measures 104 RELATIVE VERSUS ABSOLUTE AGREEMENT 104 TARGETS AND SESSIONS 105 RELATIVE AND ABSOLUTE ICCS 106 SUMMARY 109 8 Summary Statistics for Individual Codes 110 BASIC STATISTICS FOR INDIVIDUAL CODES 112 Frequency 112 Relative frequency 113 Rate 113 Duration 114 Relative duration 115 Probability 115 MEAN EVENT DURATIONS, GAPS, AND LATENCIES 116 Mean Event Duration 117 Mean Gap 117 Latency 117 RECOMMENDED STATISTICS FOR INDIVIDUAL CODES 118 SUMMARY 119 9 Cell and Summary Statistics for Contingency Tables 121 INDIVIDUAL CELL STATISTICS 122 Observed Joint Frequencies and Hierarchical Tallying 122 Lagged Tallies for Single-Coded Events When Codes Can and Cannot Repeat 124 Conditional and Transitional Probabilities 125 Expected Frequencies and Adjusted Residuals 126 INDICES OF ASSOCIATION FOR TWO-DIMENSIONAL TABLES 128 CONTINGENCY INDICES FOR 2 × 2 TABLES 128 Odds Ratio and Log Odds 129 Yule’s Q 131 Vulnerability to Zero Cells 132 SUMMARY 133 10 Preparing for Sequential and Other Analyses 135 CREATING NEW CODES FROM EXISTING CODES 135 Logical Combinations for Timed-Event, Interval, and Multicode Event Data 136 RECODE for All Data Types 137 EVENT and BOUT for Timed-Event Data 138 RECODE, LUMP, and CHAIN for Single-Code Event Data 139 REMOVE and RENAME for All Data Types 140 CREATING NEW CODES AS “WINDOWS” ANCHORED TO EXISTING CODES 141 POOLED VERSUS INDIVIDUAL ANALYSES 142 PREPARING EXPORT FILES AND USING STATISTICAL PACKAGES 143 DEVIANT CELLS, TYPE I ERROR, AND WINNOWING 145 SUMMARY 148 11 Time-Window and Log-Linear Sequential Analysis 151 TIME-WINDOW SEQUENTIAL ANALYSIS OF TIMED-EVENT DATA 152 THE SIGN TEST: A NONPARAMETRIC ALTERNATIVE 154 LAG-SEQUENTIAL AND LOG-LINEAR ANALYSIS OF SINGLE-CODE EVENT DATA 155 Overlapped and Nonoverlapped Tallying of m-Event Chains 156 An Illustration of Log-Linear Basics 158 LOG-LINEAR ANALYSIS OF INTERVAL AND MULTICODE EVENT DATA 161 SUMMARY 163 12 Recurrence Analysis and Permutation Tests 165 RECURRENCE ANALYSIS 165 PERMUTATION TESTS FOR SHORT EVENT SEQUENCES 173 SUMMARY 177 EPILOGUE 180 APPENDIX A Expected Values for Kappa Comparing Two Observers 182 APPENDIX B Expected Values for Kappa Comparing with a Gold Standard 184 REFERENCES 186 INDEX 196 Contents Note Continued: 4. Representing Observational Data -- A Sequential Data Interchange Standard (sdis) -- Representing Time -- Single-code Event Sequences -- Timed-event And State Sequences -- Interval And Multicode Event Sequences -- A Universal Code-unit Grid -- Alternatives: Spreadsheet And Statistical Package Grids -- Data Management And File Formats -- Summary -- 5. Observer Agreement And Cohen's Kappa -- Point-by-point Versus Summary Agreement -- The Classic Cohen's Kappa -- When Is Kappa Big Enough? -- Is Statistical Significance Useful? -- Observer Bias And Kappa Maximum -- Observer Accuracy, Number Of Codes, And Their Prevalence -- Standards For Kappa (number Of Codes Matters) -- Comparing An Observer With A Gold Standard -- Agreement And Reliability -- Errors Of Commission And Omission -- Summary -- 6. Kappas For Point-by-point Agreement -- Event-based Agreement: The Alignment Problem -- Time-based Agreement: Inflated Counts? Contents Note Continued: Event-based Agreement For Timed-event Sequences -- Interval-based Agreement Using Cohen's Kappa -- Weighted Kappa: When Disagreements Differ In Severity -- Are All Kappas Overrated? -- Summary -- 7. The Intraclass Correlation Coefficient (icc) For Summary Measures -- Relative Versus Absolute Agreement -- Targets And Sessions -- Relative And Absolute Iccs -- Summary -- 8. Summary Statistics For Individual Codes -- Basic Statistics For Individual Codes -- Frequency -- Relative Frequency -- Rate -- Duration -- Relative Duration -- Probability -- Mean Event Durations, Gaps, And Latencies -- Mean Event Duration -- Mean Gap -- Latency -- Recommended Statistics For Individual Codes -- Summary -- 9. Cell And Summary Statistics For Contingency Tables -- Individual Cell Statistics -- Observed Joint Frequencies And Hierarchical Tallying -- Lagged Tallies For Single-coded Events When Codes Can And Cannot Repeat -- Conditional And Transitional Probabilities. Contents Note Continued: Expected Frequencies And Adjusted Residuals -- Indexes Of Association For Two-dimensional Tables -- Contingency Indexes For 2x2 Tables -- Odds Ratio And Log Odds -- Yule's Q -- Vulnerability To Zero Cells -- Summary -- 10. Preparing For Sequential And Other Analyses -- Creating New Codes From Existing Codes -- Logical Combinations For Timed-event, Interval, And Multicode Event Data -- Recode For All Data Types -- Event And Bout For Timed-event Data -- Recode, Lump, And Chain For Single-code Event Data -- Remove And Rename For All Data Types -- Creating New Codes As Windows Anchored To Existing Codes -- Pooled Versus Individual Analyses -- Preparing Export Files And Using Statistical Packages -- Deviant Cells, Type I Error, And Winnowing -- Summary -- 11. Time-window And Log-linear Sequential Analysis -- Time-window Sequential Analysis Of Timed-event Data -- The Sign Test: A Nonparametric Alternative. Contents Note Continued: Lag-sequential And Log-linear Analysis Of Single-code Event Data -- Overlapped And Nonoverlapped Tallying Of M-event Chains -- An Illustration Of Log-linear Basics -- Log-linear Analysis Of Interval And Multicode Event Data -- Summary -- 12. Recurrence Analysis And Permutation Tests -- Recurrence Analysis -- Permutation Tests For Short Event Sequences -- Summary. Roger Bakeman, Vicenç Quera. Includes Bibliographical References (p. 169-177) And Index. Behavioral scientists use many methods to measure behavior. Systematic observation is used to study relatively natural, spontaneous behavior as it unfolds sequentially in time. This book emphasizes digital means to record and code such behavior. Key topics include devising coding schemes, training observers, and assessing reliability, as well as recording, representing, and analyzing observational data. In clear and straightforward language, this book provides a thorough grounding in observational methods along with considerable practical advice. It describes standard conventions for sequential data and details how to perform sequential analysis with a computer program developed by the authors. The book is rich with examples of coding schemes and different approaches to sequential analysis, including both statistical and graphical means. Book jacket Provides a thorough grounding in observational methods along with considerable practical advice. Behavioral scientists use observational methods to study relatively natural, spontaneous behavior as it unfolds in time. This book is rich with examples of coding methods and different approaches to sequential analysis, including both statistical and graphical means.