Margins of Error: A Study of Reliability in Survey Measurement (Wiley Series in Survey Methodology)

Enhance the quality of survey results by recognizing and reducing measurement errors. Margins of Error: A Study of Reliability in Survey Measurement demonstrates how and hwy identifying the presence and extent of measurement errors in survey data is essential for improving the overall collection and analysis of the data. The author outlines the consequences of ignoring survey measurement errors and also discusses ways to detect and estimate the impact of these errors. This book also provides recommendations of improving the quality of survey data. Logically organized and clearly written, this book: Deconstructs the data gathering process into six main elements of the response process: question adequacy, comprehension, accessibility, retrieval, motivation, and communication Provides an exhaustive review of valuable reliability estimation techniques that can be applied to survey data Identifies the types of questions and interviewer practices that are essential to the collection of reliable data Addresses hypotheses regarding which survey questions, sources of information, and questionnaire formats produce the most reliable data In conjunction with research data gathered on nearly 500 survey measures and the application of an empirical approach grounded in classical measurement theory, this book discusses the sources of measurement error and provides the tools necessary for improving survey data collection methods. Margins of Error enables statisticians and researchers in the fields of public opinion and survey research to design studies that can detect, estimate, and reduce measurement errors that may have previously gone undetected. This book also serves as a supplemental textbook for both undergraduate and graduate survey methodology courses. Cover Page......Page 1 Title: Margins of Error - A Study of Reliability in Survey Measurement......Page 4 ISBN 0470081481......Page 5 Contents (with page links)......Page 7 Preface......Page 11 Acknowledgments......Page 13 Foreword......Page 15 1 Measurement Errors in Surveys......Page 17 1.1 WHY STUDY SURVEY MEASUREMENT ERROR?......Page 18 1.2 SURVEY ERRORS......Page 19 1.2.1 Classifying Types of Survey Error......Page 21 1.3 SURVEY MEASUREMENT ERRORS......Page 22 1.4 STANDARDS OF MEASUREMENT......Page 24 1.5 RELIABILITY OF MEASUREMENT......Page 25 1.6 THE NEED FOR FURTHER RESEARCH......Page 26 1.7 THE PLAN OF THIS BOOK......Page 27 2 Sources of Survey Measurement Error......Page 31 2.1 THE UBIQUITY OF MEASUREMENT ERRORS......Page 32 2.2 SOURCES OF MEASUREMENT ERROR IN SURVEY REPORTS......Page 36 2.2.1 Validity of the Question......Page 38 2.2.2 Comprehensionof the Question......Page 39 2.2.3 Information Accessibility......Page 41 2.2.4 Retrieval of Information......Page 42 2.2.5 Respondent Motivation......Page 45 2.2.6 Communication......Page 46 2.3 CONSEQUENCES OF MEASUREMENT ERROR......Page 48 3 Reliability Theory for Survey Measures......Page 51 3.2 BASIC CONCEPTS OF CLASSICAL RELIABILITY THEORY......Page 52 3.3 NONRANDOM MEASUREMENT ERROR......Page 57 3.4 THE COMMON-FACTORMODEL REPRESENTATION OF CTST......Page 58 3.5 SCALING OF VARIABLES......Page 59 3.6 DESIGNS FOR RELIABILITY ESTIMATION......Page 61 3.7 VALIDITY AND MEASUREMENT......Page 62 3.7.1 Record-Check Studies......Page 64 3.7.2 Split-Ballot Experiments......Page 66 3.8 RELIABILITY MODELS FOR COMPOSITE SCORES......Page 67 3.9 DEALING WITH NONRANDOM OR SYSTEMATIC ERROR......Page 69 3.10 SAMPLING CONSIDERATIONS......Page 71 3.11 CONCLUSIONS......Page 73 4 Reliability Methods for Multiple Measures......Page 75 4.1 MULTIPLE MEASURES VERSUS MULTIPLE INDICATORS......Page 77 4.1.1 Multiple Measures......Page 78 4.1.2 Multiple-Indicators Models......Page 81 4.2 MULTITRAIT-MULTIMETHOD APPROACHES......Page 83 4.2.1 The MTMM Matrix......Page 84 4.3 COMMON-FACTORMODELS OF THE MTMM DESIGN......Page 87 4.3.1 Decomposition of Variance/Covariance......Page 90 4.4 CLASSICAL TRUE-SCORE REPRESENTATION OF THE MTMM MODEL......Page 93 4.5 THE GROWING BODY OF MTMM STUDIES......Page 95 4.5.1 Clarification of Terminology......Page 96 4.6 AN EXAMPLE......Page 99 4.6.1 Interpretation......Page 106 4.7 CRITIQUE OF THE MTMM APPROACH......Page 107 4.8 WHERE ARE WE?......Page 109 5 Longitudinal Methods for Reliability Estimation......Page 111 5.1 THE TEST-RETEST METHOD......Page 112 5.2 SOLUTIONS TO THE PROBLEM......Page 117 5.2.1 The Quasi-Markov Simplex Model......Page 118 5.3 ESTIMATING RELIABILITY USING THE QUASI-MARKOVSIMPLEX MODEL......Page 120 5.4 CONTRIBUTIONS OF THE LONGITUDINAL APPROACH......Page 126 5.5 COMPONENTS OF THE SURVEY RESPONSE......Page 130 5.6 WHERE TO FROM HERE?......Page 132 6 Using Longitudinal Data to Estimate Reliability Parameters......Page 133 6.1 RATIONALE FOR THE PRESENT STUDY......Page 134 6.2.1 National Election Panel Studies......Page 135 6.2.3 The Study of American Families......Page 137 6.3 DOMAINS OF MEASUREMENT......Page 138 6.4 STATISTICAL ESTIMATION STRATEGIES......Page 143 6.5 COMPARISON OF METHODS OF RELIABILITY ESTIMATION......Page 146 6.6 THE PROBLEM OF ATTRITION......Page 151 6.6.1 Estimation of Reliability in the Presence of Incomplete Data......Page 153 6.7 WHICH RELIABILITY ESTIMATES?......Page 162 6.8 CONCLUSIONS......Page 163 7 The Source and Content of Survey Questions......Page 165 7.1 SOURCE OF INFORMATION......Page 166 7.2 PROXY REPORTS......Page 168 7.3 CONTENT OF QUESTIONS......Page 169 7.3.1 Reliability in Measuring Self-Reported Facts......Page 172 7.3.2 The Reliability of Measuring Self-Reported Nonfacts......Page 174 7.4 SUMMARY AND CONCLUSIONS......Page 178 8 Survey Question Context......Page 181 8.1 THE ARCHITECTURE OF SURVEY QUESTIONNAIRES......Page 183 8.2 QUESTIONS IN SERIES VERSUS QUESTIONS IN BATTERIES......Page 187 8.3 LOCATION IN THE QUESTIONNAIRE......Page 188 8.4 UNIT LENGTH AND POSITION IN SERIES AND BATTERIES......Page 191 8.5 LENGTH OF INTRODUCTIONS TO SERIES AND BATTERIES......Page 193 8.6 CONCLUSIONS......Page 195 9 Formal Properties of Survey Questions......Page 197 9.1 QUESTION FORM......Page 199 9.2 TYPES OF CLOSED-FORM QUESTIONS......Page 201 9.2.1 Agree-Disagree Questions......Page 202 9.2.3 Feeling Thermometers......Page 204 9.3 NUMBER OF RESPONSE CATEGORIES......Page 207 9.4 UNIPOLAR VERSUS BIPOLAR SCALES......Page 211 9.5 DON’T KNOW OPTIONS......Page 212 9.6 VERBAL LABELING OF RESPONSE CATEGORIES......Page 216 9.7 SURVEY QUESTION LENGTH......Page 218 9.8 CONCLUSIONS......Page 226 10 Attributes of Respondents......Page 229 10.1 RELIABILITY AS A POPULATION PARAMETER......Page 230 10.2.1 Strategies of Analysis......Page 231 10.3 AGE AND RELIABILITY OF MEASUREMENT......Page 234 10.4 SCHOOLING AND RELIABILITY OF MEASUREMENT......Page 237 10.5.1 Age Differences in Schooling......Page 239 10.5.2 Schooling-AdjustedAge Differences in Reliability......Page 241 10.6 GENERATIONAL DIFFERENCES IN RELIABILITY......Page 243 10.7 MTMM RESULTS BY AGE AND EDUCATION......Page 244 10.8 STATISTICAL ESTIMATION OF COMPONENTS VARIATION......Page 247 10.8.2 Educationand Reliability of Measurementin the HRS......Page 253 10.8.3 Age and Components of Variance......Page 258 10.8.4 Education and Components of Variance......Page 263 10.9 TESTS OF HYPOTHESES ABOUT GROUP DIFFERENCES......Page 270 10.10 CONCLUSIONS......Page 277 11 Reliability Estimation for Categorical Latent Variables......Page 279 11.1 BACKGROUND AND RATIONALE......Page 280 11.2 THE LATENT CLASS MODEL FOR MULTIPLE INDICATORS......Page 281 11.2.1 The Model......Page 282 11.2.2 An Example......Page 284 11.3 THE LATENT CLASS MODEL FOR MULTIPLE MEASURES......Page 288 11.3.1 An Example......Page 289 11.4 THE LATENT MARKOV MODEL......Page 293 11.4.1 The Model......Page 295 11.4.2 An Example......Page 296 11.5 CONCLUSIONS......Page 302 12 Final Thoughts and Future Directions......Page 305 12.1 RELIABILITY AS AN OBJECT OF STUDY......Page 306 12.2 WHY STUDY THE RELIABILITY OF SURVEY MEASUREMENT?......Page 307 12.2.1 Effects of Unreliability on Statistical Inference......Page 308 12.2.3 Effects of Unreliability on Estimates of Statistical Association......Page 314 12.3 THE LONGITUDINAL APPROACH......Page 315 12.4 ASSEMBLING KNOWLEDGE OF SURVEY MEASUREMENT RELIABILITY......Page 317 12.4.1 Some Comparisonsof Reliability Estimates......Page 318 12.4.2 Considerations in Archiving Reliability Estimates......Page 320 12.5 COMPENSATING FOR MEASUREMENT ERROR USING COMPOSITE VARIABLES......Page 324 12.6 CONCLUSIONS......Page 331 12.6.1 Future Agenda......Page 332 12.6.2 Closing......Page 334 APPENDIX Reliability of Survey Measures Used in the Present Study......Page 343 References......Page 383 Index (with page links)......Page 399 WILEY SERIES IN SURVEY METHODOLOGY......Page 407 Back Page......Page 409

Enhance the quality of survey results by recognizing and reducing measurement errors.

Margins of Error: A Study of Reliability in Survey Measurement demonstrates how and hwy identifying the presence and extent of measurement errors in survey data is essential for improving the overall collection and analysis of the data. The author outlines the consequences of ignoring survey measurement errors and also discusses ways to detect and estimate the impact of these errors. This book also provides recommendations of improving the quality of survey data.

Logically organized and clearly written, this book:

• Deconstructs the data gathering process into six main elements of the response process: question adequacy, comprehension, accessibility, retrieval, motivation, and communication
• Provides an exhaustive review of valuable reliability estimation techniques that can be applied to survey data
• Identifies the types of questions and interviewer practices that are essential to the collection of reliable data
• Addresses hypotheses regarding which survey questions, sources of information, and questionnaire formats produce the most reliable data

In conjunction with research data gathered on nearly 500 survey measures and the application of an empirical approach grounded in classical measurement theory, this book discusses the sources of measurement error and provides the tools necessary for improving survey data collection methods.

Margins of Error enables statisticians and researchers in the fields of public opinion and survey research to design studies that can detect, estimate, and reduce measurement errors that may have previously gone undetected. This book also serves as a supplemental textbook for both undergraduate and graduate survey methodology courses.

This book covers the following: deconstructs the data gathering process into six main elements of the response process: question adequacy, comprehension, accessibility, retrieval, motivation, and communication ; provides an exhaustive review of valuable reliability estimation techniques that can be applied to survey data ; identifies the types of questions and interviewer practices that are essential to the collection of reliable data ; and addresses hypotheses regarding which survey questions, sources of information, and questionnaire formats produce the most reliable data. It also discusses the sources of measurement error and provides the tools necessary for improving survey data collection methods This book argues that the consideration of the presence and extent of measurement errors in survey data leads to improvement in the overall collection and analysis of survey data. Its main purpose is to identify which types of questions and which types of interviewer practices produce the most valid and reliable data