Frontiers of Test Validity Theory: Measurement, Causation, and Meaning (Multivariate Applications Series)

This book examines test validity in the behavioral, social, and educational sciences by exploring three fundamental problems: measurement, causation and meaning. Psychometric and philosophical perspectives receive attention along with unresolved issues. The authors explore how measurement is conceived from both the classical and modern perspectives. The importance of understanding the underlying concepts as well as the practical challenges of test construction and use receive emphasis throughout. The book summarizes the current state of the test validity theory field. Necessary background on test theory and statistics is presented as a conceptual overview where needed. Each chapter begins with an overview of key material reviewed in previous chapters, concludes with a list of suggested readings, and features boxes with examples that connect theory to practice. These examples reflect actual situations that occurred in psychology, education, and other disciplines in the US and around the globe, bringing theory to life. Critical thinking questions related to the boxed material engage and challenge readers. A few examples include: What is the difference between intelligence and IQ? Can people disagree on issues of value but agree on issues of test validity? Is it possible to ask the same question in two different languages? The first part of the book contrasts theories of measurement as applied to the validity of behavioral science measures.The next part considers causal theories of measurement in relation to alternatives such as behavior domain sampling, and then unpacks the causal approach in terms of alternative theories of causation.The final section explores the meaning and interpretation of test scores as it applies to test validity. Each set of chapters opens with a review of the key theories and literature and concludes with a review of related open questions in test validity theory. Researchers, practitioners and policy makers interested in test validity or developing tests appreciate the book's cutting edge review of test validity. The book also serves as a supplement in graduate or advanced undergraduate courses on test validity, psychometrics, testing or measurement taught in psychology, education, sociology, social work, political science, business, criminal justice and other fields. The book does not assume a background in measurement. Taylor & Francis Ltd Cover 1 Half Title 2 Title Page 6 Copyright Page 7 Table of Contents 8 About the Authors 13 Preface 14 Acknowledgements 18 1. Introduction: Surveying the Field of Test Validity Theory 20 1.1. Terminology 20 1.2. The Development of Test Validity Theory 24 1.3. Measurement, Causation, Meaning, and Test Validity 34 1.4. Further Reading 37 Part I: 38 2. Philosophical Theories of Measurement 38 2.1. The Classical Theory of Measurement 40 2.2. Addition and Concatenation 44 2.3. The Representational Theory of Measurement: Stevens’ Version 46 2.4. The Representational Theory of Measurement: The Axiomatic Approach 51 2.5. The Latent Variable Perspective 55 2.6. Chapter Conclusion 60 2.7. Further Reading 62 3. Psychometric Models 63 3.1. Psychometric Models 64 3.2. Test Scores, Constructs, and Validity 71 3.3. The Relation Between Validity and Other Psychometric Concepts 75 3.4. Chapter Conclusion 83 3.5. Further Reading 84 4. Open Issues in Measurement Theory and Psychometrics 85 4.1. Measurement and the Structure of Psychological Attributes 86 4.2. Measurement and Probabilistic Models 90 4.3. A Possible Reconciliation Between Theories of Measurement 95 4.4. Is Measurement Necessary in Psychological Testing? 96 4.5. Chapter Conclusion 98 4.6. Further Reading 99 Part II: 100 5. Test Scores as Samples: Behavior Domain Theory 100 5.1. Conceptualizing Domain Scores 105 5.2. The Item Writing Argument for Behavior Domain Theory 109 5.3. The Distinctness Argument Against Domain Score Causation 111 5.4. Reconciling Behavior Domains and Causal Theories of Measurement 115 5.5. Chapter Conclusion 122 5.6. Further Reading 123 6. Causality in Measurement 124 6.1. Causal Structures 127 6.2. Implications for Validity Theory 136 6.3. Direct, Indirect, and Mixed Models 140 6.4. Other Structures 144 6.5. Chapter Conclusion 155 6.6. Further Reading 155 7. Causation, Correlation, and Reflective Measurement Models 156 7.1. Reflective Measurement With and Without Causation 159 7.2. Statistical Unidimensionality versus Causal Unidimensionality 167 7.3. Three General Theories of Causation 172 7.4. Reflective Measurement Models with Different Causal Interpretations 178 7.5. Chapter Conclusion 190 7.6. Further Reading 190 8. Problems in Causation and Validity Formative Measurement, Networks, and Individual Differences 192 8.1. Formative Measurement Models and Causation 193 8.2. Network Measurement Models 202 8.3. Causation and Individual Differences 203 8.4. Validating Causal Measurement Models 209 8.5. Chapter Conclusion 212 8.6. Further Reading 214 Part III: 85 9. Interpreting Test Responses Validity, Values, and Evaluation 215 9.1. Messick’s Unified View of Validity 216 9.2. Historical Context and Recent Literature 218 9.3. Shadish, Cook, and Campbell’s Critique 219 9.4. Scriven’s Critique 224 9.5. Searl’s Naturalistic-Fallacy Fallacy 226 9.6. Putnam’s Critique of the Fact/Value Dichotomy 229 9.7. Conceptualizing Test Validity and its Consequences 235 9.8. Chapter Conclusion 238 9.9. Further Reading 239 10. A Model of Test Score Interpretation 240 10.1. Focal Example: Tuna Cleaner Work Sample 243 10.2. Interpretation and Entailment: Construct Theories as Inference Engines 244 10.3. Interpretations of Test Scores 248 10.4. Chapter Conclusion 264 10.5. Further Reading 265 11. Open Questions About Test Score Meaning 266 11.1. The Psychologist’s Fallacy and Test Score Interpretation 267 11.2. Meaning’s in the Heads of Test Takers? 272 11.3. Heteroglossia: Speaking in Diverse Tongues 277 11.4. Test Takers and Their Beliefs 283 11.5. Validity Without Determinate Meaning 288 11.6. Chapter Conclusion 291 11.7. Further Reading 291 Part IV: 85 12. An Integrative View of Test Validity 292 12.1. An Integrative Model of Test Validation 293 12.2. Truth, Justification, Knowledge, and Action 299 12.3. Kinds of Causality and Types of Evidence 302 12.4. Justification Through Alignment 308 12.5. Knowledge, Value, and Action 310 12.6. Chapter Conclusion 311 13. Epilogue as Dialog The Future of Test ValidityTheory 313 13.1. Test Validation Methods 313 13.2. Test Validity Theory and Meaning 316 13.3. The Yardstick by Which Validation is Judged (If There is One) 319 13.4. Validity and Response Processes 323 13.5. Attributes 327 13.6. Tests and Interpretations 331 13.7. Chapter Conclusion 335 Notes 336 References 338 Author Index 355 Subject Index 357 Example Index 359 ISBN:,9781841692197,(hbk),ISBN:,978–1–84169–220–3,(pbk),ISBN:,978–0–203–50120–7,(ebk) ISBN: 9781841692197 (hbk),ISBN: 978–1–84169–220–3 (pbk),ISBN: 978–0–203–50120–7 (ebk)