Beyond significance testing : statistics reform in the behavioral sciences

This accessibly written book reviews the controversy about significance testing, which has now crossed various disciplines as diverse as psychology, ecology, commerce, education, and biology, among others. It also introduces readers to alternative methods, especially effect size estimation (at both the group and case levels) and interval estimation (confidence intervals) in comparative studies. Basics of bootstrapping and Bayesian estimation are also considered. Research examples from substance abuse, education, learning, and other areas illustrate how to apply these methods. A companion website ((http://www.apa.org/books/resources/kline) www.apa.org/books/resources/kline ) promotes learning by providing chapter exercises and sample answers, downloadable raw data files for many research examples, and links to other useful websites. "Part I is concerned with fundamental concepts and summarizes the significance testing controversy. Outlined in Chapter 1 is the rationale of statistics reform. The history of the controversy about significance testing in psychology and other disciplines is recounted in this chapter. Principles of sampling and estimation that underlie confidence intervals and statistical tests are reviewed in Chapter 2. The logic and illogic of significance testing is considered in Chapter 3, and misunderstandings about p> values are elaborated in Chapter 4. The purpose of Chapters 3-4 is to help you to understand critical weaknesses of statistical tests. Part II comprises four chapters about effect size estimation in comparative studies, where at least two different groups or conditions are contrasted. In Chapter 5, the rationale of effect size estimation is outlined and basic effect sizes for continuous outcomes are introduced. The problem of evaluating substantive significance is also considered. Effect sizes for categorical outcomes, such as relapsed versus not relapsed, are covered in Chapter 6. Chapters 7 and 8 concern effect size estimation in, respectively, single-factor designs with at least three conditions and factorial designs with two or more factors and continuous outcomes. Many empirical examples are offered in Part II. There are exercises for Chapters 2-8 and suggested answers are available on the book's website. Part III includes two chapters that cover alternatives to significance testing. Chapter 9 deals with replication and meta-analysis. The main points of this chapter are that a larger role for replication will require a cultural change in the behavioral sciences and that meta-analysis is an important tool for research synthesis but is no substitute for explicit replication. Bayesian estimation is the subject of Chapter 10. Bayesian statistics are overlooked in psychology research, but this approach offers an inference framework consistent with many goals of statistics reform. Best practice recommendations are also summarized in this chapter"--Introduction. (PsycINFO Database Record (c) 2013 APA, all rights reserved) "Part I is concerned with fundamental concepts and summarizes the significance testing controversy. Outlined in Chapter 1 is the rationale of statistics reform. The history of the controversy about significance testing in psychology and other disciplines is recounted in this chapter. Principles of sampling and estimation that underlie confidence intervals and statistical tests are reviewed in Chapter 2. The logic and illogic of significance testing is considered in Chapter 3, and misunderstandings about p values are elaborated in Chapter 4. The purpose of Chapters 3-4 is to help you to understand critical weaknesses of statistical tests. Part II comprises four chapters about effect size estimation in comparative studies, where at least two different groups or conditions are contrasted. In Chapter 5, the rationale of effect size estimation is outlined and basic effect sizes for continuous outcomes are introduced. The problem of evaluating substantive significance is also considered. Effect sizes for categorical outcomes, such as relapsed versus not relapsed, are covered in Chapter 6. Chapters 7 and 8 concern effect size estimation in, respectively, single-factor designs with at least three conditions and factorial designs with two or more factors and continuous outcomes. Many empirical examples are offered in Part II. There are exercises for Chapters 2-8 and suggested answers are available on the book's website. Part III includes two chapters that cover alternatives to significance testing. Chapter 9 deals with replication and meta-analysis. The main points of this chapter are that a larger role for replication will require a cultural change in the behavioral sciences and that meta-analysis is an important tool for research synthesis but is no substitute for explicit replication. Bayesian estimation is the subject of Chapter 10. Bayesian statistics are overlooked in psychology research, but this approach offers an inference framework consistent with many goals of statistics reform. Best practice recommendations are also summarized in this chapter"--Introduction. (PsycINFO Database Record (c) 2014 APA, all rights reserved). Contents......Page 10 Acknowledgments......Page 12 Introduction......Page 15 Chapter 1. Changing Times......Page 21 Chapter 2. Sampling and Estimation......Page 41 Chapter 3. Logic and Illogic of Significance Testing......Page 79 Chapter 4. Cognitive Distortions in Significance Testing......Page 107 Chapter 5. Continuous Outcomes......Page 135 Chapter 6. Categorical Outcomes......Page 175 Chapter 7. Single-Factor Designs......Page 201 Chapter 8. Multifactor Designs......Page 233 Chapter 9. Replication and Meta-Analysis......Page 277 Chapter 10. Bayesian Estimation and Best Practices Summary......Page 301 References......Page 325 Index......Page 347 About the Author......Page 361 I. Fundamental Concepts. Changing Times -- Sampling And Estimation -- Logic And Illogic Of Significance Testing -- Cognitive Distortions In Significance Testing -- Ii. Effect Size Estimation In Comparative Studies. Continuous Outcomes -- Categorical Outcomes -- Single-factor Designs -- Multifactor Designs -- Iii. Alternatives To Significance Testing. Replication And Meta-analysis -- Bayesian Estimation And Best Practices Summary. Rex B. Kline. Second Edition--introduction. Rev Ed. Of: Beyond Significance Testing : Reforming Data Analysis Methods In Behavioral Research. C2004. Includes Bibliographical References And Index.