Public Program Evaluation : A Statistical Guide

Contents......Page 5 Preface......Page 9 What Is Program Evaluation?......Page 13 Types of Program Evaluations......Page 18 Basic Characteristics of Program Evaluation......Page 23 Assessing Government Performance: Program Evaluation and GPRA......Page 25 A Brief History of Program Evaluation......Page 26 What Comes Next......Page 28 Key Concepts......Page 29 Do It Yourself......Page 30 Program Evaluation and Performance Measurement: What Is the Difference?......Page 31 Benchmarking......Page 34 Reporting Performance Results......Page 35 Conclusion......Page 37 Do It Yourself......Page 39 Defining Defensibility......Page 43 Types of Validity: Definitions......Page 44 Types of Validity: Threats and Simple Remedies......Page 45 Basic Concepts......Page 64 Do It Yourself......Page 65 The Logic of Internal Validity......Page 66 Making Comparisons: Cross Sections and Time Series......Page 69 Threats to Internal Validity......Page 70 Summary......Page 78 Three Basic Research Designs......Page 79 Rethinking Validity: The Causal Model Workhorse......Page 81 Basic Concepts......Page 83 A Summary of Threats to Internal Validity......Page 84 Basic Characteristics......Page 86 Brief History......Page 87 Caveats and Cautions About Randomized Experiments......Page 88 Types of RFEs......Page 92 Issues in Implementing RFEs......Page 104 Threats to the Validity of RFEs: Internal Validity......Page 108 Threats to the Validity of RFEs: External Validity......Page 111 Conclusion......Page 113 Basic Concepts......Page 114 Do It Yourself: Design a Randomized Field Experiment......Page 115 Defining Quasi-Experimental Designs......Page 116 The One-Shot Case Study......Page 117 The Posttest-Only Comparison-Group (PTCG) Design......Page 119 The Pretest Posttest Comparison-Group (PTPTCG) (The Nonequivalent Control-Group) Design......Page 125 The Pretest Posttest (Single-Group) Design......Page 128 The Single Interrupted Time-Series Design......Page 130 The Interrupted Time-Series Comparison-Group (ITSCG) Design......Page 137 The Multiple Comparison-Group Time-Series Design......Page 140 Summary of Quasi-Experimental Design......Page 141 Basic Concepts......Page 142 Do It Yourself......Page 143 What Is a Nonexperimental Design?......Page 144 Back to the Basics: The Workhorse Diagram......Page 145 The Nonexperimental Workhorse Regression Equation......Page 146 Data for the Workhorse Regression Equation......Page 148 Interpreting Multiple Regression Output......Page 151 Assumptions Needed to Believe That b Is a Valid Estimate of B [E(b) = B]......Page 165 Assumptions Needed to Believe the Significance Test for b......Page 183 What Happened to the R 2 ?......Page 189 Basic Concepts......Page 190 Introduction to Stata......Page 193 Do It Yourself: Interpreting Nonexperimental Results......Page 197 8 Designing Useful Surveys for Evaluation......Page 202 The Response Rate......Page 203 How to Write Questions to Get Unbiased, Accurate, Informative Responses......Page 210 Turning Responses into Useful Information......Page 217 For Further Reading......Page 225 Basic Concepts......Page 226 Do It Yourself......Page 227 9 Summing It Up Meta-Analysis......Page 230 Example of a Meta-Analyis: Data......Page 231 Example of a Meta-Analysis: Variables......Page 232 Example of a Meta-Analysis: Data Analysis......Page 233 The Role of Meta-Analysis in Program Evaluation and Causal Conclusions......Page 234 For Further Reading......Page 235 Notes......Page 237 Index......Page 259 About the Authors......Page 271 "Readable and comprehensive, this text is designed to equip students and practitioners with the statistical skills needed to meet government standards regarding public program evaluation. Even those with little or no statistical training will find the explanations clear, with many illustrative examples, case studies, and applications. Far more than a cookbook of statistical techniques, the book begins with chapters on the overall context for successful program evaluations, and carefully explains statistical methods--and threats to internal and statistical validity--that correspond to each evaluation design. Laura Langbein then presents a variety of methods for program analysis, and advises readers on how to select the mix of methods most appropriate for the issues they deal with-always balancing methodology with the need for generality, the size of the evaluator's budget, the availability of data, and the need for quick results. Among this text's many important features: maintains a practical focus on doing evaluation; integrates research design with corresponding statistical/econometric estimation methods; uses examples from many policy fields, not just social services; uses examples from domestic programs as well as developing countries; links program evaluation to the larger field of policy analysis."--Publisher's website This readable and comprehensive text is designed to equip students and practitioners with the statistical skills needed to meet government standards regarding public program evaluation. It maintains a practical focus on doing evaluation, with clear explanations and illustrative examples, case studies, and applications from many policy fields. Designed to equip students and practitioners with the statistical skills needed to meet government standards regarding public program evaluation. Beginning with chapters on the overall context for successful program evaluations, this book explains the various forms of experimental validity that relate to evaluation designs