Measuring Productivity in Education and Not-for-Profits: With Tools and Examples in R (Management for Professionals)

This book takes the reader through real-world examples for how to characterize and measure the productivity and performance of NFPs and education institutions―that is, organisations that produce value for society, which cannot be measured accurately in financial KPIs. It focuses on how best to frame non-profit performance and productivity, and provides a suite of tools for measurement and benchmarking. It further challenges the reader to consider alternative and appropriate uses of quantitative measures, which are fit-for-purpose in individual contexts. It is true that the risk of misusing quantitative measures is ever-present. But does that risk outweigh the benefits of forming a more precise and shared understanding of what could generate better outcomes? There will always be concerns about policy and performance management. Goodheart’s Law states that once a measure becomes a target, it is no longer a good measure. This book helps to strike a meaningful balance between what can be measured, what cannot, and how best to use quantitative information in sectors that are often averse to being held up to the light and put on a scale by outsiders. Preface 6 Contents 8 1: Introduction 12 1.1 The Value of This Book 12 1.2 Who Is This Book For? 13 1.3 Background and Motivation 13 1.4 Assumptions 13 1.5 Why R? 14 1.6 Terminology and Contextual Considerations 15 1.7 Structure of the Book 16 1.7.1 Chapter Structure 16 1.7.2 Chapter Progression 16 1.8 Getting the Most Out of this Book 17 1.9 Tutorial: A Brief Intro to Some Tidyverse Functions 18 1.9.1 Setup 18 1.9.2 Intro to the “Pipe” 18 1.9.3 Using Filter, Select, Mutate, Group, and Summarise 19 1.9.4 Long and Wide Format 20 1.9.5 Using ggplot2 for Visualization 21 1.9.6 Using Purrr and Map for Iteration 23 1.10 Reflections on the Tidyverse in Practice 25 2: Inputs and Outputs 26 2.1 Objective 26 2.2 What Are Inputs and Outputs? 26 2.3 A Detailed Look at Inputs and Outputs 27 2.3.1 The Input-Output Paradigm 27 2.3.2 Identifying Inputs and Outputs 28 2.4 Tutorial: Organizing Input-Output Data 29 2.4.1 Setup 29 2.4.2 Introduction 29 2.4.3 Clean the Data 31 2.4.4 Input-Output Structure 33 2.4.5 Add in More Data 35 2.4.6 Fashion a Data Set for Specific Analysis 37 2.5 Reflections on Inputs and Outputs in Practice 38 3: The Productivity Ratio 39 3.1 Objective 39 3.2 What Is the Productivity Ratio? 39 3.3 A Detailed Look at the Productivity Ratio 40 3.3.1 Total-Factor, Multi-Factor, and Single-Factor Productivity 40 3.3.2 Selecting Appropriate Indicators for Education 40 3.3.3 Value-Add 41 3.4 Tutorial: Competing Production Functions 42 3.4.1 Setup 42 3.4.2 Introduction 42 3.4.3 Create and Join the New Test Scores Data 43 3.4.4 Operational Efficiency Productivity 44 3.4.5 Reputational Productivity 46 3.4.6 Value-Add Productivity 49 3.4.7 Compile All the Results 51 3.5 Reflections on the Productivity Ratio in Practice 53 4: Productivity Change 54 4.1 Objective 54 4.2 What Is Productivity Change? 54 4.3 A Detailed Look at Measuring Productivity Change 55 4.3.1 Estimation Technique 55 4.3.2 Defining TI 56 4.3.3 NRC Model (Adjusted Load) 57 4.3.4 Adjusting for the Real Value of Money 58 4.4 Tutorial: Measuring University Productivity Change 59 4.4.1 Setup 59 4.4.2 Introduction 59 4.4.3 Get the Raw Data 59 4.4.4 Adjust for RVM and Adjusted Load 61 4.4.5 Calculate Change Indexes 63 4.4.6 Calculate Cumulative Change 66 4.4.7 Further Functionalize Our Code 68 4.4.8 Calculate Data for TI Weights 69 4.4.9 Calculate TIs 72 4.5 Reflections on Productivity Change in Practice 74 5: Productivity Change with Threshold Analysis 76 5.1 Objective 76 5.2 What Is Threshold Analysis? 76 5.3 A Detailed Look at Threshold Analysis for Universities 77 5.3.1 Multiple Outputs 77 5.3.2 Characterizing Production Technology 78 5.3.3 Adjusted Publications 78 5.4 Tutorial: Teaching-Research Nexus or Divide? 79 5.4.1 Setup 79 5.4.2 Introduction 79 5.4.3 Create New Data Set 80 5.4.4 Examine Institutional Change Profiles 83 5.4.5 Calculate Productivity 85 5.4.6 Threshold Analysis 88 5.4.7 Rankings Analysis 91 5.5 Reflections on Threshold Analysis in Practice 93 6: Frontier Analysis with DEA 94 6.1 Objective 94 6.2 What Is Frontier Analysis? 94 6.3 A Detailed Look at Frontier Analysis with DEA 96 6.3.1 A Not-for-Profit Example 96 6.3.2 DEA Objective 97 6.3.3 DEA Algorithm Walk-Through 97 6.4 Tutorial: Which NFP Is the Top Performer? 99 6.4.1 Setup 99 6.4.2 Introduction 99 6.4.3 Create New Data Set 100 6.4.4 Define a Linear Program for Org A 101 6.4.5 Create a Linear Program to Iterate for DEA 104 6.5 Reflections on DEA in Practice 107 7: Values Analysis with DEA 108 7.1 Objective 108 7.2 What Is Values Analysis with DEA? 108 7.3 A Detailed Look at Values Analysis with DEA 109 7.3.1 The Procedure 109 7.4 Tutorial: DEA Values Analysis for Decision-Making 110 7.4.1 Setup 110 7.4.2 Introduction 110 7.4.3 Prepare the Data for Values Analysis 111 7.4.4 Production Function Analysis 113 7.4.5 Values Analysis for Examining the Production Frontier 116 7.4.6 Visualize the Frontiers 120 7.4.7 What Happens When No DEA Scheme Represents Our Values? 123 7.5 Reflections on DEA Values Analysis in Practice 127 8: Threshold Analysis for Absolute Productivity 128 8.1 Objective 128 8.2 What Is Absolute Productivity Threshold Analysis? 128 8.3 A Detailed Look at Absolute Productivity Threshold Analysis 129 8.3.1 Overview of the Procedure 129 8.3.2 Estimating Absolute Productivity 130 8.4 Tutorial: Teaching Research Nexus? 131 8.4.1 Setup 131 8.4.2 Introduction 131 8.4.3 Feature Scaling All Data Elements 132 8.4.4 Find Absolute Productivity for the Base Year 133 8.4.5 Threshold Analysis 135 8.4.6 Plot the Data 136 8.4.7 Rankings Analysis 138 8.5 Reflections on Absolute Productivity Threshold Analysis in Practice 139 9: Hypothesis Testing 141 9.1 Objective 141 9.2 What Is Hypothesis Testing with Productivity Data? 141 9.3 A Detailed Look at Hypothesis Testing for Productivity Change 142 9.3.1 Initial Caveat 142 9.3.2 Model Fitting with GLS 142 9.4 Tutorial: Hypothesis Testing Using Generalized Least Squares 142 9.4.1 Setup 142 9.4.2 Introduction 143 9.4.3 Test Two Sets of Fitted Data 143 9.4.4 Add Longitudinal Data 146 9.4.5 Did Our Intervention Make a Difference? 148 9.5 Reflections on Hypothesis Testing in Practice 150 Index 152 This book takes the reader through real-world examples for how to characterize and measure the productivity and performance of NFPs and education institutions that is, organisations that produce value for society, which cannot be measured accurately in financial KPIs. It focuses on how best to frame non-profit performance and productivity, and provides a suite of tools for measurement and benchmarking. It further challenges the reader to consider alternative and appropriate uses of quantitative measures, which are fit-for-purpose in individual contexts. It is true that the risk of misusing quantitative measures is ever-present. But does that risk outweigh the benefits of forming a more precise and shared understanding of what could generate better outcomes? There will always be concerns about policy and performance management. Goodheart's Law states that once a measure becomes a target, it is no longer a good measure. This book helps to strike a meaningful balance between what can be measured, what cannot, and how best to use quantitative information in sectors that are often averse to being held up to the light and put on a scale by outsiders