Agile Metrics in Action: Measuring and Enhancing the Performance of Agile Teams

Project tracking systems, test and build tools, source control, continuous integration, and other built-in parts of the software development lifecycle generate a wealth of data that can be used to track and improve the quality and performance of products, processes, and teams. Although the iterative nature of Agile development is perfect for data-driven continuous improvement, the collection, analysis, and application of meaningful metrics often fades in favor of subjective measures that offer less insight into the real challenges of making better software. Agile Metrics in Action: Measuring and enhancing the performance of Agile teams is a practical book that shows how to take the data already being generated to make teams, processes, and products better. It points out which metrics to use to objectively measure performance and what data really counts, along with where to find it, how to get it, and how to analyze it. The book also shows how all team members can publish their own metrics through dashboards and radiators, taking charge of communicating performance and individual accountability. Along the way, it offers practical data analysis techniques, including a few emerging Big Data practices. Agile Metrics in Action 1 brief contents 6 contents 8 foreword 14 preface 16 acknowledgments 18 about this book 20 Roadmap 20 Code conventions and downloads 21 Author Online 21 About the author 22 About the cover illustration 22 Part 1 Measuring agile teams 24 1 Measuring agile performance 26 1.1 Collect, measure, react, repeat—the feedback loop 27 1.1.1 What are metrics? 28 1.2 Why agile teams struggle with measurement 28 1.2.1 Problem: agile definitions of measurement are not straightforward 29 1.2.2 Problem: agile focuses on a product, not a project 30 1.2.3 Problem: data is all over the place without a unified view 31 1.3 What questions can metrics answer, and where do I get the data to answer them? 32 1.3.1 Project tracking 33 1.3.2 Source control 34 1.3.3 The build system 34 1.3.4 System monitoring 35 1.4 Analyzing what you have and what to do with the data 36 1.4.1 Figuring out what matters 37 1.4.2 Visualizing your data 37 1.5 Applying lessons learned 39 1.6 Taking ownership and measuring your team 39 1.6.1 Getting buy-in 40 1.6.2 Metric naysayers 41 1.7 Summary 42 2 Observing a live project 43 2.1 A typical agile project 43 2.1.1 How Blastamo Music used agile 44 2.2 A problem arises 44 2.3 Determining the right solution 45 2.4 Analyzing and presenting the data 49 2.4.1 Solving the problems 50 2.4.2 Visualizing the final product for leadership 51 2.5 Building on the system and improving their processes 54 2.5.1 Using data to improve what they do every day 55 2.6 Summary 56 Part 2 Collecting and analyzing your team’s data 58 3 Trends and data from project-tracking systems 60 3.1 Typical agile measurements using PTS data 62 3.1.1 Burn down 62 3.1.2 Velocity 63 3.1.3 Cumulative flow 64 3.1.4 Lead time 65 3.1.5 Bug counts 65 3.2 Prepare for analysis; generate the richest set of data you can 67 3.2.1 Tip 1: Make sure everyone uses your PTS 68 3.2.2 Tip 2: Tag tasks with as much data as possible 69 3.2.3 Tip 3: Estimate how long you think your tasks will take 70 3.2.4 Tip 4: Clearly define when tasks are done 72 3.2.5 Tip 5: Clearly define when tasks are completed in a good way 73 3.3 Key project management metrics; spotting trends in data 75 3.3.1 Task volume 75 3.3.2 Bugs 76 3.3.3 Measuring task movement; recidivism and workflow 77 3.3.4 Sorting with tags and labels 78 3.4 Case study: identifying tech debt trending with project tracking data 80 3.5 Summary 83 4 Trends and data from source control 85 4.1 What is source control management? 86 4.2 Preparing for analysis: generate the richest set of data you can 87 4.2.1 Tip 1: Use distributed version control and pull requests 88 4.3 The data you’ll be working with; what you can get from SCM 91 4.3.1 The data you can get from a DVCS 91 4.3.2 Data you can get from centralized SCM 94 4.3.3 What you can tell from SCM alone 94 4.4 Key SCM metrics: spotting trends in your data 100 4.4.1 Charting SCM activity 101 4.5 Case study: moving to the pull request workflow and incorporating quality engineering 102 4.6 Summary 105 5 Trends and data from CI and deployment servers 107 5.1 What is continuous development? 109 5.1.1 Continuous integration 109 5.1.2 Continuous delivery 111 5.1.3 Continuous testing 112 5.2 Preparing for analysis: generate the richest set of data you can 113 5.2.1 Set up a delivery pipeline 113 5.3 The data you’ll be working with: what you can get from your CI APIs 114 5.3.1 The data you can get from your CI server 114 5.3.2 What you can tell from CI alone 119 5.4 Key CI metrics: spotting trends in your data 120 5.4.1 Getting CI data and adding it to your charts 120 5.5 Case study: measuring benefits of process change through CI data 124 5.6 Summary 128 6 Data from your production systems 130 6.1 Preparing for analysis: generating the richest set of data you can 132 6.1.1 Adding arbitrary metrics to your development cycle 133 6.1.2 Utilizing the features of your application performance monitoring system 136 6.1.3 Using logging best practices 138 6.1.4 Using social network interaction to connect with your consumers 139 6.2 The data you’ll be working with: what you can get from your APM systems 141 6.2.1 Server health statistics 141 6.2.2 Consumer usage 143 6.2.3 Semantic logging analysis 143 6.2.4 Tools used to collect production system data 144 6.3 Case study: a team moves to DevOps and continuous delivery 145 6.4 Summary 147 Part 3 Applying metrics to your teams, processes, and software 148 7 Working with the data you’re collecting: the sum of the parts 150 7.1 Combining data points to create metrics 150 7.2 Using your data to define “good” 152 7.2.1 Turning subjectivity into objectivity 153 7.2.2 Working backward from good releases 155 7.3 How to create metrics 158 7.3.1 Step 1: explore your data 159 7.3.2 Step 2: break it down—determine what to track 162 7.3.3 Step 3: create formulas around multiple data points to create metrics 163 7.4 Case study: creating and using a new metric to measure continuous release quality 167 7.5 Summary 176 8 Measuring the technical quality of your software 177 8.1 Preparing for analysis: setting up to measure your code 178 8.2 Measuring the NFRs through the code “ilities” 179 8.3 Measuring maintainability 181 8.3.1 MTTR and lead time 181 8.3.2 Adding SCM and build data 185 8.3.3 Code coverage 187 8.3.4 Adding static code analysis 188 8.3.5 Adding more PTS data 190 8.4 Measuring usability 191 8.4.1 Reliability and availability 192 8.4.2 Security 194 8.5 Case study: finding anomalies in lead time 196 8.6 Summary 199 9 Publishing metrics 200 9.1 The right data for the right audience 201 9.1.1 What to use on your team 203 9.1.2 What managers want to see 207 9.1.3 What executives care about 211 9.1.4 Using metrics to prove a point or effect change 212 9.2 Different publishing methods 214 9.2.1 Building dashboards 215 9.2.2 Using email 216 9.3 Case study: driving visibility toward a strategic goal 217 9.4 Summary 222 10 Measuring your team against the agile principles 224 10.1 Breaking the agile principles into measurable components 225 10.1.1 Aligning the principles with the delivery lifecycle 227 10.2 Three principles for effective software 228 10.2.1 Measuring effective software 229 10.3 Four principles for effective process 230 10.3.1 Measuring effective processes 231 10.4 Four principles for an effective team 233 10.4.1 Measuring an effective development team 233 10.5 One principle for effective requirements 236 10.5.1 Measuring effective requirements 236 10.6 Case study: a new agile team 238 10.7 Summary 242 appendix A DIY analytics using ELK 244 A.1 Setting up your system 247 A.1.1 Checking the database 249 A.1.2 Configuring your data collector 249 A.2 Creating the dashboard 251 A.3 Summary 251 appendix B Collecting data from source systems with Grails 252 B.1 Architectural overview 253 B.1.1 Domain objects 255 B.1.2 The data you’re working with 256 B.1.3 Data collection services 258 B.1.4 Scheduling jobs for data collection 260 B.2 Summary 262 index 264 A 264 B 264 C 265 D 265 E 265 F 266 G 266 H 266 I 266 J 266 K 266 L 266 M 266 N 267 O 267 P 267 Q 268 R 268 S 268 T 268 U 269 V 269 W 269 X 269 Y 269 Agile Metrics back 270 SummaryAgile Metrics in Action is a rich resource for agile teams that aim to use metrics to objectively measure performance. You'll learn how to gather data that really counts, along with how to effectively analyze and act upon the results.Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.About the BookThe iterative nature of agile development is perfect for experience-based, continuous improvement. Tracking systems, test and build tools, source control, continuous integration, and other built-in parts of a project lifecycle throw off a wealth of data you can use to improve your products, processes, and teams. The question is, how to do it?Agile Metrics in Action teaches you how. This practical book is a rich resource for an agile team that aims to use metrics to objectively measure performance. You'll learn how to gather the data that really count, along with how to effectively analyze and act upon the results. Along the way, you'll discover techniques all team members can use for better individual accountability and team performance.Practices in this book will work with any development process or tool stack. For code-based examples, this book uses Groovy, Grails, and MongoDB.What's InsideUse the data you generate every day from CI and ScrumImprove communication, productivity, transparency, and moraleObjectively measure performanceMake metrics a natural byproduct of your development processAbout the AuthorChristopher Davis has been a software engineer and team leader for over 15 years. He has led numerous teams to successful delivery using agile methodologies.Table of ContentsPART 1 MEASURING AGILE TEAMSMeasuring agile performanceObserving a live projectPART 2 COLLECTING AND ANALYZING YOUR TEAM'S DATATrends and data from project-tracking systemsTrends and data from source controlTrends and data from CI and deployment serversData from your production systemsPART 3 APPLYING METRICS TO YOUR TEAMS, PROCESSES, AND SOFTWAREWorking with the data you're collecting: the sum of the partsMeasuring the technical quality of your softwarePublishing metricsMeasuring your team against the agile principles Agile Metrics in Action is a rich resource for agile teams that aim to use metrics to objectively measure performance. You'll learn how to gather data that really count, along with how to effectively analyze and act upon the results. About the Book The iterative nature of agile development is perfect for experience-based, continuous improvement. Tracking systems, test and build tools, source control, continuous integration, and other built-in parts of a project lifecycle throw off a wealth of data you can use to improve your products, processes, and teams. The question is, how to do it? Agile Metrics in Action teaches you how. This practical book is a rich resource for an agile team that aims to use metrics to objectively measure performance. You'll learn how to gather the data that really count, along with how to effectively analyze and act upon the results. Along the way, you'll discover techniques all team members can use for better individual accountability and team performance. Practices in this book will work with any development process or tool stack. For code-based examples, this book uses Groovy, Grails, and MongoDB. What's Inside Use the data you generate every day from CI and Scrum Improve communication, productivity, transparency, and morale Objectively measure performance Make metrics a natural byproduct of your development process About the Author Christopher Davis has been a software engineer and team leader for over 15 years. He has led numerous teams to successful delivery using agile methodologies "The iterative nature of agile development is perfect for experience-based, continuous improvement. Tracking systems, test and build tools, source control, continuous integration, and other built-in parts of a project lifecycle throw off a wealth of data you can use to improve your products, processes, and teams. The question is, how to do it? Agile Metrics in Action teaches you how. This practical book is a rich resource for an agile team that aims to use metrics to objectively measure performance. You'll learn how to gather the data that really count, along with how to effectively analyze and act upon the results. Along the way, you'll discover techniques all team members can use for better individual accountability and team performance"--Publisher's description