Building your own test framework : a practical guide to writing better automated tests

Learn to write better automated tests that will dramatically increase your productivity and have fun while doing so. This book is a build-your-own adventure designed for individual reading and for collaborative workshops. You will build an xUnit automated test framework using initially a clone of Jest, but adding a couple of neat features borrowed from RSpec, the genre-defining tool for behavior-driven development (BDD). Along the way, you will explore the philosophy behind automated testing best practices. The automated test runner is one of the most important innovations within software engineering. But for many programmers, automated testing remains a mystery, and knowing how to write good tests is akin to sorcery. As the chapters of this book unfold, you will see how the humble test runner is an elegant and simple piece of software. Each chapter picks a single feature to build, like the "it" function or the "beforeEach" block. It picks apart the theory of why the feature needs to exist, and how to use it effectively in your own test suites. Every chapter ends with a set of ideas for extension points should you wish to explore further, alone or in groups. The book culminates in an implementation of test doubles and mocks―one of the most difficult and misunderstood concepts within automated testing. By the end of the book, you will have gained a solid understanding of automated testing principles that you can immediately apply to your work projects. What You'll Learn Build an xUnit automated test framework See how an automated test runner works Understand the best practices for automated unit testing Effectively use test doubles and mocks Who This Book Is For Software developers with JavaScript experience who are seeking to master the art of automated testing. Table of Contents 4 About the Author 12 About the Technical Reviewer 13 Acknowledgments 14 Foreword 15 Introduction 16 Chapter 1: Creating a Barebones Test Runner 22 What Is an Entrypoint? 22 The First Stage of Automated Test Enlightenment 24 The Second Stage of Automated Test Enlightenment 25 The Inner Workings of a Unit Test Runner 26 The Third Stage of Automated Test Enlightenment 27 Cloning the Project Repository 29 Creating an NPM Project for the Test Runner 30 Setting Package Configuration 30 Creating the Test Runner Entrypoint 31 Making the Package Available for Use 32 Using the Sample Application Project 33 npm unlink 33 Update the Test Runner to Run Our Test Suite 34 Verify That It Works by Breaking a Test 36 Catch Exceptions from the Await Call 38 Committing to Git 39 Discussion Questions 40 Summary 40 Chapter 2: Building a Function to Represent a Test Case 41 The it Function 42 Common Test Verbs 43 Using the it Function to Group and Describe Test Cases 44 Handling Exceptions 51 Printing Test Descriptions 52 Support CI with Correct Exit Codes 55 Summarizing a Test Run 57 Exercises 58 Summary 59 Chapter 3: Grouping Tests 60 The Basics of Unit Test Organization 60 A Starting Point for describe 62 Rethinking Test Output 63 Printing a Pass/Fail Response for Each Test 65 Ending a Test Report with Test Failure Details 66 Saving Test Context Information 69 Supporting Nested describe Blocks 71 Exercises 77 Summary 77 Chapter 4: Promoting Conciseness with Shared Setup and Teardown 78 The Arrange, Act, Assert Pattern 79 Arrange 80 Act 80 Assert 80 Why Is This Pattern Important? 81 Introducing the beforeEach Function 83 Applying beforeEach Blocks to Our Test 84 Refactoring describeStack to an Object 86 Defining beforeEach 87 Updating the Sample Application 89 Defining afterEach 92 Generalizing beforeEach 92 Generalizing invokeBefores 93 Exercise 94 Discussion Question 95 Summary 95 Chapter 5: Improving Legibility with Expectations and Matchers 96 Using Matchers for Clarity and Reuse 97 Building the First Matcher: toBeDefined 98 Creating an Error Subtype 105 Allowing Multiple Failures per Test 108 Making a Mess 109 Testing Your Solution 112 Exercises 113 Discussion Questions 114 Summary 115 Chapter 6: Formatting Expectation Errors 116 Utilizing Stack Traces with Testing Workflows 116 Building stackTraceFormatter 120 Joining Up Our Formatter with the Runner 130 Exercises 132 Discussion Questions 132 Summary 133 Chapter 7: Automatically Discovering Test Files 134 Development Workflows and Test Runners 134 What About Watch Mode? 136 Discovering Files 137 Running a Single File 139 Exercises 141 Discussion Question 142 Summary 143 Chapter 8: Focusing on Specific Tests 144 The Refactor Workflow 145 Introducing a Parse Step 145 Implementing the Parse Phase 146 Parsing the describe Function 147 Adding the it Function 149 Adding the beforeEach and afterEach Functions 150 Implementing the Run Phase 151 The Global currentTest Variable 153 Updating the Runner 154 Moving the expect Function 156 Adding the Focus Functions 157 Filtering Tests 158 Exercises 160 Discussion Question 160 Summary 161 Chapter 9: Supporting Asynchronous Tests 162 Event Loops in Runtime Environments 163 Synchronicity and the Test Runner 164 Waiting for Promise Completion 165 Testing It Out 168 Catching Exceptions 168 Timing Out Tests After a Period of Time with the it.timesOutAfter Modifier 169 Testing It Out 172 Exercise 173 Discussion Questions 173 Summary 174 Chapter 10: Reporting 175 The Observer Pattern, in a Half-Baked Style 175 Adding an Event Dispatcher 176 Dispatching a Begin Suite Event 178 Dispatching a Test Finished Event 180 Adding finishedTestRun 182 Getting Rid of Failures and Successes 183 Exercises 186 Summary 187 Chapter 11: Shared Examples 188 Methods of Reuse 188 Implementing a Shared Example Repository 191 Importing Shared Examples 195 Exercise 197 Discussion Questions 197 Summary 197 Chapter 12: Tagging Tests 198 Another Way of Slicing Tests 199 Thinking Through Design Options 200 Jest’s Approach 200 Supporting Polymorphic Calls to Test Functions 203 Filtering Test Runs Using the tags Property 207 Reading Tags from the Process Arguments 209 Adding Tags to the Sample Application 210 Exercises 211 Discussion Question 211 Summary 211 Chapter 13: Skipping Tests 213 Taming the Test Runner 213 Adding the it.skip Modifier 215 Testing It Out 216 Supporting Empty-Body describe and it Calls 218 Exercises 219 Discussion Question 220 Summary 220 Chapter 14: Randomizing Tests 221 Test Runner Use Cases 222 Randomizing Tests in CI Environments 222 Randomizing Tests on Developer Machines 223 Adding the Flag 223 Testing It Out 225 Exercises 226 Discussion Question 226 Summary 226 Chapter 15: Deep Equality and Constraining Matchers 228 How Are Constraining Functions Useful? 229 Implementing the equals Function 230 Checking if Two Arrays Have Equal Size 231 Checking if Two Objects Have Equal Keys 231 Recursively Checking Element Equality 232 Implementing the contains Constraint 233 Exercises 235 Discussion Questions 236 Summary 237 Chapter 16: Test Doubles 238 How Are Test Doubles Useful? 238 Spies and Stubs 239 Adding the New Tests 241 Implementing the spy Function 243 Implementing the toBeCalledWith Matcher 244 Exercises 245 Discussion Questions 246 Summary 247 Chapter 17: Module Mocks 248 How Are Module Mocks Useful? 249 The Module Mock API 250 Implementing the Mock Registry 252 Implementing the Node Loader 253 Adding the New Bin Script 256 Bypassing the Module Cache 257 Creating a Forwarding Reporter 259 Defining the Worker Thread Entrypoint 260 Updating the Test Runner to Use Worker Threads 261 Changing Reported Test Objects to Be Serializable 264 Exercises 266 Discussion Question 266 Summary 267