Implementing Automated Software Testing: How to Save Time and Lower Costs While Raising Quality: How to Save Time and Lower Costs While Raising Quality

"This book fills a huge gap in our knowledge of software testing. It does an excellent job describing how test automation differs from other test activities, and clearly lays out what kind of skills and knowledge are needed to automate tests. The book is essential reading for students of testing and a bible for practitioners." --Jeff Offutt, Professor of Software Engineering, George Mason University "This new book naturally expands upon its predecessor, Automated Software Testing, and is the perfect reference for software practitioners applying automated software testing to their development efforts. Mandatory reading for software testing professionals!" --Jeff Rashka, PMP, Coauthor of Automated Software Testing and Quality Web Systems Testing accounts for an increasingly large percentage of the time and cost of new software development. Using automated software testing (AST), developers and software testers can optimize the software testing lifecycle and thus reduce cost. As technologies and development grow increasingly complex, AST becomes even more indispensable. This book builds on some of the proven practices and the automated testing lifecycle methodology (ATLM) described in Automated Software Testing and provides a renewed practical, start-to-finish guide to implementing AST successfully. In Implementing Automated Software Testing, three leading experts explain AST in detail, systematically reviewing its components, capabilities, and limitations. Drawing on their experience deploying AST in both defense and commercial industry, they walk you through the entire implementation process--identifying best practices, crucial success factors, and key pitfalls along with solutions for avoiding them. You will learn how to: Make a realistic business case for AST, and use it to drive your initiative Clarify your testing requirements and develop an automation strategy that reflects them Build efficient test environments and choose the right automation tools and techniques for your environment Use proven metrics to continuously track your progress and adjust accordingly Whether you're a test professional, QA specialist, project manager, or developer, this book can help you bring unprecedented efficiency to testing--and then use AST to improve your entire development lifecycle. Cover......Page 1 Contents......Page 6 Foreword......Page 16 Foreword......Page 18 Preface......Page 20 Acknowledgments......Page 24 About the Authors......Page 26 I. What Is Automated Software Testing and Why Should We Automate?......Page 28 1. What Is Effective Automated Software Testing (AST)?......Page 30 1.1 Automated Software Testing Definition......Page 31 1.2 Automated Software Testing Recipes......Page 32 1.3 Advances in AST Technologies......Page 35 1.4 Automating Various Software Testing Types......Page 38 1.5 Providing AST-Based Production Support......Page 43 1.6 Automating Standards Assessments......Page 45 Summary......Page 47 Notes......Page 48 2. Why Automate?......Page 50 2.1 The Challenges of Testing Software Today......Page 51 2.2 Reducing the Time and Cost of Software Testing......Page 53 2.3 Impacting Software Quality......Page 65 2.4 Improvements to Your Software Test Program......Page 69 Summary......Page 76 Notes......Page 77 3.1 Definition of the Business Case......Page 78 3.2 Identifying the Business Needs......Page 80 3.3 Justifying Automation in Terms of Cost and Benefits......Page 82 3.4 Risks......Page 92 3.5 Other Considerations......Page 94 Notes......Page 95 4. Why Automated Software Testing Fails and Pitfalls to Avoid......Page 96 4.1 R&D Does Not Generally Focus on Automated or Manual Testing Efforts......Page 98 4.2 AST Myths and Realities......Page 101 4.3 Lack of Software Development Considerations for AST......Page 110 4.4 The Forest for the Trees—Not Knowing Which Tool to Pick......Page 118 4.5 Lack of Automation Standards across Tool Vendors......Page 121 Notes......Page 124 II. How to Automate: Top Six Keys for Automation Payoff......Page 126 5. Key 1: Know Your Requirements......Page 128 5.1 Understand the AST Support Requirements......Page 129 5.2 Additional Information in Support of AST Requirements......Page 141 5.3 When Information Is Not Available......Page 143 5.4 Start Implementing Your Requirements Traceability Matrix (RTM)......Page 151 Notes......Page 155 6. Key 2: Develop the Automated Test Strategy......Page 156 6.1 The AST Strategy Document......Page 158 6.2 Scope and Automated Test Objectives......Page 159 6.3 Identify the Approach......Page 166 6.4 Automated Software Test Framework (ASTF)......Page 173 6.5 AST Environment/Configuration......Page 177 6.6 Automating the RTM......Page 186 Summary......Page 191 Notes......Page 192 7. Key 3: Test the Automated Software Test Framework (ASTF)......Page 194 7.1 Verify That the ASTF Meets Specified Requirements and That Features Behave As Expected......Page 196 7.2 Peer-Review All ASTF-Related Artifacts, Including Design, Development, and Test Cases......Page 197 7.3 Verify Requirements and Coverage......Page 209 7.4 Hold a Customer Review......Page 210 Notes......Page 211 8. Key 4: Continuously Track Progress—and Adjust Accordingly......Page 214 8.1 AST Program Tracking and Defect Prevention......Page 215 8.2 AST Metrics......Page 219 8.3 Root Cause Analysis......Page 232 Summary......Page 233 Notes......Page 234 9. Key 5: Implement AST Processes......Page 236 9.1 AST Phases and Milestones......Page 238 9.2 AST Phase 1: Requirements Gathering—Analyze Automated Testing Needs......Page 239 9.3 AST Phase 2: Test Case Design and Development......Page 242 9.4 AST Phase 3: Automated Software Testing Framework (ASTF) and Test Script Development......Page 243 9.5 AST Phase 4: Automated Test Execution and Results Reporting......Page 244 9.6 AST Phase 5: Program Review and Assessment......Page 245 9.7 Virtual Quality Gates......Page 246 9.8 Process Measurement......Page 247 Summary......Page 248 Notes......Page 249 10. Key 6: Put the Right People on the Project—Know the Skill Sets Required......Page 250 10.1 Program Management......Page 259 10.2 Systems Engineering......Page 263 10.3 Software Development......Page 266 10.4 Configuration Management......Page 269 10.5 Quality Assurance......Page 271 10.6 Subject Matter Experts (SMEs)......Page 273 Summary......Page 274 Notes......Page 275 Appendices......Page 276 A. Process Checklist......Page 278 A.1 AST Phase 1: Requirements Gathering—Analyze Automated Testing Needs......Page 279 A.2 AST Phase 2: Test Case Design and Development......Page 280 A.3 AST Phase 3: Automated Software Testing Framework (ASTF) and Test Script Development......Page 281 A.4 AST Phase 4: Automated Test Execution and Results Reporting......Page 282 A.5 AST Phase 5: Program Review and Assessment......Page 283 B.1 Security Testing......Page 284 B.2 Soak Testing......Page 288 B.3 Concurrency Testing......Page 290 B.4 Performance Testing......Page 292 B.5 Code Coverage Testing......Page 296 B.6 Unit Testing......Page 298 Notes......Page 301 C. The Right Tool for the Job......Page 302 C.1 Requirements Management (RM)......Page 303 C.2 Unit Testing Frameworks—Example Evaluation......Page 308 C.3 Configuration Management—Example Evaluation......Page 311 C.4 Defect Tracking—Example Evaluation......Page 319 C.5 Security Testing......Page 326 C.6 Automated Software Testing Framework (ASTF)—Example Evaluation......Page 333 C.7 Other STL Support Tools......Page 343 Notes......Page 346 D. Case Study: An Automated Software Testing Framework (ASTF) Example......Page 348 D.1 Key Design Features......Page 350 D.2 Test Manager......Page 352 D.3 More on Automated Test Case and Test Code Generation......Page 353 D.5 Automated Defect Reporting......Page 355 Notes......Page 356 Contributing Authors......Page 358 A......Page 360 B......Page 363 I......Page 364 R......Page 365 T......Page 366 W......Page 367 __"This book fills a huge gap in our knowledge of software testing. It does an excellent job describing how test automation differs from other test activities, and clearly lays out what kind of skills and knowledge are needed to automate tests. The book is essential reading for students of testing and a bible for practitioners."__--Jeff Offutt, Professor of Software Engineering, George Mason University__"This new book naturally expands upon its predecessor,__Automated Software Testing,__and is the perfect reference for software practitioners applying automated software testing to their development efforts. Mandatory reading for software testing professionals!"__--Jeff Rashka, PMP, Coauthor ofTesting accounts for an increasingly large percentage of the time and cost of new software development. Using automated software testing (AST), developers and software testers can optimize the software testing lifecycle and thus reduce cost. As technologies and development grow increasingly complex, AST becomes even more indispensable. This book builds on some of the proven practices and the automated testing lifecycle methodology (ATLM) described inand provides a renewed practical, start-to-finish guide to implementing AST successfully.Inthree leading experts explain AST in detail, systematically reviewing its components, capabilities, and limitations. Drawing on their experience deploying AST in both defense and commercial industry, they walk you through the entire implementation process--identifying best practices, crucial success factors, and key pitfalls along with solutions for avoiding them. You will learn how to:Make a realistic business case for AST, and use it to drive your initiative Clarify your testing requirements and develop an automation strategy that reflects them Build efficient test environments and choose the right automation tools and techniques for your environment Use proven metrics to continuously track your progress and adjust accordingly Whether you're a test professional, QA specialist, project manager, or developer, this book can help you bring unprecedented efficiency to testing--and then use AST to improve your entire development lifecycle. "This book fills a huge gap in our knowledge of software testing. It does an excellent job describing how test automation differs from other test activities, and clearly lays out what kind of skills and knowledge are needed to automate tests. The book is essential reading for students of testing and a bible for practitioners." Jeff Offutt, Professor of Software Engineering, George Mason University "This new book naturally expands upon its predecessor, Automated Software Testing, and is the perfect reference for software practitioners applying automated software testing to their development efforts. Mandatory reading for software testing professionals!" Jeff Rashka, PMP, Coauthor of Automated Software Testing and Quality Web Systems Testing accounts for an increasingly large percentage of the time and cost of new software development. Using automated software testing (AST), developers and software testers can optimize the software testing lifecycle and thus reduce cost. As technologies and development grow increasingly complex, AST becomes even more indispensable. This book builds on some of the proven practices and the automated testing lifecycle methodology (ATLM) described in Automated Software Testing and provides a renewed practical, start-to-finish guide to implementing AST successfully. InImplementing Automated Software Testing, three leading experts explain AST in detail, systematically reviewing its components, capabilities, and limitations. Drawing on their experience deploying AST in both defense and commercial industry, they walk you through the entire implementation process identifying best practices, crucial success factors, and key pitfalls along with solutions for avoiding them. You will learn how to: Make a realistic business case for AST, and use it to drive your initiative Clarify your testing requirements and develop an automation strategy that reflects them Build efficient test environments and choose the right automation tools and techniques for