The Art of Computer Virus Research and Defense

"Of all the computer-related books I've read recently, this one influenced my thoughts about security the most. There is very little trustworthy information about computer viruses. Peter Szor is one of the best virus analysts in the world and has the perfect credentials to write this book." --Halvar Flake, Reverse Engineer, SABRE Security GmbH Symantec's chief antivirus researcher has written the definitive guide to contemporary virus threats, defense techniques, and analysis tools. Unlike most books on computer viruses, The Art of Computer Virus Research and Defense is a reference written strictly for white hats: IT and security professionals responsible for protecting their organizations against malware. Peter Szor systematically covers everything you need to know, including virus behavior and classification, protection strategies, antivirus and worm-blocking techniques, and much more. Szor presents the state-of-the-art in both malware and protection, providing the full technical detail that professionals need to handle increasingly complex attacks. Along the way, he provides extensive information on code metamorphism and other emerging techniques, so you can anticipate and prepare for future threats. Szor also offers the most thorough and practical primer on virus analysis ever published--addressing everything from creating your own personal laboratory to automating the analysis process. This book's coverage includes Discovering how malicious code attacks on a variety of platforms Classifying malware strategies for infection, in-memory operation, self-protection, payload delivery, exploitation, and more Identifying and responding to code obfuscation threats: encrypted, polymorphic, and metamorphic Mastering empirical methods for analyzing malicious code--and what to do with what you learn Reverse-engineering malicious code with disassemblers, debuggers, emulators, and virtual machines Implementing technical defenses: scanning, code emulation, disinfection, inoculation, integrity checking, sandboxing, honeypots, behavior blocking, and much more Using worm blocking, host-based intrusion prevention, and network-level defense strategies © Copyright Pearson Education. All rights reserved TABLE OF CONTENTS 8 ABOUT THE AUTHOR 22 PREFACE 23 ACKNOWLEDGMENTS 26 PART I: Strategies of the Attacker 30 1 INTRODUCTION TO THE GAMES OF NATURE 32 1.1 Early Models of Self-Replicating Structures 33 1.2 Genesis of Computer Viruses 46 1.3 Automated Replicating Code: The Theory and Definition of Computer Viruses 47 References 49 2 THE FASCINATION OF MALICIOUS CODE ANALYSIS 52 2.1 Common Patterns of Virus Research 55 2.2 Antivirus Defense Development 56 2.3 Terminology of Malicious Programs 57 2.4 Other Categories 65 2.5 Computer Malware Naming Scheme 67 2.6 Annotated List of Officially Recognized Platform Names 71 References 75 3 MALICIOUS CODE ENVIRONMENTS 78 3.1 Computer Architecture Dependency 81 3.2 CPU Dependency 82 3.3 Operating System Dependency 84 3.4 Operating System Version Dependency 84 3.5 File System Dependency 85 3.6 File Format Dependency 88 3.7 Interpreted Environment Dependency 95 3.8 Vulnerability Dependency 127 3.9 Date and Time Dependency 127 3.10 JIT Dependency: Microsoft .NET Viruses 128 3.11 Archive Format Dependency 129 3.12 File Format Dependency Based on Extension 130 3.13 Network Protocol Dependency 131 3.14 Source Code Dependency 131 3.15 Resource Dependency on Mac and Palm Platforms 133 3.16 Host Size Dependency 134 3.17 Debugger Dependency 135 3.18 Compiler and Linker Dependency 137 3.19 Device Translator Layer Dependency 138 3.20 Embedded Object Insertion Dependency 141 3.21 Self-Contained Environment Dependency 142 3.22 Multipartite Viruses 144 3.23 Conclusion 145 References 146 4 CLASSIFICATION OF INFECTION STRATEGIES 150 4.1 Boot Viruses 151 4.2 File Infection Techniques 158 4.3 An In-Depth Look at Win32 Viruses 186 4.4 Conclusion 212 References 213 5 CLASSIFICATION OF IN-MEMORY STRATEGIES 214 5.1 Direct-Action Viruses 215 5.2 Memory-Resident Viruses 215 5.3 Temporary Memory-Resident Viruses 239 5.4 Swapping Viruses 240 5.5 Viruses in Processes (in User Mode) 240 5.6 Viruses in Kernel Mode (Windows 9x/Me) 241 5.7 Viruses in Kernel Mode (Windows NT/2000/XP) 242 5.8 In-Memory Injectors over Networks 244 References 245 6 BASIC SELF-PROTECTION STRATEGIES 246 6.1 Tunneling Viruses 247 6.2 Armored Viruses 249 6.3 Aggressive Retroviruses 276 References 279 7 ADVANCED CODE EVOLUTION TECHNIQUES AND COMPUTER VIRUS GENERATOR KITS 280 7.1 Introduction 281 7.2 Evolution of Code 281 7.3 Encrypted Viruses 282 7.4 Oligomorphic Viruses 288 7.5 Polymorphic Viruses 290 7.6 Metamorphic Viruses 298 7.7 Virus Construction Kits 317 References 322 8 CLASSIFICATION ACCORDING TO PAYLOAD 324 8.1 No-Payload 325 8.2 Accidentally Destructive Payload 326 8.3 Nondestructive Payload 326 8.4 Somewhat Destructive Payload 329 8.5 Highly Destructive Payload 330 8.6 DoS (Denial of Service) Attacks 335 8.7 Data Stealers: Making Money with Viruses 337 8.8 Conclusion 341 References 341 9 STRATEGIES OF COMPUTER WORMS 342 9.1 Introduction 343 9.2 The Generic Structure of Computer Worms 344 9.3 Target Locator 348 9.4 Infection Propagators 360 9.5 Common Worm Code Transfer and Execution Techniques 367 9.6 Update Strategies of Computer Worms 374 9.7 Remote Control via Signaling 380 9.8 Intentional and Accidental Interactions 383 9.9 Wireless Mobile Worms 388 References 390 10 EXPLOITS, VULNERABILITIES, AND BUFFER OVERFLOW ATTACKS 394 10.1 Introduction 395 10.2 Background 396 10.3 Types of Vulnerabilities 397 10.4 Current and Previous Threats 423 10.5 Summary 448 References 449 Part II: STRATEGIES OF THE DEFENDER 452 11 ANTIVIRUS DEFENSE TECHNIQUES 454 11.1 First-Generation Scanners 457 11.2 Second-Generation Scanners 466 11.3 Algorithmic Scanning Methods 470 11.4 Code Emulation 480 11.5 Metamorphic Virus Detection Examples 490 11.6 Heuristic Analysis of 32-Bit Windows Viruses 496 11.7 Heuristic Analysis Using Neural Networks 501 11.8 Regular and Generic Disinfection Methods 503 11.9 Inoculation 510 11.10 Access Control Systems 511 11.11 Integrity Checking 513 11.12 Behavior Blocking 516 11.13 Sand-Boxing 518 11.14 Conclusion 520 References 520 12 MEMORY SCANNING AND DISINFECTION 524 12.1 Introduction 526 12.2 The Windows NT Virtual Memory System 528 12.3 Virtual Address Spaces 530 12.4 Memory Scanning in User Mode 534 12.5 Memory Scanning and Paging 544 12.6 Memory Disinfection 546 12.7 Memory Scanning in Kernel Mode 552 12.8 Possible Attacks Against Memory Scanning 561 12.9 Conclusion and Future Work 563 References 564 13 WORM-BLOCKING TECHNIQUES AND HOST-BASED INTRUSION PREVENTION 566 13.1 Introduction 567 13.2 Techniques to Block Buffer Overflow Attacks 572 13.3 Worm-Blocking Techniques 586 13.4 Possible Future Worm Attacks 604 13.5 Conclusion 607 References 609 14 NETWORK-LEVEL DEFENSE STRATEGIES 612 14.1 Introduction 613 14.2 Using Router Access Lists 614 14.3 Firewall Protection 617 14.4 Network-Intrusion Detection Systems 620 14.5 Honeypot Systems 622 14.6 Counterattacks 625 14.7 Early Warning Systems 627 14.8 Worm Behavior Patterns on the Network 627 14.9 Conclusion 638 References 638 15 MALICIOUS CODE ANALYSIS TECHNIQUES 640 15.1 Your Personal Virus Analysis Laboratory 641 15.2 Information, Information, Information 644 15.3 Dedicated Virus Analysis on VMWARE 645 15.4 The Process of Computer Virus Analysis 647 15.5 Maintaining a Malicious Code Collection 690 15.6 Automated Analysis: The Digital Immune System 690 References 694 16 CONCLUSION 696 Further Reading 698 INDEX 704 A 704 B 707 C 709 D 711 E 713 F 715 G 716 H 717 I 718 J 720 K 720 L 721 M 721 N 724 O 725 P 726 Q 728 R 728 S 729 T 733 U 734 V 734 W 736 X 742 Y 742 Z 742