Learning Python : Includes index

Portable, powerful, and a breeze to use, Python is ideal for both standalone programs and scripting applications. With this hands-on book, you can master the fundamentals of the core Python language quickly and efficiently, whether you're new to programming or just new to Python. Once you finish, you will know enough about the language to use it in any application domain you choose. Learning Python is based on material from author Mark Lutz's popular training courses, which he's taught over the past decade. Each chapter is a self-contained lesson that helps you thoroughly understand a key component of Python before you continue. Along with plenty of annotated examples, illustrations, and chapter summaries, every chapter also contains Brain Builder , a unique section with practical exercises and review quizzes that let you practice new skills and test your understanding as you go. This book covers: Types and Operations -- Python's major built-in object types in depth: numbers, lists, dictionaries, and more Statements and Syntax -- the code you type to create and process objects in Python, along with Python's general syntax model Functions -- Python's basic procedural tool for structuring and reusing code Modules -- packages of statements, functions, and other tools organized into larger components Classes and OOP -- Python's optional object-oriented programming tool for structuring code for customization and reuse Exceptions and Tools -- exception handling model and statements, plus a look at development tools for writing larger programs Learning Python gives you a deep and complete understanding of the language that will help you comprehend any application-level examples of Python that you later encounter. If you're ready to discover what Google and YouTube see in Python, this book is the best way to get started. Read more... Abstract: Portable, powerful, and a breeze to use, Python is ideal for both standalone programs and scripting applications. With this hands-on book, you can master the fundamentals of the core Python language quickly and efficiently, whether you're new to programming or just new to Python. Once you finish, you will know enough about the language to use it in any application domain you choose. Learning Python is based on material from author Mark Lutz's popular training courses, which he's taught over the past decade. Each chapter is a self-contained lesson that helps you thoroughly understand a key component of Python before you continue. Along with plenty of annotated examples, illustrations, and chapter summaries, every chapter also contains Brain Builder , a unique section with practical exercises and review quizzes that let you practice new skills and test your understanding as you go. This book covers: Types and Operations -- Python's major built-in object types in depth: numbers, lists, dictionaries, and more Statements and Syntax -- the code you type to create and process objects in Python, along with Python's general syntax model Functions -- Python's basic procedural tool for structuring and reusing code Modules -- packages of statements, functions, and other tools organized into larger components Classes and OOP -- Python's optional object-oriented programming tool for structuring code for customization and reuse Exceptions and Tools -- exception handling model and statements, plus a look at development tools for writing larger programs Learning Python gives you a deep and complete understanding of the language that will help you comprehend any application-level examples of Python that you later encounter. If you're ready to discover what Google and YouTube see in Python, this book is the best way to get started Learning Python, Third Edition 1 Table of Contents 8 Preface 30 About This Third Edition 30 This Edition’s Python Language Changes 30 This Edition’s Python Training Changes 31 This Edition’s Structural Changes 32 This Edition’s Scope Changes 33 About This Book 33 This Book’s Prerequisites 33 This Book’s Scope and Other Books 34 This Book’s Style and Structure 35 Book Updates 37 About the Programs in This Book 37 Preparing for Python 3.0 38 About This Series 41 Using Code Examples 41 Font Conventions 42 Safari® Books Online 43 How to Contact Us 43 Acknowledgments 43 Part I 46 A Python Q&A Session 48 Why Do People Use Python? 48 Software Quality 50 Developer Productivity 50 Is Python a “Scripting Language”? 51 OK, but What’s the Downside? 52 Who Uses Python Today? 53 What Can I Do with Python? 54 Systems Programming 54 GUIs 54 Internet Scripting 55 Component Integration 55 Database Programming 56 Rapid Prototyping 56 Numeric and Scientific Programming 56 Gaming, Images, AI, XML, Robots, and More 57 What Are Python’s Technical Strengths? 57 It’s Object Oriented 57 It’s Free 58 It’s Portable 58 It’s Powerful 59 It’s Mixable 60 It’s Easy to Use 60 It’s Easy to Learn 62 It’s Named After Monty Python 62 How Does Python Stack Up to Language X? 63 Chapter Summary 64 Quiz Answers 65 How Python Runs Programs 67 Introducing the Python Interpreter 67 Program Execution 69 The Programmer’s View 69 Python’s View 70 Byte code compilation 70 The Python Virtual Machine (PVM) 71 Performance implications 71 Development implications 72 Execution Model Variations 72 Python Implementation Alternatives 73 CPython 73 Jython 73 IronPython 74 Execution Optimization Tools 74 The Psyco just-in-time compiler 74 The Shedskin C++ translator 75 Frozen Binaries 76 Future Possibilities? 77 Chapter Summary 77 Quiz Answers 78 How You Run Programs 79 Interactive Coding 79 Using the Interactive Prompt 82 System Command Lines and Files 82 Using Command Lines and Files 85 Unix Executable Scripts (#!) 86 Clicking File Icons 87 Clicking Icons on Windows 87 The raw_input Trick 89 Other Icon-Click Limitations 90 Module Imports and Reloads 90 The Grander Module Story: Attributes 92 Modules and namespaces 94 import and reload Usage Notes 94 The IDLE User Interface 95 IDLE Basics 96 Using IDLE 97 Advanced IDLE Tools 99 Other IDEs 99 Embedding Calls 101 Frozen Binary Executables 101 Text Editor Launch Options 102 Other Launch Options 102 Future Possibilities? 102 Which Option Should I Use? 103 Chapter Summary 103 Quiz Answers 104 Part II 108 Introducing Python Object Types 110 Why Use Built-in Types? 111 Python’s Core Data Types 112 Numbers 113 Strings 114 Sequence Operations 115 Immutability 116 Type-Specific Methods 117 Getting Help 118 Other Ways to Code Strings 119 Pattern Matching 120 Lists 120 Sequence Operations 121 Type-Specific Operations 121 Bounds Checking 122 Nesting 122 List Comprehensions 123 Dictionaries 124 Mapping Operations 124 Nesting Revisited 125 Sorting Keys: for Loops 126 Iteration and Optimization 128 Missing Keys: if Tests 129 Tuples 130 Why Tuples? 130 Files 130 Other File-Like Tools 131 Other Core Types 132 How to Break Your Code’s Flexibility 133 User-Defined Classes 133 And Everything Else 134 Chapter Summary 135 Quiz Answers 136 Numbers 138 Python Numeric Types 138 Numeric Literals 139 Built-in Numeric Tools and Extensions 140 Python Expression Operators 141 Mixed Operators Follow Operator Precedence 142 Parentheses Group Subexpressions 142 Mixed Types Are Converted Up 142 Preview: Operator Overloading 143 Numbers in Action 144 Variables and Basic Expressions 144 Numeric Display Formats 145 Division: Classic, Floor, and True 147 Bitwise Operations 148 Long Integers 148 Complex Numbers 149 Hexadecimal and Octal Notation 150 Other Built-in Numeric Tools 151 Other Numeric Types 152 Decimal Numbers 152 Sets 153 Booleans 154 Third-Party Extensions 155 Chapter Summary 155 Quiz Answers 156 The Dynamic Typing Interlude 157 The Case of the Missing Declaration Statements 157 Variables, Objects, and References 157 Types Live with Objects, Not Variables 159 Objects Are Garbage-Collected 160 Shared References 161 Shared References and In-Place Changes 163 Shared References and Equality 164 Dynamic Typing Is Everywhere 166 Chapter Summary 166 Quiz Answers 167 Strings 168 String Literals 169 Single- and Double-Quoted Strings Are the Same 170 Escape Sequences Represent Special Bytes 170 Raw Strings Suppress Escapes 172 Triple Quotes Code Multiline Block Strings 174 Unicode Strings Encode Larger Character Sets 175 Strings in Action 177 Basic Operations 177 Indexing and Slicing 178 Extended slicing: the third limit 180 String Conversion Tools 181 Character code conversions 183 Changing Strings 184 String Formatting 185 Advanced String Formatting 186 Dictionary-Based String Formatting 187 String Methods 188 String Method Examples: Changing Strings 189 String Method Examples: Parsing Text 191 Other Common String Methods in Action 192 The Original string Module 193 General Type Categories 194 Types Share Operation Sets by Categories 194 Mutable Types Can Be Changed In-Place 195 Chapter Summary 195 Quiz Answers 196 Lists and Dictionaries 197 Lists 197 Lists in Action 199 Basic List Operations 199 Indexing, Slicing, and Matrixes 200 Changing Lists In-Place 201 Index and slice assignments 201 List method calls 202 Other common list operations 204 Dictionaries 205 Dictionaries in Action 206 Basic Dictionary Operations 207 Changing Dictionaries In-Place 208 More Dictionary Methods 208 A Languages Table 210 Dictionary Usage Notes 211 Using dictionaries to simulate flexible lists 211 Using dictionaries for sparse data structures 212 Avoiding missing-key errors 212 Using dictionaries as “records” 213 Other ways to make dictionaries 214 Chapter Summary 215 Quiz Answers 216 Tuples, Files, and Everything Else 217 Tuples 217 Tuples in Action 218 Tuple syntax peculiarities: commas and parentheses 219 Conversions and immutability 219 Why Lists and Tuples? 220 Files 221 Opening Files 221 Using Files 222 Files in Action 223 Storing and parsing Python objects in files 223 Storing native Python objects with pickle 225 Storing and parsing packed binary data in files 226 Other File Tools 227 Type Categories Revisited 227 Object Flexibility 228 References Versus Copies 229 Comparisons, Equality, and Truth 231 The Meaning of True and False in Python 233 Python’s Type Hierarchies 234 Other Types in Python 236 Built-in Type Gotchas 236 Assignment Creates References, Not Copies 236 Repetition Adds One Level Deep 237 Beware of Cyclic Data Structures 238 Immutable Types Can’t Be Changed In-Place 238 Chapter Summary 238 Quiz Answers 240 Part III 244 Introducing Python Statements 246 Python Program Structure Revisited 246 Python’s Statements 247 A Tale of Two ifs 248 What Python Adds 249 What Python Removes 249 Parentheses are optional 249 End of line is end of statement 249 End of indentation is end of block 250 Why Indentation Syntax? 251 A Few Special Cases 253 Statement rule special cases 253 Block rule special case 254 A Quick Example: Interactive Loops 255 A Simple Interactive Loop 255 Doing Math on User Inputs 256 Handling Errors by Testing Inputs 257 Handling Errors with try Statements 258 Nesting Code Three Levels Deep 259 Chapter Summary 260 Quiz Answers 261 Assignment, Expressions, and print 262 Assignment Statements 262 Assignment Statement Forms 263 Sequence Assignments 264 Advanced sequence assignment patterns 265 Multiple-Target Assignments 267 Multiple-target assignment and shared references 267 Augmented Assignments 268 Augmented assignment and shared references 270 Variable Name Rules 270 Naming conventions 272 Names have no type, but objects do 272 Expression Statements 273 Expression Statements and In-Place Changes 274 print Statements 274 The Python “Hello World” Program 275 Redirecting the Output Stream 276 The print >> file Extension 277 Chapter Summary 279 Quiz Answers 280 if Tests 281 if Statements 281 General Format 281 Basic Examples 282 Multiway Branching 282 Python Syntax Rules 284 Block Delimiters 285 Statement Delimiters 286 A Few Special Cases 287 Truth Tests 288 The if/else Ternary Expression 289 Chapter Summary 291 Quiz Answers 292 while and for Loops 293 while Loops 293 General Format 294 Examples 294 break, continue, pass, and the Loop else 295 General Loop Format 295 Examples 296 pass 296 continue 296 break 297 else 297 More on the loop else clause 298 for Loops 299 General Format 299 Examples 301 Basic usage 301 Other data types 301 Tuple assignment in for 302 Nested for loops 302 Iterators: A First Look 303 File Iterators 305 Other Built-in Type Iterators 307 Other Iteration Contexts 308 User-Defined Iterators 309 Loop Coding Techniques 310 Counter Loops: while and range 310 Nonexhaustive Traversals: range 311 Changing Lists: range 312 Parallel Traversals: zip and map 313 Dictionary construction with zip 315 Generating Both Offsets and Items: enumerate 316 List Comprehensions: A First Look 317 List Comprehension Basics 317 Using List Comprehensions on Files 318 Extended List Comprehension Syntax 319 Chapter Summary 320 Quiz Answers 321 The Documentation Interlude 323 Python Documentation Sources 323 # Comments 324 The dir Function 324 Docstrings: __doc__ 325 User-defined docstrings 326 Docstring standards 327 Built-in docstrings 327 PyDoc: The help Function 328 PyDoc: HTML Reports 330 Standard Manual Set 334 Web Resources 334 Published Books 335 Common Coding Gotchas 336 Chapter Summary 338 Quiz Answers 339 Part IV 342 Function Basics 344 Why Use Functions? 345 Coding Functions 345 def Statements 347 def Executes at Runtime 348 A First Example: Definitions and Calls 348 Definition 349 Calls 349 Polymorphism in Python 350 A Second Example: Intersecting Sequences 351 Definition 351 Calls 351 Polymorphism Revisited 352 Local Variables 353 Chapter Summary 353 Quiz Answers 354 Scopes and Arguments 355 Scope Rules 355 Python Scope Basics 356 Name Resolution: The LEGB Rule 357 Scope Example 359 The Built-in Scope 359 The global Statement 361 Minimize Global Variables 362 Minimize Cross-File Changes 363 Other Ways to Access Globals 364 Scopes and Nested Functions 365 Nested Scope Details 365 Nested Scope Examples 366 Factory functions 366 Retaining enclosing scopes’ state with defaults 368 Nested scopes and lambdas 369 Scopes versus defaults with loop variables 369 Arbitrary scope nesting 371 Passing Arguments 371 Arguments and Shared References 372 Avoiding Mutable Argument Changes 374 Simulating Output Parameters 374 Special Argument-Matching Modes 375 Keyword and Default Examples 377 Keywords 377 Defaults 378 Arbitrary Arguments Examples 378 Collecting arguments 379 Unpacking arguments 379 Combining Keywords and Defaults 380 The min Wakeup Call 381 Full credit 381 Bonus points 382 The punch line 383 A More Useful Example: General Set Functions 383 Argument Matching: The Gritty Details 384 Chapter Summary 385 Quiz Answers 388 Advanced Function Topics 389 Anonymous Functions: lambda 389 lambda Expressions 389 Why Use lambda? 391 How (Not) to Obfuscate Your Python Code 392 Nested lambdas and Scopes 393 Applying Functions to Arguments 395 The apply Built-in 395 Passing keyword arguments 396 apply-Like Call Syntax 396 Mapping Functions over Sequences: map 397 Functional Programming Tools: filter and reduce 398 List Comprehensions Revisited: Mappings 400 List Comprehension Basics 400 Adding Tests and Nested Loops 401 List Comprehensions and Matrixes 403 Comprehending List Comprehensions 405 Iterators Revisited: Generators 405 Generator Function Example 407 Extended Generator Function Protocol: send Versus next 409 Iterators and Built-in Types 409 Generator Expressions: Iterators Meet List Comprehensions 410 Timing Iteration Alternatives 411 Function Design Concepts 414 Functions Are Objects: Indirect Calls 415 Function Gotchas 416 Local Names Are Detected Statically 417 Defaults and Mutable Objects 418 Functions Without returns 420 Enclosing Scope Loop Variables 420 Chapter Summary 420 Quiz Answers 422 Part V 428 Modules: The Big Picture 430 Why Use Modules? 430 Python Program Architecture 431 How to Structure a Program 432 Imports and Attributes 432 Standard Library Modules 434 How Imports Work 434 1. Find It 435 The module search path 435 The sys.path list 437 Module file selection 438 Advanced module selection concepts 438 2. Compile It (Maybe) 439 3. Run It 439 Chapter Summary 440 Quiz Answers 442 Module Coding Basics 443 Module Creation 443 Module Usage 444 The import Statement 444 The from statement 445 The from * Statement 445 Imports Happen Only Once 445 import and from Are Assignments 446 Cross-File Name Changes 447 import and from Equivalence 447 Potential Pitfalls of the from Statement 448 When import is required 449 Module Namespaces 449 Files Generate Namespaces 450 Attribute Name Qualification 451 Imports Versus Scopes 452 Namespace Nesting 453 Reloading Modules 454 reload Basics 455 reload Example 456 Chapter Summary 457 Quiz Answers 459 Module Packages 460 Package Import Basics 460 Packages and Search Path Settings 461 Package __init__.py Files 461 Package Import Example 463 from Versus import with Packages 464 Why Use Package Imports? 465 A Tale of Three Systems 466 Chapter Summary 469 Quiz Answers 470 Advanced Module Topics 471 Data Hiding in Modules 471 Minimizing from * Damage: _X and __all__ 471 Enabling Future Language Features 472 Mixed Usage Modes: __name__ and __main__ 473 Unit Tests with __name__ 474 Changing the Module Search Path 475 The import as Extension 476 Relative Import Syntax 476 Why Relative Imports? 477 Module Design Concepts 479 Modules Are Objects: Metaprograms 480 Module Gotchas 482 Statement Order Matters in Top-Level Code 482 Importing Modules by Name String 483 from Copies Names but Doesn’t Link 484 from * Can Obscure the Meaning of Variables 485 reload May Not Impact from Imports 485 reload, from, and Interactive Testing 486 reload Isn’t Applied Transitively 487 Recursive from Imports May Not Work 488 Chapter Summary 489 Quiz Answers 490 Part VI 494 OOP: The Big Picture 496 Why Use Classes? 497 OOP from 30,000 Feet 498 Attribute Inheritance Search 498 Classes and Instances 500 Class Method Calls 501 Coding Class Trees 501 OOP Is About Code Reuse 504 Chapter Summary 507 Quiz Answers 508 Class Coding Basics 510 Classes Generate Multiple Instance Objects 510 Class Objects Provide Default Behavior 511 Instance Objects Are Concrete Items 511 A First Example 512 Classes Are Customized by Inheritance 514 A Second Example 515 Classes Are Attributes in Modules 516 Classes Can Intercept Python Operators 517 A Third Example 519 Why Use Operator Overloading? 520 The World’s Simplest Python Class 521 Chapter Summary 523 Quiz Answers 524 Class Coding Details 526 The class Statement 526 General Form 526 Example 527 Methods 529 Example 530 Calling Superclass Constructors 531 Other Method Call Possibilities 531 Inheritance 531 Attribute Tree Construction 532 Specializing Inherited Methods 533 Class Interface Techniques 534 Abstract Superclasses 535 Operator Overloading 536 Common Operator Overloading Methods 537 __getitem__ Intercepts Index References 538 __getitem__ and __iter__ Implement Iteration 538 User-Defined Iterators 539 Multiple iterators on one object 541 __getattr__ and __setattr__ Catch Attribute References 543 Emulating Privacy for Instance Attributes 544 __repr__ and __str__ Return String Representations 545 __radd__ Handles Right-Side Addition 547 __call__ Intercepts Calls 547 Function Interfaces and Callback-Based Code 548 __del__ Is a Destructor 550 Namespaces: The Whole Story 551 Simple Names: Global Unless Assigned 551 Attribute Names: Object Namespaces 551 The “Zen” of Python Namespaces: Assignments Classify Names 551 Namespace Dictionaries 553 Namespace Links 556 A More Realistic Example 557 Chapter Summary 560 Quiz Answers 561 Designing with Classes 563 Python and OOP 563 Overloading by Call Signatures (or Not) 564 Classes As Records 564 OOP and Inheritance: “Is-a” Relationships 566 OOP and Composition: “Has-a” Relationships 568 Stream Processors Revisited 569 OOP and Delegation 572 Multiple Inheritance 574 Classes Are Objects: Generic Object Factories 577 Why Factories? 578 Methods Are Objects: Bound or Unbound 579 Documentation Strings Revisited 580 Classes Versus Modules 582 Chapter Summary 582 Quiz Answers 583 Advanced Class Topics 584 Extending Built-in Types 584 Extending Types by Embedding 585 Extending Types by Subclassing 585 Pseudoprivate Class Attributes 588 Name Mangling Overview 588 Why Use Pseudoprivate Attributes? 589 New-Style Classes 590 Diamond Inheritance Change 591 Diamond inheritance example 592 Explicit conflict resolution 592 Other New-Style Class Extensions 594 Static and class methods 594 Instance slots 594 Class properties 595 New __getattribute__ overloading method 597 Static and Class Methods 597 Using Static and Class Methods 599 Function Decorators 601 Decorator Example 603 Class Gotchas 604 Changing Class Attributes Can Have Side Effects 604 Multiple Inheritance: Order Matters 605 Methods, Classes, and Nested Scopes 606 “Overwrapping-itis” 608 Chapter Summary 609 Quiz Answers 610 Part VII 618 Exception Basics 620 Why Use Exceptions? 621 Exception Roles 621 Exception Handling: The Short Story 622 The try/except/else Statement 626 try Statement Clauses 627 The try/else Clause 630 Example: Default Behavior 630 Example: Catching Built-in Exceptions 631 The try/finally Statement 632 Example: Coding Termination Actions with try/finally 633 Unified try/except/finally 634 Combining finally and except by Nesting 635 Unified try Example 636 The raise Statement 637 Example: Raising and Catching User-Defined Exceptions 638 Example: Passing Extra Data with raise 638 Example: Propagating Exceptions with raise 639 The assert Statement 640 Example: Trapping Constraints (but Not Errors) 640 with/as Context Managers 641 Basic Usage 641 The Context Management Protocol 643 Chapter Summary 645 Quiz Answers 646 Exception Objects 647 String-Based Exceptions 648 String Exceptions Are Right Out! 648 Class-Based Exceptions 649 Class Exception Example 649 Why Class Exceptions? 651 Built-in Exception Classes 654 Specifying Exception Text 655 Sending Extra Data and Behavior in Instances 656 Example: Extra data with classes and strings 656 General raise Statement Forms 658 Chapter Summary 660 Quiz Answers 661 Designing with Exceptions 662 Nesting Exception Handlers 662 Example: Control-Flow Nesting 664 Example: Syntactic Nesting 664 Exception Idioms 666 Exceptions Aren’t Always Errors 666 Functions Signal Conditions with raise 667 Debugging with Outer try Statements 667 Running In-Process Tests 668 More on sys.exc_info 669 Exception Design Tips 669 What Should Be Wrapped 669 Catching Too Much: Avoid Empty excepts 670 Catching Too Little: Use Class-Based Categories 672 Exception Gotchas 672 String Exceptions Match by Identity, Not by Value 673 Catching the Wrong Thing 674 Core Language Summary 674 The Python Toolset 675 Development Tools for Larger Projects 676 Chapter Summary 679 Quiz Answers 680 Part VIII 682 Installation and Configuration 684 Installing the Python Interpreter 684 Is Python Already Present? 684 Where to Fetch Python 684 Installation Steps 685 Configuring Python 686 Python Environment Variables 687 How to Set Configuration Options 689 Unix/Linux shell variables 689 DOS variables (Windows) 689 Other Windows options 690 Path files 690 Solutions to End-of-Part Exercises 691 Part I, Getting Started 691 Part II, Types and Operations 693 Part III, Statements and Syntax 699 Part IV, Functions 701 Part V, Modules 706 Part VI, Classes and OOP 709 Part VII, Exceptions and Tools 717 Index 726