Programming Python, 4th Edition

If you've mastered Python's fundamentals, you're ready to start using it to get real work done. Programming Python will show you how, with in-depth tutorials on the language's primary application domains: system administration, GUIs, and the Web. You'll also explore how Python is used in databases, networking, front-end scripting layers, text processing, and more. This book focuses on commonly used tools and libraries to give you a comprehensive understanding of Python’s many roles in practical, real-world programming. You'll learn language syntax and programming techniques in a clear and concise manner, with lots of examples that illustrate both correct usage and common idioms. Completely updated for version 3.x, Programming Python also delves into the language as a software development tool, with many code examples scaled specifically for that purpose. Topics include: * **Quick Python tour: Build a simple demo that includes data representation, object-oriented programming, object persistence, GUIs, and website basics** * **System programming: Explore system interface tools and techniques for command-line scripting, processing files and folders, running programs in parallel, and more** * **GUI programming: Learn to use Python’s tkinter widget library** * **Internet programming: Access client-side network protocols and email tools, use CGI scripts, and learn website implementation techniques** * **More ways to apply Python: Implement data structures, parse text-based information, interface with databases, and extend and embed Python** Table of Contents 7 Preface 25 “And Now for Something Completely Different...” 25 About This Book 26 This Book’s Ecosystem 26 What This Book Is Not 27 About This Fourth Edition 27 Specific Changes in This Edition 28 What’s Left, Then? 32 Python 3.X Impacts on This Book 33 Specific 3.X Changes 34 Language Versus Library: Unicode 35 Python 3.1 Limitations: Email, CGI 35 Using Book Examples 36 Where to Look for Examples and Updates 36 Example Portability 37 Demo Launchers 38 Code Reuse Policies 38 Contacting O’Reilly 38 Conventions Used in This Book 39 Acknowledgments 39 Part I. The Beginning 43 Chapter 1. A Sneak Preview 45 “Programming Python: The Short Story” 45 The Task 46 Step 1: Representing Records 46 Using Lists 46 Start-up pointers 47 A database list 48 Field labels 49 Using Dictionaries 51 Other ways to make dictionaries 51 Lists of dictionaries 52 Nested structures 53 Dictionaries of dictionaries 54 Step 2: Storing Records Persistently 56 Using Formatted Files 56 Test data script 56 File name conventions 57 Script start-up pointers 58 Data format script 58 Utility scripts 60 Using Pickle Files 61 Using Per-Record Pickle Files 64 Using Shelves 65 Step 3: Stepping Up to OOP 68 Using Classes 69 Adding Behavior 71 Adding Inheritance 71 Refactoring Code 73 Augmenting methods 73 Display format 73 Constructor customization 74 Alternative classes 74 Adding Persistence 76 Other Database Options 78 Step 4: Adding Console Interaction 79 A Console Shelve Interface 79 Step 5: Adding a GUI 82 GUI Basics 82 Using OOP for GUIs 84 Getting Input from a User 86 A GUI Shelve Interface 88 Coding the GUI 88 Using the GUI 90 Future directions 91 Step 6: Adding a Web Interface 94 CGI Basics 94 GUIs versus the Web 97 Running a Web Server 97 Using Query Strings and urllib 99 Formatting Reply Text 101 A Web-Based Shelve Interface 102 Coding the website 102 Directories, string formatting, and security 105 Using the website 106 Future directions 110 The End of the Demo 111 Part II. System Programming 113 Chapter 2. System Tools 115 “The os.path to Knowledge” 115 Why Python Here? 115 The Next Five Chapters 116 System Scripting Overview 117 Python System Modules 118 Module Documentation Sources 119 Paging Documentation Strings 120 A Custom Paging Script 121 String Method Basics 122 Other String Concepts in Python 3.X: Unicode and bytes 124 File Operation Basics 125 Using Programs in Two Ways 126 Python Library Manuals 127 Commercially Published References 128 Introducing the sys Module 128 Platforms and Versions 128 The Module Search Path 129 The Loaded Modules Table 130 Exception Details 131 Other sys Module Exports 132 Introducing the os Module 132 Tools in the os Module 132 Administrative Tools 133 Portability Constants 134 Common os.path Tools 134 Running Shell Commands from Scripts 136 What’s a shell command? 137 Running shell commands 137 Communicating with shell commands 138 The subprocess module alternative 139 Shell command limitations 141 Other os Module Exports 142 Chapter 3. Script Execution Context 145 “I’d Like to Have an Argument, Please” 145 Current Working Directory 146 CWD, Files, and Import Paths 146 CWD and Command Lines 148 Command-Line Arguments 148 Parsing Command-Line Arguments 149 Shell Environment Variables 151 Fetching Shell Variables 152 Changing Shell Variables 153 Shell Variable Fine Points: Parents, putenv, and getenv 154 Standard Streams 155 Redirecting Streams to Files and Programs 156 Chaining programs with pipes 158 Coding alternatives for adders and sorters 160 Redirected Streams and User Interaction 161 Redirecting Streams to Python Objects 165 The io.StringIO and io.BytesIO Utility Classes 168 Capturing the stderr Stream 169 Redirection Syntax in Print Calls 169 Other Redirection Options: os.popen and subprocess Revisited 170 Redirecting input or output with os.popen 171 Redirecting input and output with subprocess 172 Chapter 4. File and Directory Tools 177 “Erase Your Hard Drive in Five Easy Steps!” 177 File Tools 177 The File Object Model in Python 3.X 178 Using Built-in File Objects 179 Output files 179 Opening 180 Writing 180 Ensuring file closure: Exception handlers and context managers 181 Closing 181 Input files 183 Reading lines with file iterators 185 Other open options 186 Binary and Text Files 188 Unicode encodings for text files 189 End-of-line translations for text files 191 Parsing packed binary data with the struct module 193 Random access files 195 Lower-Level File Tools in the os Module 197 Using os.open files 198 os.open mode flags 199 Wrapping descriptors in file objects 200 Other os module file tools 201 File Scanners 202 File filters 204 Directory Tools 205 Walking One Directory 206 Running shell listing commands with os.popen 206 The glob module 208 The os.listdir call 209 Splitting and joining listing results 210 Walking Directory Trees 210 The os.walk visitor 211 Recursive os.listdir traversals 213 Handling Unicode Filenames in 3.X: listdir, walk, glob 214 Unicode policies: File content versus file names 216 Chapter 5. Parallel System Tools 219 “Telling the Monkeys What to Do” 219 Forking Processes 221 The fork/exec Combination 224 os.exec call formats 225 Spawned child program 226 Threads 228 The _thread Module 231 Basic usage 231 Other ways to code threads with _thread 233 Running multiple threads 233 Synchronizing access to shared objects and names 235 Waiting for spawned thread exits 237 Coding alternatives: busy loops, arguments, and context managers 239 The threading Module 241 Other ways to code threads with threading 243 Synchronizing access to shared objects and names revisited 244 The queue Module 246 Arguments versus globals 248 Program exit with child threads 248 Running the script 249 Preview: GUIs and Threads 250 More on the Global Interpreter Lock 253 The thread switch interval 254 Atomic operations 254 C API thread considerations 254 A process-based alternative: multiprocessing (ahead) 255 Program Exits 255 sys Module Exits 256 os Module Exits 257 Shell Command Exit Status Codes 258 Exit status with os.system and os.popen 258 Output stream buffering: A first look 260 Exit status with subprocess 260 Process Exit Status and Shared State 261 Thread Exits and Shared State 262 Interprocess Communication 264 Anonymous Pipes 266 Anonymous pipe basics 266 Wrapping pipe descriptors in file objects 268 Anonymous pipes and threads 269 Bidirectional IPC with anonymous pipes 270 Output stream buffering revisited: Deadlocks and flushes 273 Named Pipes (Fifos) 276 Named pipe basics 277 Named pipe use cases 278 Sockets: A First Look 278 Socket basics 279 Sockets and independent programs 280 Socket use cases 281 Signals 282 The multiprocessing Module 285 Why multiprocessing? 285 The Basics: Processes and Locks 287 Implementation and usage rules 288 IPC Tools: Pipes, Shared Memory, and Queues 290 multiprocessing pipes 291 Shared memory and globals 292 Queues and subclassing 294 Starting Independent Programs 296 And Much More 298 And a little less... 298 Why multiprocessing? The Conclusion 299 Other Ways to Start Programs 300 The os.spawn Calls 300 The os.startfile call on Windows 303 Using the DOS start command 303 Using start in Python scripts 304 The os.startfile call 305 A Portable Program-Launch Framework 305 Other System Tools Coverage 310 Chapter 6. Complete System Programs 313 “The Greps of Wrath” 313 A Quick Game of “Find the Biggest Python File” 314 Scanning the Standard Library Directory 314 Scanning the Standard Library Tree 315 Scanning the Module Search Path 316 Scanning the Entire Machine 318 Printing Unicode Filenames 321 Splitting and Joining Files 324 Splitting Files Portably 325 Joining Files Portably 328 Usage Variations 331 Generating Redirection Web Pages 334 Page Template File 335 Page Generator Script 336 A Regression Test Script 339 Running the Test Driver 341 Copying Directory Trees 346 Comparing Directory Trees 350 Finding Directory Differences 351 Finding Tree Differences 353 Running the Script 356 Verifying Backups 358 Reporting Differences and Other Ideas 359 Searching Directory Trees 361 Greps and Globs and Finds 362 Rolling Your Own find Module 363 The fnmatch module 365 Cleaning Up Bytecode Files 366 A Python Tree Searcher 369 Visitor: Walking Directories “++” 372 Editing Files in Directory Trees (Visitor) 376 Global Replacements in Directory Trees (Visitor) 378 Counting Source Code Lines (Visitor) 380 Recoding Copies with Classes (Visitor) 381 Other Visitor Examples (External) 383 Playing Media Files 385 The Python webbrowser Module 389 The Python mimetypes Module 390 Using mimetypes guesses for SearchVisitor 391 Running the Script 392 Automated Program Launchers (External) 393 Part III. GUI Programming 395 Chapter 7. Graphical User Interfaces 397 “Here’s Looking at You, Kid” 397 GUI Programming Topics 397 Running the Examples 399 Python GUI Development Options 400 tkinter Overview 405 tkinter Pragmatics 405 tkinter Documentation 406 tkinter Extensions 406 tkinter Structure 408 Implementation structure 408 Programming structure 409 Climbing the GUI Learning Curve 410 “Hello World” in Four Lines (or Less) 410 tkinter Coding Basics 411 Making Widgets 412 Geometry Managers 412 Running GUI Programs 413 Avoiding DOS consoles on Windows 413 tkinter Coding Alternatives 414 Widget Resizing Basics 415 Configuring Widget Options and Window Titles 417 One More for Old Times’ Sake 418 Packing Widgets Without Saving Them 419 Adding Buttons and Callbacks 421 Widget Resizing Revisited: Expansion 422 Adding User-Defined Callback Handlers 424 Lambda Callback Handlers 425 Deferring Calls with Lambdas and Object References 426 Callback Scope Issues 427 Arguments versus globals 427 Passing in enclosing scope values with default arguments 428 Passing in enclosing scope values with automatic references 429 But you must still sometimes use defaults instead of enclosing scopes 429 Bound Method Callback Handlers 433 Callable Class Object Callback Handlers 434 Other tkinter Callback Protocols 435 Binding Events 436 Adding Multiple Widgets 437 Widget Resizing Revisited: Clipping 438 Attaching Widgets to Frames 439 Layout: Packing Order and Side Attachments 439 The Packer’s Expand and Fill Revisited 440 Using Anchor to Position Instead of Stretch 441 Customizing Widgets with Classes 442 Standardizing Behavior and Appearance 443 Common behavior 444 Common appearance 444 Reusable GUI Components with Classes 445 Attaching Class Components 447 Extending Class Components 449 Standalone Container Classes 450 The End of the Tutorial 452 Python/tkinter for Tcl/Tk Converts 454 Chapter 8. A tkinter Tour, Part 1 457 “Widgets and Gadgets and GUIs, Oh My!” 457 This Chapter’s Topics 457 Configuring Widget Appearance 458 Top-Level Windows 461 Toplevel and Tk Widgets 463 Top-Level Window Protocols 464 Dialogs 468 Standard (Common) Dialogs 468 A “smart” and reusable Quit button 471 A dialog demo launcher bar 472 Printing dialog results and passing callback data with lambdas 476 Letting users select colors on the fly 479 Other standard dialog calls 480 The Old-Style Dialog Module 480 Custom Dialogs 481 Making custom dialogs modal 482 Other ways to be modal 484 Binding Events 485 Other bind Events 489 More on events and the quit and destroy methods 489 Message and Entry 490 Message 490 Entry 491 Programming Entry widgets 492 Laying Out Input Forms 493 Going modal again 494 tkinter “Variables” and Form Layout Alternatives 496 Checkbutton, Radiobutton, and Scale 499 Checkbuttons 499 Check buttons and variables 502 Radio Buttons 504 Radio buttons and variables 505 Radio buttons without variables 507 Hold onto your variables! 508 Scales (Sliders) 509 Scales and variables 511 Running GUI Code Three Ways 513 Attaching Frames 513 Importing by name string 515 Configuring at construction time 516 Independent Windows 518 Running Programs 520 Launching GUIs as programs other ways: multiprocessing 521 Cross-program communication 522 Coding for reusability 524 Images 526 Fun with Buttons and Pictures 529 Viewing and Processing Images with PIL 533 PIL Basics 533 Displaying Other Image Types with PIL 535 Displaying all images in a directory 537 Creating Image Thumbnails with PIL 538 Performance: Saving thumbnail files 542 Layout options: Gridding 543 Layout options: Fixed-size buttons 544 Scrolling and canvases (ahead) 546 Chapter 9. A tkinter Tour, Part 2 549 “On Today’s Menu: Spam, Spam, and Spam” 549 Menus 549 Top-Level Window Menus 550 Frame- and Menubutton-Based Menus 554 Using Menubuttons and Optionmenus 557 Windows with Both Menus and Toolbars 559 Using images in toolbars, too 562 Automating menu construction 563 Listboxes and Scrollbars 564 Programming Listboxes 566 Programming Scroll Bars 567 Packing Scroll Bars 568 Text 570 Programming the Text Widget 572 Text is a Python string 572 String positions 573 Text indexes 573 Text marks 574 Text tags 574 Adding Text-Editing Operations 575 Using the clipboard 578 Composition versus inheritance 579 It’s called “Simple” for a reason: PyEdit (ahead) 580 Unicode and the Text Widget 580 String types in the Text widget 581 Unicode text in strings 582 Unicode text in files 585 Unicode and the Text widget 585 The problem with treating text as bytes 587 Other binary mode considerations 588 Supporting Unicode in PyEdit (ahead) 589 Advanced Text and Tag Operations 590 Canvas 592 Basic Canvas Operations 592 Programming the Canvas Widget 593 Coordinates 593 Object construction 594 Object identifiers and operations 595 Canvas object tags 595 Scrolling Canvases 596 Scrollable Canvases and Image Thumbnails 599 Scrolling images too: PyPhoto (ahead) 602 Using Canvas Events 602 Binding events on specific items 605 Grids 606 Why Grids? 606 Grid Basics: Input Forms Revisited 607 Comparing grid and pack 608 Combining grid and pack 610 Making Gridded Widgets Expandable 612 Resizing in grids 614 Spanning columns and rows 615 Laying Out Larger Tables with grid 616 Time Tools, Threads, and Animation 624 Using Threads with tkinter GUIs 626 Using the after Method 627 Hiding and redrawing widgets and windows 629 Simple Animation Techniques 630 Using time.sleep loops 631 Using widget.after events 633 Using multiple time.sleep loop threads 634 Other Animation Topics 635 Other animation effects 636 Threads and animation 636 Graphics and gaming toolkits 636 The End of the Tour 637 Other Widgets and Options 637 Chapter 10. GUI Coding Techniques 639 “Building a Better Mousetrap” 639 GuiMixin: Common Tool Mixin Classes 640 Widget Builder Functions 640 Mixin Utility Classes 641 GuiMaker: Automating Menus and Toolbars 645 Subclass Protocols 649 GuiMaker Classes 650 GuiMaker Self-Test 650 BigGui: A Client Demo Program 651 ShellGui: GUIs for Command-Line Tools 655 A Generic Shell-Tools Display 655 Application-Specific Tool Set Classes 657 Adding GUI Frontends to Command Lines 659 Non-GUI scripts 659 GUI input dialogs 661 Room for improvement 664 GuiStreams: Redirecting Streams to Widgets 665 Using Redirection for the Packing Scripts 669 Reloading Callback Handlers Dynamically 670 Wrapping Up Top-Level Window Interfaces 672 GUIs, Threads, and Queues 677 Placing Data on Queues 678 Recoding with classes and bound methods 680 Thread exits in GUIs 681 Placing Callbacks on Queues 682 Passing bound method callbacks on queues 686 More Ways to Add GUIs to Non-GUI Code 688 Popping Up GUI Windows on Demand 689 Adding a GUI As a Separate Program: Sockets (A Second Look) 691 Adding a GUI As a Separate Program: Command Pipes 696 The specter of blocking input calls 698 Updating GUIs within threads...and other nonsolutions 699 Avoiding blocking input calls with non-GUI threads 700 Sockets and pipes: Compare and contrast 701 Other uses for threaded pipe GUIs 702 The PyDemos and PyGadgets Launchers 704 PyDemos Launcher Bar (Mostly External) 704 PyGadgets Launcher Bar 709 Chapter 11. Complete GUI Programs 713 “Python, Open Source, and Camaros” 713 Examples in Other Chapters 714 This Chapter’s Strategy 715 PyEdit: A Text Editor Program/Object 716 Running PyEdit 717 Menus and toolbars 718 Dialogs 719 Running program code 719 Multiple windows 720 Other PyEdit examples and screenshots in this book 724 PyEdit Changes in Version 2.0 (Third Edition) 724 Font dialog 725 Undo, redo, and modified tests 725 Configuration module 725 PyEdit Changes in Version 2.1 (Fourth Edition) 726 Modal dialog state fix 726 Cross-process change tests on Quit 727 New Grep dialog: Threaded and Unicode-aware file tree search 727 Grep threading model 727 Grep Unicode model 728 Update for initial positioning 729 Improvements for running code 729 Unicode (Internationalized) text support 730 Unicode file and display model 730 Unicode options and choices 732 More on Quit checks: The event revisited 734 PyEdit Source Code 735 User configurations file 736 Windows (and other) launch files 737 Main implementation file 738 PyPhoto: An Image Viewer and Resizer 758 Running PyPhoto 759 PyPhoto Source Code 761 PyView: An Image and Notes Slideshow 769 Running PyView 769 Embedding PyEdit in PyView 771 PyView Source Code 774 PyDraw: Painting and Moving Graphics 780 Running PyDraw 780 PyDraw Source Code 780 PyClock: An Analog/Digital Clock Widget 789 A Quick Geometry Lesson 789 Running PyClock 793 PyClock Source Code 796 PyToe: A Tic-Tac-Toe Game Widget 804 Running PyToe 804 PyToe Source Code (External) 805 Where to Go from Here 808 Part IV. Internet Programming 811 Chapter 12. Network Scripting 813 “Tune In, Log On, and Drop Out” 813 Internet Scripting Topics 814 What we will cover 815 What we won’t cover 816 Other themes in this part of the book 816 Running Examples in This Part of the Book 817 Python Internet Development Options 819 Plumbing the Internet 822 The Socket Layer 823 Machine identifiers 823 The Protocol Layer 824 Port number rules 825 Clients and servers 825 Protocol structures 826 Python’s Internet Library Modules 827 Socket Programming 829 Socket Basics 830 Server socket calls 832 Transferring byte strings and objects 833 Client socket calls 834 Running Socket Programs Locally 836 Running Socket Programs Remotely 837 Socket pragmatics 838 Spawning Clients in Parallel 840 Preview: Denied client connections 841 Talking to Reserved Ports 843 Binding reserved port servers 844 Handling Multiple Clients 844 Forking Servers 845 Running the forking server 846 Other run modes: Local servers with Cygwin and remote clients 848 Forked processes and sockets 849 Exiting from children 849 Killing the zombies: Don’t fear the reaper! 849 Preventing zombies with signal handlers on Linux 851 Why multiprocessing doesn’t help with socket server portability 855 Threading Servers 857 Standard Library Server Classes 860 Multiplexing Servers with select 862 A select-based echo server 863 Running the select server 865 Summary: Choosing a Server Scheme 868 Making Sockets Look Like Files and Streams 869 A Stream Redirection Utility 870 Text-mode files and buffered output streams 874 Stream requirements 876 Line buffering 877 Solutions 879 Buffering in other contexts: Command pipes revisited 880 Sockets versus command pipes 881 A Simple Python File Server 882 Running the File Server and Clients 884 Adding a User-Interface Frontend 885 Using row frames and command lines 885 Using grids and function calls 887 Using a reusable form-layout class 888 Chapter 13. Client-Side Scripting 895 “Socket to Me!” 895 FTP: Transferring Files over the Net 896 Transferring Files with ftplib 896 Using urllib to Download Files 899 FTP get and put Utilities 902 Download utility 903 Upload utility 906 Playing the Monty Python theme song 907 Adding a User Interface 909 Transferring Directories with ftplib 916 Downloading Site Directories 916 Uploading Site Directories 922 Refactoring Uploads and Downloads for Reuse 926 Refactoring with functions 926 Refactoring with classes 930 Transferring Directory Trees with ftplib 934 Uploading Local Trees 935 Deleting Remote Trees 937 Downloading Remote Trees 941 Processing Internet Email 941 Unicode in Python 3.X and Email Tools 942 POP: Fetching Email 943 Mail Configuration Module 944 POP Mail Reader Script 947 Fetching Messages 948 Fetching Email at the Interactive Prompt 951 SMTP: Sending Email 952 SMTP Mail Sender Script 953 Sending Messages 955 Verifying receipt 957 Manipulating both From and To 958 Sending Email at the Interactive Prompt 961 email: Parsing and Composing Mail Content 963 Message Objects 964 Basic email Package Interfaces in Action 966 Handling multipart messages 967 Unicode, Internationalization, and the Python 3.1 email Package 968 Parser decoding requirement 969 Text payload encodings: Handling mixed type results 971 Text payload encodings: Using header information to decode 973 Message header encodings: email package support 975 Message address header encodings and parsing, and header creation 977 Workaround: Message text generation for binary attachment payloads is broken 980 Workaround: Message composition for non-ASCII text parts is broken 983 Summary: Solutions and workarounds 988 A Console-Based Email Client 989 Running the pymail Console Client 994 The mailtools Utility Package 998 Initialization File 999 MailTool Class 1000 MailSender Class 1001 Unicode issues for attachments, save files, and headers 1001 MailFetcher Class 1009 General usage 1009 Unicode decoding for full mail text on fetches 1009 Inbox synchronization tools 1010 MailParser Class 1018 Unicode decoding for text part payloads and message headers 1018 Self-Test Script 1025 Running the self-test 1026 Updating the pymail Console Client 1028 Running the pymail2 console client 1031 NNTP: Accessing Newsgroups 1033 HTTP: Accessing Websites 1036 The urllib Package Revisited 1039 Other urllib Interfaces 1041 Invoking programs and escaping text 1042 Other Client-Side Scripting Options 1044 Chapter 14. The PyMailGUI Client 1047 “Use the Source, Luke” 1047 Source Code Modules and Size 1048 Code size 1049 Code Structure 1050 Why PyMailGUI? 1050 Running PyMailGUI 1052 Presentation Strategy 1052 Major PyMailGUI Changes 1053 New in Version 2.1 and 2.0 (Third Edition) 1053 New in Version 3.0 (Fourth Edition) 1054 Version 3.0 Unicode support policies 1059 A PyMailGUI Demo 1061 Getting Started 1062 Loading Mail 1067 Threading Model 1069 Threading model implementation 1071 Load Server Interface 1072 Offline Processing with Save and Open 1073 Sending Email and Attachments 1075 Viewing Email and Attachments 1079 Email Replies and Forwards and Recipient Options 1085 Deleting Email 1091 POP Message Numbers and Synchronization 1093 Handling HTML Content in Email 1095 Mail Content Internationalization Support 1097 Alternative Configurations and Accounts 1101 Multiple Windows and Status Messages 1102 PyMailGUI Implementation 1104 PyMailGUI: The Main Module 1105 SharedNames: Program-Wide Globals 1108 ListWindows: Message List Windows 1109 ViewWindows: Message View Windows 1127 messagecache: Message Cache Manager 1137 popuputil: General-Purpose GUI Pop Ups 1140 wraplines: Line Split Tools 1142 html2text: Extracting Text from HTML (Prototype, Preview) 1144 mailconfig: User Configurations 1147 textConfig: Customizing Pop-Up PyEdit Windows 1152 PyMailGUIHelp: User Help Text and Display 1153 altconfigs: Configuring for Multiple Accounts 1156 Ideas for Improvement 1158 Chapter 15. Server-Side Scripting 1167 “Oh, What a Tangled Web We Weave” 1167 What’s a Server-Side CGI Script? 1168 The Script Behind the Curtain 1168 Writing CGI Scripts in Python 1170 Running Server-Side Examples 1172 Web Server Options 1172 Running a Local Web Server 1173 The Server-Side Examples Root Page 1175 Viewing Server-Side Examples and Output 1176 Climbing the CGI Learning Curve 1177 A First Web Page 1177 HTML basics 1178 Internet addresses (URLs) 1179 Using minimal URLs 1181 HTML file permission constraints 1182 A First CGI Script 1183 Installing CGI scripts 1184 Finding Python on remote servers 1187 Adding Pictures and Generating Tables 1188 Table tags 1190 Adding User Interaction 1191 Submission page 1191 More on form tags 1192 Response script 1194 Passing parameters in URLs 1195 Testing outside browsers with the module urllib.request 1197 Using Tables to Lay Out Forms 1199 Converting strings in CGI scripts 1203 Debugging CGI scripts 1203 Adding Common Input Devices 1205 Changing Input Layouts 1208 Keeping display and logic separate 1211 Passing Parameters in Hardcoded URLs 1212 Passing Parameters in Hidden Form Fields 1214 Saving State Information in CGI Scripts 1216 URL Query Parameters 1218 Hidden Form Input Fields 1218 HTTP “Cookies” 1219 Creating a cookie 1220 Receiving a cookie 1221 Using cookies in CGI scripts 1221 Handling cookies with the urllib.request module 1222 Server-Side Databases 1223 Extensions to the CGI Model 1224 Combining Techniques 1225 The Hello World Selector 1225 Checking for Missing and Invalid Inputs 1232 Refactoring Code for Maintainability 1234 Step 1: Sharing Objects Between Pages—A New Input Form 1235 Step 2: A Reusable Form Mock-Up Utility 1238 Step 3: Putting It All Together—A New Reply Script 1241 More on HTML and URL Escapes 1243 URL Escape Code Conventions 1244 Python HTML and URL Escape Tools 1245 Escaping HTML Code 1245 Escaping URLs 1246 Escaping URLs Embedded in HTML Code 1247 HTML and URL conflicts: & 1248 Avoiding conflicts 1249 Transferring Files to Clients and Servers 1251 Displaying Arbitrary Server Files on the Client 1253 Handling private files and errors 1257 Uploading Client Files to the Server 1260 Handling client path formats 1266 More Than One Way to Push Bits over the Net 1269 Chapter 16. The PyMailCGI Server 1271 “Things to Do When Visiting Chicago” 1271 The PyMailCGI Website 1272 Implementation Overview 1272 New in This Fourth Edition (Version 3.0) 1275 New in the Prior Edition (Version 2.0) 1277 Presentation Overview 1278 Running This Chapter’s Examples 1279 The Root Page 1281 Configuring PyMailCGI 1282 Sending Mail by SMTP 1283 The Message Composition Page 1284 The Send Mail Script 1284 Error Pages 1288 Common Look-and-Feel 1288 Using the Send Mail Script Outside a Browser 1289 Reading POP Email 1291 The POP Password Page 1292 The Mail Selection List Page 1293 Passing State Information in URL Link Parameters 1296 Security Protocols 1299 Reading mail with direct URLs 1300 The Message View Page 1301 Passing State Information in HTML Hidden Input Fields 1304 Escaping Mail Text and Passwords in HTML 1306 Processing Fetched Mail 1308 Reply and Forward 1309 Delete 1310 Deletions and POP Message Numbers 1314 Inbox synchronization error potential 1314 Alternative: Passing header text in hidden input fields (PyMailCGI_2.1) 1315 Alternative: Server-side files for headers 1317 Alternative: Delete on load 1317 Utility Modules 1318 External Components and Configuration 1318 POP Mail Interface 1319 POP Password Encryption 1320 Manual data encryption: rotor (defunct) 1321 Manual data encryption: PyCrypto 1322 HTTPS: Secure HTTP transmissions 1323 Secure cookies 1324 The secret.py module 1324 Rolling your own encryptor 1328 Common Utilities Module 1328 Web Scripting Trade-Offs 1333 PyMailCGI Versus PyMailGUI 1334 The Web Versus the Desktop 1335 Other Approaches 1338 Part V. Tools and Techniques 1343 Chapter 17. Databases and Persistence 1345 “Give Me an Order of Persistence, but Hold the Pickles” 1345 Persistence Options in Python 1345 DBM Files 1347 Using DBM Files 1347 DBM Details: Files, Portability, and Close 1350 Pickled Objects 1351 Using Object Pickling 1352 Pickling in Action 1353 Pickle Details: Protocols, Binary Modes, and _pickle 1356 Shelve Files 1357 Using Shelves 1358 Storing Built-in Object Types in Shelves 1359 Storing Class Instances in Shelves 1360 Changing Classes of Objects Stored in Shelves 1362 Shelve Constraints 1363 Keys must be strings (and str) 1363 Objects are unique only within a key 1364 Updates must treat shelves as fetch-modify-store mappings 1364 Concurrent updates are not directly supported 1364 Underlying DBM format portability 1365 Pickled Class Constraints 1365 Other Shelve Limitations 1366 The ZODB Object-Oriented Database 1367 The Mostly Missing ZODB Tutorial 1368 SQL Database Interfaces 1371 SQL Interface Overview 1372 An SQL Database API Tutorial with SQLite 1374 Getting started 1375 Making databases and tables 1376 Adding records 1376 Running queries 1378 Running updates 1380 Building Record Dictionaries 1381 Using table descriptions 1381 Record dictionaries construction 1382 Automating with scripts and modules 1383 Tying the Pieces Together 1384 Loading Database Tables from Files 1386 Loading with SQL and Python 1386 Python versus SQL 1387 SQL Utility Scripts 1389 Table load scripts 1389 Table display script 1391 Using the scripts 1393 SQL Resources 1396 ORMs: Object Relational Mappers 1396 PyForm: A Persistent Object Viewer (External) 1398 Chapter 18. Data Structures 1401 “Roses Are Red, Violets Are Blue; Lists Are Mutable, and So Is Set Foo” 1401 Implementing Stacks 1402 Built-in Options 1402 A Stack Module 1404 A Stack Class 1406 Customization: Performance Monitors 1408 Optimization: Tuple Tree Stacks 1409 Optimization: In-Place List Modifications 1411 Timing the Improvements 1413 Results under Python 3.1 1414 More on performance analysis 1415 Implementing Sets 1415 Built-in Options 1416 Set Functions 1417 Supporting multiple operands 1418 Set Classes 1419 Optimization: Moving Sets to Dictionaries 1420 Timing the results under Python 3.1 1423 Adding Relational Algebra to Sets (External) 1424 Subclassing Built-in Types 1425 Binary Search Trees 1427 Built-in Options 1427 Implementing Binary Trees 1428 Trees with Both Keys and Values 1430 Graph Searching 1432 Implementing Graph Search 1432 Moving Graphs to Classes 1435 Permuting Sequences 1437 Reversing and Sorting Sequences 1439 Implementing Reversals 1440 Implementing Sorts 1441 Adding comparison functions 1442 Data Structures Versus Built-ins: The Conclusion 1442 PyTree: A Ge If you've mastered Python's fundamentals, you're ready to start using it to get real work done. Programming Python will show you how, with in-depth tutorials on the language's primary application domains: system administration, GUIs, and the Web. You'll also explore how Python is used in databases, networking, front-end scripting layers, text processing, and more. This book focuses on commonly used tools and libraries to give you a comprehensive understanding of Python’s many roles in practical, real-world programming.You'll learn language syntax and programming techniques in a clear and concise manner, with lots of examples that illustrate both correct usage and common idioms. Completely updated for version 3.x, Programming Python also delves into the language as a software development tool, with many code examples scaled specifically for that purpose.Topics include:Quick Python tour: Build a simple demo that includes data representation, object-oriented programming, object persistence, GUIs, and website basicsSystem programming: Explore system interface tools and techniques for command-line scripting, processing files and folders, running programs in parallel, and moreGUI programming: Learn to use Python’s tkinter widget libraryInternet programming: Access client-side network protocols and email tools, use CGI scripts, and learn website implementation techniquesMore ways to apply Python: Implement data structures, parse text-based information, interface with databases, and extend and embed Python