Python for Unix and Linux System Administration

"Python for Unix and Linux System Administration demonstrates how the Python language offers you a more efficient way to handle various tasks when managing Unix and Linux servers. Each chapter presents a particular administrative issue, such as concurrency or data backup, and offers Python solutions through hands-on examples. You'll learn to develop your own set of command-line utilities with Python to tackle a wide range of problems."--Jacket Python for Unix and Linux System Administration 1 Table of Contents 8 Foreword 12 Preface 14 Conventions Used in This Book 14 Using Code Examples 14 Safari® Books Online 15 How to Contact Us 15 Acknowledgments 16 Noah’s Acknowledgments 16 Jeremy’s Acknowledgments 19 Chapter 1. Introduction 22 Why Python? 22 Motivation 27 The Basics 29 Executing Statements in Python 29 Summary 33 Using Functions in Python 33 Reusing Code with the Import Statement 37 Chapter 2. IPython 42 Installing IPython 43 Basic Concepts 44 Interacting with IPython 45 Tab Completion 49 Magic Edit 50 Configuring IPython 51 Help with Magic Functions 51 Unix Shell 55 alias 55 Shell Execute 57 rehash 58 rehashx 59 cd 60 bookmark 62 dhist 63 pwd 64 Variable Expansion 64 String Processing 65 sh Profile 69 Information Gathering 72 page 72 pdef 72 pdoc 73 pfile 73 pinfo 74 psource 75 psearch 76 who 78 who_ls 79 whos 79 History 80 Readline support 80 hist command 81 History results 83 Automation and Shortcuts 85 alias 85 macro 85 store 86 reset 86 run 87 save 87 rep 87 Summary 90 Chapter 3. Text 92 Python Built-ins and Modules 92 str 92 Creating strings 93 Built-in methods for str data extraction 96 Unicode strings 106 re 109 Apache Config File Hacking 118 Working with Files 121 Creating files 121 Reading files 123 Writing files 125 Additional resources 126 Standard Input and Output 126 StringIO 129 urllib 130 Log Parsing 131 ElementTree 137 Summary 141 Chapter 4. Documentation and Reporting 144 Automated Information Gathering 144 Receiving Email 146 Manual Information Gathering 147 Information Formatting 156 Graphical Images 157 PDFs 159 Information Distribution 162 Sending Email 162 Sending basic messages 162 Using SMTP authentication 163 Sending attachments with Python 164 Trac 165 Summary 166 Chapter 5. Networking 168 Network Clients 168 socket 168 httplib 174 ftplib 176 urllib 178 urllib2 179 Remote Procedure Call Facilities 179 XML-RPC 179 Pyro 182 SSH 185 Twisted 188 Scapy 194 Creating Scripts with Scapy 196 Chapter 6. Data 198 Introduction 198 Using the OS Module to Interact with Data 199 Copying, Moving, Renaming, and Deleting Data 200 Working with Paths, Directories, and Files 202 Comparing Data 206 Using the filecmp Module 206 Using os.list 207 Merging Data 208 MD5 Checksum Comparisons 208 Pattern Matching Files and Directories 214 Wrapping Up rsync 216 Metadata: Data About Data 218 Archiving, Compressing, Imaging, and Restoring 220 Using tarfile Module to Create TAR Archives 220 Using a tarfile Module to Examine the Contents of TAR Files 222 Chapter 7. SNMP 226 Introduction 226 Brief Introduction to SNMP 226 SNMP Overview 226 SNMP Installation and Configuration 227 IPython and Net-SNMP 229 Discovering a Data Center 232 Retrieving Multiple-Values with Net-SNMP 235 Finding Memory 237 Creating Hybrid SNMP Tools 241 Extending Net-SNMP 243 SNMP Device Control 245 Enterprise SNMP Integration with Zenoss 246 Zenoss API 246 Using Zendmd 247 Device API 247 Chapter 8. OS Soup 248 Introduction 248 Cross-Platform Unix Programming in Python 249 Using SSH Keys, NFS-Mounted Source Directory, and Cross-Platform Python to Manage Systems 252 Creating a Cross-Platform, Systems Management Tool 254 Creating a Cross-Platform Build Network 258 PyInotify 259 OS X 261 Scripting DSCL or Directory Services Utility 261 OS X Scripting APIs 262 Automatically Re-Imaging Machines 263 Managing Plist Files from Python 266 Red Hat Linux Systems Administration 266 Ubuntu Administration 266 Solaris Systems Administration 266 Virtualization 267 VMware 267 Cloud Computing 268 Amazon Web Services with Boto 268 Google App Engine 269 Building a sample Google App Engine application 270 Using Zenoss to Manage Windows Servers from Linux 274 Chapter 9. Package Management 278 Introduction 278 Setuptools and Python Eggs 279 Using easy_install 279 easy_install Advanced Features 282 Search for Packages on a Web Page 282 Install Source Distribution from URL 282 Install Egg Located on Local or Network Filesystem 283 Upgrading Packages 283 Install an Unpacked Source Distribution in Current Working Directory 285 Extract Source Distribution to Specified Directory 285 Change Active Version of Package 285 Changing Standalone .py File into egg 285 Authenticating to a Password Protected Site 286 Using Configuration Files 286 Easy Install Advanced Features Summary 287 Creating Eggs 287 Entry Points and Console Scripts 291 Registering a Package with the Python Package Index 292 Where Can I Learn More About ... 294 Distutils 294 Buildout 296 Using Buildout 297 Developing with Buildout 300 virtualenv 300 Creating a Custom Bootstrapped Virtual Environment 302 EPM Package Manager 304 EPM Package Manager Requirements and Installation 305 Creating a Hello World Command-Line Tool to Distribute 305 Creating Platform-Specific Packages with EPM 306 Making the Package 308 EPM Summary: It Really Is That Easy 309 Chapter 10. Processes and Concurrency 310 Introduction 310 Subprocess 310 Using Return Codes with Subprocess 311 Using Supervisor to Manage Processes 319 Using Screen to Manage Processes 321 Threads in Python 322 Timed Delay of Threads with threading.Timer 331 Threaded Event Handler 332 Processes 334 Processing Module 334 Scheduling Python Processes 337 daemonizer 339 Summary 342 Chapter 11. Building GUIs 344 GUI Building Theory 344 Building a Simple PyGTK App 345 Building an Apache Log Viewer Using PyGTK 347 Building an Apache Log Viewer Using Curses 351 Web Applications 355 Django 356 Apache Log Viewer Application 356 Simple Database Application 363 Conclusion 376 Chapter 12. Data Persistence 378 Simple Serialization 378 Pickle 378 cPickle 383 shelve 384 YAML 387 ZODB 391 Relational Serialization 397 SQLite 397 Storm ORM 400 SQLAlchemy ORM 403 Summary 406 Chapter 13. Command Line 408 Introduction 408 Basic Standard Input Usage 409 Introduction to Optparse 410 Simple Optparse Usage Patterns 411 No Options Usage Pattern 411 True/False Usage Pattern 412 Counting Options Usage Pattern 414 Choices Usage Pattern 415 Option with Multiple Arguments Usage Pattern 417 Unix Mashups: Integrating Shell Commands into Python Command-Line Tools 418 Kudzu Usage Pattern: Wrapping a Tool in Python 418 Hybrid Kudzu Design Pattern: Wrapping a Tool in Python, and Then Changing the Behavior 420 Hybrid Kudzu Design Pattern: Wrapping a Unix Tool in Python to Spawn Processes 422 Integrating Configuration Files 423 Summary 425 Chapter 14. Pragmatic Examples 426 Managing DNS with Python 426 Using LDAP with OpenLDAP, Active Directory, and More with Python 427 Importing an LDIF File 429 Apache Log Reporting 429 FTP Mirror 436 Appendix. Callbacks 440 Index 444 Annotation Python is an ideal language for solving problems, especially in Linux and Unix networks. With this pragmatic book, administrators can review various tasks that often occur in the management of these systems, and learn how Python can provide a more efficient and less painful way to handle them. Each chapter in Python for Unix and Linux System Administration presents a particular administrative issue, such as concurrency or data backup, and presents Python solutions through hands-on examples. Once you finish this book, you'll be able to develop your own set of command-line utilities with Python to tackle a wide range of problems. Discover how this language can help you: Read text files and extract information Run tasks concurrently using the threading and forking options Get information from one process to another using network facilities Create clickable GUIs to handle large and complex utilities Monitor large clusters of machines by interacting with SNMP programmatically Master the IPython Interactive Python shell to replace or augment Bash, Korn, or Z-Shell Integrate Cloud Computing into your infrastructure, and learn to write a Google App Engine Application Solve unique data backup challenges with customized scripts Interact with MySQL, SQLite, Oracle, Postgres, Django ORM, and SQLAlchemy With this book, you'll learn how to package and deploy your Python applications and libraries, and write code that runs equally well on multiple Unix platforms. You'll also learn about several Python-related technologies that will make your life much easier. Python is an ideal language for solving problems, especially for Linux and Unix. With this pragmatic book, administrators can review various tasks that often occur in the management of these systems, and learn how Python can provide a more efficient way to handle them. Once you finish this book, you'll be able to develop your own set of command-line utilities with Python to tackle a wide range of problems.