Text processing in Python

Text Processing with Python, written by Python authority David Mertz, teaches programmers how to accomplish numerous text processing tasks using the Python language. Filled with concrete examples, this book provides effective solutions to specific text processing problems and practical strategies for dealing with all types of text processing challenges. It provides the answers to questions such as: What is the best way to convert from binary to ASCII?; How do I work with full text indexing?; How do I find a URL or an email address in text?; What are the different levels of pattern matching?; How do I process a report with a concrete state machine?; How do I parse, create and manipulate HTML documents?; How do I handle a lossless and lossy compression?; And what is the most efficient way to find codepoints in Unicode? Main Page......Page 1 Table of content......Page 3 Copyright......Page 5 Preface......Page 7 0.1 What Is Text Processing?......Page 8 0.2 The Philosophy of Text Processing......Page 9 0.3 What You'll Need to Use This Book......Page 10 0.4 Conventions Used in This Book......Page 11 0.5 A Word on Source Code Examples......Page 14 0.6 External Resources......Page 15 Acknowledgments......Page 17 Chapter 1. Python Basics......Page 21 1.1 Techniques and Patterns......Page 22 1.2 Standard Modules......Page 59 1.3 Other Modules in the Standard Library......Page 113 Chapter 2. Basic String Operations......Page 141 2.1 Some Common Tasks......Page 142 2.2 Standard Modules......Page 155 2.3 Solving Problems......Page 220 Chapter 3. Regular Expressions......Page 228 3.1 A Regular Expression Tutorial......Page 229 3.2 Some Common Tasks......Page 241 3.3 Standard Modules......Page 249 Chapter 4. Parsers and State Machines......Page 274 4.1 An Introduction to Parsers......Page 275 4.2 An Introduction to State Machines......Page 282 4.3 Parser Libraries for Python......Page 294 Chapter 5. Internet Tools and Techniques......Page 346 5.1 Working with Email and Newsgroups......Page 347 5.2 World Wide Web Applications......Page 380 5.3 Synopses of Other Internet Modules......Page 398 5.4 Understanding XML......Page 405 Appendix A. A Selective and Impressionistic Short Review of Python......Page 418 A.1 What Kind of Language Is Python?......Page 419 A.2 Namespaces and Bindings......Page 420 A.3 Datatypes......Page 424 A.4 Flow Control......Page 433 A.5 Functional Programming......Page 444 Appendix B. A Data Compression Primer......Page 448 B.1 Introduction......Page 449 B.2 Lossless and Lossy Compression......Page 450 B.3 A Data Set Example......Page 451 B.4 Whitespace Compression......Page 452 B.5 Run-Length Encoding......Page 453 B.6 Huffman Encoding......Page 454 B.7 Lempel Ziv-Compression......Page 455 B.8 Solving the Right Problem......Page 456 B.9 A Custom Text Compressor......Page 457 B.10 References......Page 461 Appendix C. Understanding Unicode......Page 462 C.1 Some Background on Characters......Page 463 C.2 What Is Unicode?......Page 464 C.3 Encodings......Page 465 C.4 Declarations......Page 466 C.5 Finding Codepoints......Page 467 C.6 Resources......Page 468 Appendix D. A State Machine for Adding Markup to Text......Page 469 Appendix E. Glossary......Page 475

Text Processing in Python is an example-driven, hands-on tutorial that carefully teaches programmers how to accomplish numerous text processing tasks using the Python language. Filled with concrete examples, this book provides efficient and effective solutions to specific text processing problems and practical strategies for dealing with all types of text processing challenges.

Text Processing in Python begins with an introduction to text processing and contains a quick Python tutorial to get you up to speed. It then delves into essential text processing subject areas, including string operations, regular expressions, parsers and state machines, and Internet tools and techniques. Appendixes cover such important topics as data compression and Unicode. A comprehensive index and plentiful cross-referencing offer easy access to available information. In addition, exercises throughout the book provide readers with further opportunity to hone their skills either on their own or in the classroom. A companion Web site (http://gnosis.cx/TPiP) contains source code and examples from the book.

Here is some of what you will find in thie book:

• When do I use formal parsers to process structured and semi-structured data? Page 257
• How do I work with full text indexing? Page 199
• What patterns in text can be expressed using regular expressions? Page 204
• How do I find a URL or an email address in text? Page 228
• How do I process a report with a concrete state machine? Page 274
• How do I parse, create, and manipulate internet formats? Page 345
• How do I handle lossless and lossy compression? Page 454
• How do I find codepoints in Unicode? Page 465

0321112547B05022003

Text Processing in Python describes techniques for manipulation of text using the Python programming language. At the broadest level, text processing is simply taking textual information and doing something with it. This might be restructuring or reformatting it, extracting smaller bits of information from it, or performing calculations that depend on the text. Text processing is arguably what most programmers spend most of their time doing. Because Python is clear, expressive, and object-oriented it is a perfect language for doing text processing, even better than Perl. As the amount of data everywhere continues to increase, this is more and more of a challenge for programmers. This book is not a tutorial on Python. It has two other helping the programmer get the job done pragmatically and efficiently; and giving the reader an understanding - both theoretically and conceptually - of why what works works and what doesn't work doesn't work. Mertz provides practical pointers and tips that emphasize efficent, flexible, and maintainable approaches to the textprocessing tasks that working programmers face daily.