Perl and LWP Sourcecode

Perl soared to popularity as a language for creating and managing web content, but with LWP (Library for WWW in Perl), Perl is equally adept at consuming information on the Web. LWP is a suite of modules for fetching and processing web pages.The Web is a vast data source that contains everything from stock prices to movie credits, and with LWP all that data is just a few lines of code away. Anything you do on the Web, whether it's buying or selling, reading or writing, uploading or downloading, news to e-commerce, can be controlled with Perl and LWP. You can automate Web-based purchase orders as easily as you can set up a program to download MP3 files from a web site.Perl & LWP covers:Understanding LWP and its designFetching and analyzing URLsExtracting information from HTML using regular expressions and tokensWorking with the structure of HTML documents using treesSetting and inspecting HTTP headers and response codesManaging cookiesAccessing information that requires authenticationExtracting linksCooperating with proxy cachesWriting web spiders (also known as robots) in a safe fashionPerl & LWP includes many step-by-step examples that show how to apply the various techniques. Programs to extract information from the web sites of BBC News, Altavista, ABEBooks.com, and the Weather Underground, to name just a few, are explained in detail, so that you understand how and why they work.Perl programmers who want to automate and mine the web can pick up this book and be immediately productive. Written by a contributor to LWP, and with a foreword by one of LWP's creators, Perl & LWP is the authoritative guide to this powerful and popular toolkit. Perl soared to popularity as a language for creating and managing web content, but with LWP (Library for WWW in Perl), Perl is equally adept at consuming information on the Web. LWP is a suite of modules for fetching and processing web pages. The Web is a vast data source that contains everything from stock prices to movie credits, and with LWP all that data is just a few lines of code away. Anything you do on the Web, whether it's buying or selling, reading or writing, uploading or downloading, for e-commerce or recreation, can be controlled with Perl and LWP. You can automate web-based purchase orders as easily as you can set up a program to download MP3 files from a web site. Chapter 1: Introduction to Web Automation Chapter 2: Web Basics Chapter 3: The LWP Class Model Chapter 4: URLs Chapter 5: Forms Chapter 6: Simple HTML Processing with Regular Expressions Chapter 7: HTML Processing with Tokens Chapter 8: Tokenizing Walkthrough Chapter 9: HTML Processing with Trees Chapter 10: Modifying HTML with Trees Chapter 11: Cookies, Authentication, and Advanced Requests Chapter 12: Spiders Appendix A: LWP Modules Appendix B: HTTP Status Codes Appendix C: Common MIME Types Appendix D: Language Tags Appendix E: Common Content Encodings Appendix F: ASCII Table Appendix G: User's View of Object-Oriented Modules "Perl & LWP includes many step-by-step examples that show how to apply the various techniques. Programs to extract information from the web sites of BBC News, AltaVista, ABEBooks.com, and the Weather Underground, to name a few, are explained in detail, so that you understand how and why they work." "Perl programmers who want to automate and mine the Web can pick up this book and be immediately productive. Written by a contributor to LWP, with a foreword by one of LWP's creators, Perl & LWP is the authoritative guide to this powerful and popular toolkit."--Jacket