Unix Power Tools, Third Edition

With the growing popularity of Linux and the advent of Darwin, Unix has metamorphosed into something new and exciting. No longer perceived as a difficult operating system, more and more users are discovering the advantages of Unix for the first time. But whether you are a newcomer or a Unix power user, you'll find yourself thumbing through the goldmine of information in the new edition of Unix Power Tools to add to your store of knowledge. Want to try something new? Check this book first, and you're sure to find a tip or trick that will prevent you from learning things the hard way.The latest edition of this best-selling favorite is loaded with advice about almost every aspect of Unix, covering all the new technologies that users need to know. In addition to vital information on Linux, Darwin, and BSD, Unix Power Tools 3rd Edition now offers more coverage of bash, zsh, and other new shells, along with discussions about modern utilities and applications. Several sections focus on security and Internet access. And there is a new chapter on access to Unix from Windows, addressing the heterogeneous nature of systems today. You'll also find expanded coverage of software installation and packaging, as well as basic information on Perl and Python.Unix Power Tools 3rd Edition is a browser's book...like a magazine that you don't read from start to finish, but leaf through repeatedly until you realize that you've read it all. Bursting with cross-references, interesting sidebars explore syntax or point out other directions for exploration, including relevant technical details that might not be immediately apparent. The book includes articles abstracted from other O'Reilly books, new information that highlights program tricks and gotchas, tips posted to the Net over the years, and other accumulated wisdom.Affectionately referred to by readers as "the" Unix book, UNIX Power Tools provides access to information every Unix user is going to need to know. It will help you think creatively about UNIX, and will help you get to the point where you can analyze your own problems. Your own solutions won't be far behind. Table of Contents 5 Preface 31 A Book for Browsing 31 Like an Almanac 31 Like a News Magazine 31 Like a Hypertext Database 32 Programs on the Web 32 About Unix Versions 32 Cross-References 33 What’s New in the Third Edition 33 Typefaces and Other Conventions 33 The Authors 34 The Fine Print 35 Request for Comments 35 Acknowledgments for the FirstEdition 36 Acknowledgments for the SecondEdition 39 Acknowledgments for the ThirdEdition 39 Part I 41 Introduction 43 1.1 What’s Special About Unix? 43 1.2 Power Grows on You 44 1.3 The Core of Unix 46 1.4 Communication with Unix 46 1.5 Programs Are Designed to Work Together 48 1.6 There Are Many Shells 49 1.7 Which Shell Am I Running? 51 1.8 Anyone Can Program the Shell 51 1.9 Internal and External Commands 53 1.10 The Kernel and Daemons 54 1.11 Filenames 56 1.12 Filename Extensions 57 1.13 Wildcards 58 1.14 The Tree Structure of the Filesystem 60 1.15 Your Home Directory 62 1.16 Making Pathnames 63 1.17 File Access Permissions 65 1.18 The Superuser (Root) 66 1.19 When Is a File Not a File? 67 1.20 Scripting 67 1.21 Unix Networking and Communications 68 1.22 The X Window System 70 Getting Help 72 2.1 The man Command 72 2.2 whatis: One-Line Command Summaries 73 2.3 whereis: Finding Where a Command Is Located 73 2.4 Searching Online Manual Pages 74 2.5 How Unix Systems Remember TheirNames 76 2.6 Which Version Am I Using? 76 2.7 What tty Am I On? 77 2.8 Who’s On? 78 2.9 The info Command 78 Part II 81 Setting Up Your Unix Shell 83 3.1 What Happens When You Log In 83 3.2 The Mac OS X Terminal Application 84 3.3 Shell Setup Files—Which, Where, andWhy 84 3.4 Login Shells, Interactive Shells 88 Login Shells 88 Interactive Shells 89 3.5 What Goes in Shell Setup Files? 89 3.6 Tip for Changing Account Setup: Keep a Shell Ready 90 3.7 Use Absolute Pathnames in Shell Setup Files 91 3.8 Setup Files Aren’t Read When YouWant? 91 3.9 Gotchas in set prompt Test 93 3.10 Automatic Setups for DifferentTerminals 94 3.11 Terminal Setup: Testing TERM 95 3.12 Terminal Setup: Testing Remote Hostname and X Display 96 3.13 Terminal Setup: Testing Port 97 3.14 Terminal Setup: Testing EnvironmentVariables 97 3.15 Terminal Setup: Searching TerminalTable 98 3.16 Terminal Setup: Testing Window Size 98 3.17 Terminal Setup: Setting and Testing Window Name 99 3.18 A .cshrc.$HOST File forPerHostSetup 100 3.19 Making a “Login” Shell 101 3.20 RC Files 102 3.21 Make Your Own Manpages Without Learning troff 105 3.22 Writing a Simple Manpage with the –man Macros 107 Interacting with Your Environment 110 4.1 Basics of Setting the Prompt 110 4.2 Static Prompts 110 4.3 Dynamic Prompts 111 4.4 Simulating Dynamic Prompts 112 4.5 C-Shell Prompt Causes Problems invi,rsh,etc. 113 4.6 Faster Prompt Setting with Built-ins 114 4.7 Multiline Shell Prompts 116 4.8 Session Info in Window Title orStatusLine 117 4.9 A “Menu Prompt” for Naive Users 119 4.10 Highlighting and Color in Shell Prompts 119 4.11 Right-Side Prompts 121 4.12 Show Subshell Level with $SHLVL 122 4.13 What Good Is a Blank Shell Prompt? 123 4.14 dirs in Your Prompt: Better Than$cwd 124 4.15 External Commands Send Signals toSetVariables 126 4.16 Preprompt, Pre-execution, andPeriodicCommands 127 4.17 Running Commands When You LogOut 129 4.18 Running Commands at Bourne/Korn Shell Logout 130 4.19 Stop Accidental Bourne-Shell Logouts 130 Getting the Most out of Terminals, xterm, and X Windows 132 5.1 There’s a Lot to Know AboutTerminals 132 5.2 The Idea of a Terminal Database 133 5.3 Setting the Terminal Type When You Log In 135 5.4 Querying Your Terminal Type: qterm 137 5.5 Querying Your xterm Size: resize 139 5.6 Checklist: Terminal Hangs WhenILogIn 140 Output Stopped? 142 Job Stopped? 142 Program Waiting for Input? 142 Stalled Data Connection? 142 Aborting Programs 143 5.7 Find Out Terminal Settings with stty 144 5.8 Setting Your Erase, Kill, and InterruptCharacters 144 5.9 Working with xterm and Friends 146 5.10 Login xterms and rxvts 147 5.11 Working with Scrollbars 148 5.12 How Many Lines to Save? 149 5.13 Simple Copy and Paste in xterm 149 5.14 Defining What Makes Up a Word forSelection Purposes 150 5.15 Setting the Titlebar and Icon Text 151 5.16 The Simple Way to Pick a Font 152 5.17 The xterm Menus 153 5.18 Changing Fonts Dynamically 155 VT Fonts Menu 155 Enabling Escape Sequence and Selection 156 5.19 Working with xclipboard 157 5.20 Problems with Large Selections 159 5.21 Tips for Copy and Paste BetweenWindows 160 5.22 Running a Single Command withxterm –e 162 5.23 Don’t Quote Arguments to xterm –e 163 Your X Environment 164 6.1 Defining Keys and Button Presses with xmodmap 164 6.2 Using xev to Learn Keysym Mappings 167 6.3 X Resource Syntax 168 6.4 X Event Translations 170 6.5 Setting X Resources: Overview 173 6.6 Setting Resources with the–xrmOption 175 6.7 How –name Affects Resources 175 6.8 Setting Resources with xrdb 176 6.9 Listing the Current Resources foraClient: appres 179 6.10 Starting Remote X Clients 180 Starting Remote X Clients from Interactive Logins 181 Starting a Remote Client with rsh and ssh 182 Part III 185 Directory Organization 187 7.1 What? Me, Organized? 187 7.2 Many Homes 188 7.3 Access to Directories 188 7.4 A bin Directory for Your Programs and Scripts 189 7.5 Private (Personal) Directories 190 7.6 Naming Files 190 7.7 Make More Directories! 191 7.8 Making Directories Made Easier 192 Directories and Files 194 8.1 Everything but the find Command 194 8.2 The Three Unix File Times 194 8.3 Finding Oldest or Newest Files withls–tand ls –u 195 8.4 List All Subdirectories with ls –R 197 8.5 The ls –d Option 197 8.6 Color ls 198 Trying It 198 Configuring It 199 The --color Option 200 Another color ls 201 8.7 Some GNU ls Features 201 8.8 A csh Alias to List Recently Changed Files 202 8.9 Showing Hidden Files with ls –A and –a 203 8.10 Useful ls Aliases 203 8.11 Can’t Access a File? Look for Spaces in the Name 205 8.12 Showing Nonprintable Characters in Filenames 206 8.13 Counting Files by Types 207 8.14 Listing Files by Age and Size 208 8.15 newer: Print the Name oftheNewestFile 209 8.16 oldlinks: Find Unconnected Symbolic Links 209 8.17 Picking a Unique Filename Automatically 210 Finding Files with find 211 9.1 How to Use find 211 9.2 Delving Through a Deep DirectoryTree 213 9.3 Don’t Forget –print 215 9.4 Looking for Files with ParticularNames 215 9.5 Searching for Old Files 215 9.6 Be an Expert on find SearchOperators 216 9.7 The Times That find Finds 218 9.8 Exact File-Time Comparisons 219 9.9 Running Commands on What YouFind 219 9.10 Using –exec to Create Custom Tests 221 9.11 Custom –exec Tests Applied 222 9.12 Finding Many Things withOneCommand 222 9.13 Searching for Files by Type 224 9.14 Searching for Files by Size 225 9.15 Searching for Files by Permission 225 9.16 Searching by Owner and Group 226 9.17 Duplicating a Directory Tree 227 9.18 Using “Fast find” Databases 227 9.19 Wildcards with “Fast find” Database 229 9.20 Finding Files (Much) Faster withafind Database 230 9.21 grepping a Directory Tree 232 9.22 lookfor: Which File Has That Word? 233 9.23 Using Shell Arrays to BrowseDirectories 234 Using the Stored Lists 234 Expanding Ranges 236 9.24 Finding the (Hard) Links to a File 237 9.25 Finding Files with –prune 238 9.26 Quick finds in the Current Directory 239 9.27 Skipping Parts of a Tree in find 239 9.28 Keeping find from Searching Networked Filesystem 240 Linking, Renaming, and Copying Files 241 10.1 What’s So Complicated AboutCopying Files 241 10.2 What’s Really in a Directory? 241 10.3 Files with Two or More Names 243 10.4 More About Links 245 Differences Between Hard and Symbolic Links 246 Links to a Directory 247 10.5 Creating and Removing Links 248 10.6 Stale Symbolic Links 249 10.7 Linking Directories 250 10.8 Showing the Actual Filenames forSymbolic Links 252 10.9 Renaming, Copying, or Comparing aSet of Files 252 10.10 Renaming a List of Files Interactively 253 10.11 One More Way to Do It 253 10.12 Copying Directory Trees with cp –r 254 10.13 Copying Directory Trees withtarandPipes 256 Comparing Files 258 11.1 Checking Differences with diff 258 11.2 Comparing Three Different Versions with diff3 260 11.3 Context diffs 261 11.4 Side-by-Side diffs: sdiff 264 11.5 Choosing Sides with sdiff 265 11.6 Problems with diff and Tabstops 265 11.7 cmp and diff 266 11.8 Comparing Two Files with comm 267 11.9 More Friendly comm Output 269 11.10 make Isn’t Just for Programmers! 270 11.11 Even More Uses for make 272 Showing What’s in a File 274 12.1 Cracking the Nut 274 12.2 What Good Is a cat? 274 12.3 “less” is More 276 12.4 Show Nonprinting Characters withcat –v or od –c 277 12.5 What’s in That Whitespace? 279 12.6 Finding File Types 280 12.7 Squash Extra Blank Lines 281 12.8 How to Look at the End of a File: tail 282 12.9 Finer Control on tail 283 12.10 How to Look at Files as They Grow 283 12.11 GNU tail File Following 285 12.12 Printing the Top of a File 286 12.13 Numbering Lines 286 Searching Through Files 287 13.1 Different Versions of grep 287 13.2 Searching for Text with grep 288 13.3 Finding Text That Doesn’t Match 289 13.4 Extended Searching for Text withegrep 290 13.5 grepping for a List of Patterns 291 13.6 Approximate grep: agrep 291 13.7 Search RCS Files with rcsgrep 292 rcsgrep, rcsegrep, rcsfgrep 293 rcsegrep.fast 293 13.8 GNU Context greps 294 13.9 A Multiline Context grep Using sed 295 13.10 Compound Searches 296 13.11 Narrowing a Search Quickly 297 13.12 Faking Case-Insensitive Searches 298 13.13 Finding a Character in a Column 298 13.14 Fast Searches and Spelling Checks with “look” 299 13.15 Finding Words Inside Binary Files 299 13.16 A Highlighting grep 300 Removing Files 302 14.1 The Cycle of Creation and Destruction 302 14.2 How Unix Keeps Track ofFiles:Inodes 302 14.3 rm and Its Dangers 303 14.4 Tricks for Making rm Safer 305 14.5 Answer “Yes” or “No” Forever withyes 305 14.6 Remove Some, Leave Some 306 14.7 A Faster Way to Remove Files Interactively 306 14.8 Safer File Deletion in SomeDirectories 307 14.9 Safe Delete: Pros and Cons 308 14.10 Deletion with Prejudice: rm –f 309 14.11 Deleting Files with Odd Names 309 14.12 Using Wildcards to Delete Files withStrange Names 310 14.13 Handling a Filename Starting withaDash (–) 311 14.14 Using unlink to Remove a File withaStrange Name 311 14.15 Removing a Strange File by its i-number 312 14.16 Problems Deleting Directories 312 14.17 Deleting Stale Files 314 14.18 Removing Every File but One 315 14.19 Using find to Clear Out UnneededFiles 316 Optimizing Disk Space 317 15.1 Disk Space Is Cheap 317 15.2 Instead of Removing a File, Empty It 317 15.3 Save Space with “Bit Bucket” LogFiles and Mailboxes 319 15.4 Save Space with a Link 319 15.5 Limiting File Sizes 320 limit and ulimit 320 Other Ideas 321 15.6 Compressing Files to Save Space 321 15.7 Save Space: tar and compress aDirectoryTree 324 15.8 How Much Disk Space? 326 15.9 Compressing a Directory Tree: Fine-Tuning 328 15.10 Save Space in Executable Files withstrip 329 15.11 Disk Quotas 330 Part IV 333 Spell Checking, Word Counting, andTextualAnalysis 335 16.1 The Unix spell Command 335 16.2 Check Spelling Interactively withispell 336 16.3 How Do I Spell That Word? 338 16.4 Inside spell 339 16.5 Adding Words to ispell’s Dictionary 341 16.6 Counting Lines, Words, andCharacters: wc 343 16.7 Find a a Doubled Word 345 16.8 Looking for Closure 345 16.9 Just the Words, Please 346 vi Tips and Tricks 348 17.1 The vi Editor: Why So Much Material? 348 17.2 What We Cover 349 17.3 Editing Multiple Files with vi 349 17.4 Edits Between Files 351 17.5 Local Settings for vi 352 17.6 Using Buffers to Move or Copy Text 353 17.7 Get Back What You Deleted withNumbered Buffers 353 17.8 Using Search Patterns and Global Commands 354 Global Searches 355 17.9 Confirming Substitutions in vi 355 17.10 Keep Your Original File, Write toaNew File 356 17.11 Saving Part of a File 356 17.12 Appending to an Existing File 357 17.13 Moving Blocks of Text by Patterns 357 17.14 Useful Global Commands (withPattern Matches) 358 17.15 Counting Occurrences; Stopping Search Wraps 360 17.16 Capitalizing Every Word on a Line 360 17.17 Per-File Setups in Separate Files 361 17.18 Filtering Text Through aUnixCommand 362 17.19 vi File Recovery Versus NetworkedFilesystems 364 17.20 Be Careful with vi –r RecoveredBuffers 365 17.21 Shell Escapes: Running One Unix Command While Using Another 366 17.22 vi Compound Searches 367 17.23 vi Word Abbreviation 368 17.24 Using vi Abbreviations as Commands (Cut and Paste Between vi’s) 370 17.25 Fixing Typos with vi Abbreviations 370 17.26 vi Line Commands Versus CharacterCommands 371 17.27 Out of Temporary Space? UseAnother Directory 372 17.28 Neatening Lines 373 17.29 Finding Your Place with Undo 374 17.30 Setting Up vi with the .exrc File 374 Creating Custom Commands in vi 376 18.1 Why Type More Than You Have To? 376 18.2 Save Time and Typing with the vi map Commands 376 Command Mode Maps 377 Text-Input Mode Maps 378 18.3 What You Lose When You Use map! 379 18.4 vi @-Functions 380 Defining and Using Simple @-Functions 380 Combining @-Functions 381 Reusing a Definition 382 Newlines in an @-Function 382 18.5 Keymaps for Pasting into a Window Running vi 383 18.6 Protecting Keys from Interpretation by ex 383 18.7 Maps for Repeated Edits 385 18.8 More Examples of Mapping Keys in vi 387 18.9 Repeating a vi Keymap 388 18.10 Typing in Uppercase WithoutCAPSLOCK 388 18.11 Text-Input Mode Cursor Motion withNo Arrow Keys 389 18.12 Don’t Lose Important Functions withvi Maps: Use noremap 390 18.13 vi Macro for Splitting Long Lines 390 18.14 File-Backup Macros 391 GNU Emacs 393 19.1 Emacs: The Other Editor 393 19.2 Emacs Features: A Laundry List 394 19.3 Customizations and How toAvoidThem 398 19.4 Backup and Auto-Save Files 398 19.5 Putting Emacs in Overwrite Mode 400 19.6 Command Completion 400 19.7 Mike’s Favorite Timesavers 401 19.8 Rational Searches 402 19.9 Unset PWD Before Using Emacs 403 19.10 Inserting Binary Characters into Files 403 19.11 Using Word-Abbreviation Mode 404 Trying Word Abbreviations for One Session 404 Making Word Abbreviations Part of Your Startup 405 19.12 Directories for Emacs Hacks 406 19.13 An Absurd Amusement 406 Batch Editing 407 20.1 Why Line Editors Aren’t Dinosaurs 407 20.2 Writing Editing Scripts 408 20.3 Line Addressing 409 20.4 Useful ex Commands 410 20.5 Running Editing Scripts Within vi 413 20.6 Change Many Files by Editing JustOne 413 20.7 ed/ex Batch Edits: A Typical Example 415 20.8 Batch Editing Gotcha: Editors Fail onBig Files 416 20.9 patch: Generalized Updating of Files That Differ 417 20.10 Quick Reference: awk 418 Command-Line Syntax 419 Patterns and Procedures 420 Patterns 420 Procedures 421 Simple pattern-procedure examples 421 awk System Variables 422 Operators 422 Variables and Array Assignments 423 Group Listing of awk Commands 423 Alphabetical Summary of Commands 423 20.11 Versions of awk 428 You Can’t Quite Call This Editing 430 21.1 And Why Not? 430 21.2 Neatening Text with fmt 431 21.3 Alternatives to fmt 432 21.4 Clean Up Program Comment Blocks 434 The recomment Script 434 fmt –p 435 21.5 Remove Mail/News Headers withbehead 435 21.6 Low-Level File Butchery with dd 436 21.7 offset: Indent Text 436 21.8 Centering Lines in a File 437 21.9 Splitting Files at Fixed Points: split 438 21.10 Splitting Files by Context: csplit 441 21.11 Hacking on Characters with tr 444 21.12 Encoding “Binary” Files into ASCII 446 uuencoding 447 MIME Encoding 448 21.13 Text Conversion with dd 450 21.14 Cutting Columns or Fields 450 21.15 Making Text in Columns with pr 451 One File per Column: –m 452 One File, Several Columns: –number 452 Order Lines Across Columns: –l 453 21.16 Make Columns Automatically withcolumn 453 21.17 Straightening Jagged Columns 455 21.18 Pasting Things in Columns 456 21.19 Joining Lines with join 457 21.20 What Is (or Isn’t) Unique? 458 21.21 Rotating Text 459 Sorting 461 22.1 Putting Things in Order 461 22.2 Sort Fields: How sort Sorts 462 22.3 Changing the sort Field Delimiter 464 22.4 Confusion with Whitespace Field Delimiters 464 22.5 Alphabetic and Numeric Sorting 466 22.6 Miscellaneous sort Hints 467 Dealing with Repeated Lines 467 Ignoring Blanks 468 Case-Insensitive Sorts 468 Dictionary Order 468 Month Order 468 Reverse Sort 468 22.7 lensort: Sort Lines by Length 469 22.8 Sorting a List of People by Last Name 470 Part V 473 Job Control 475 23.1 Job Control in a Nutshell 475 23.2 Job Control Basics 477 How Job Control Works 477 Using Job Control from Your Shell 477 23.3 Using jobs Effectively 478 23.4 Some Gotchas with Job Control 480 23.5 The “Current Job” Isn’t Always What You Expect 482 23.6 Job Control and autowrite: RealTimesavers! 482 23.7 System Overloaded? Try Stopping Some Jobs 483 23.8 Notification When Jobs Change State 484 23.9 Stop Background Output withsttytostop 484 23.10 nohup 485 23.11 Disowning Processes 486 23.12 Linux Virtual Consoles 487 What Are They? 487 Scrolling, Using a Mouse 488 23.13 Stopping Remote Login Sessions 489 Starting, Stopping, and KillingProcesses 491 24.1 What’s in This Chapter 491 24.2 fork and exec 492 24.3 Managing Processes: OverallConcepts 493 24.4 Subshells 495 24.5 The ps Command 496 24.6 The Controlling Terminal 499 24.7 Tracking Down Processes 500 System V 500 BSD 501 24.8 Why ps Prints Some Commands inParentheses 502 24.9 The /proc Filesystem 503 Memory Information 503 Kernel and System Statistics 504 Statistics of the Current Process 504 Statistics of Processes by PID 505 A Glimpse at Hardware 506 24.10 What Are Signals? 508 24.11 Killing Foreground Jobs 509 24.12 Destroying Processes with kill 510 24.13 Printer Queue Watcher: A Restartable Daemon Shell Script 511 24.14 Killing All Your Processes 513 24.15 Killing Processes by Name? 514 24.16 Kill Processes Interactively 516 killall –i 516 zap 517 24.17 Processes Out of Control? JustSTOPThem 518 24.18 Cleaning Up an Unkillable Process 519 24.19 Why You Can’t Kill a Zombie 520 24.20 The Process Chain to Your Window 520 24.21 Terminal Windows Without Shells 522 24.22 Close a Window by Killing ItsProcess(es) 524 Example #1: An xterm Window 524 Example #2: A Web Browser 525 Closing a Window from a Shell Script 526 Delayed Execution 528 25.1 Building Software Robots theEasyWay 528 25.2 Periodic Program Execution: ThecronFacility 529 Execution Scheduling 529 A Little Help, etc. 532 25.3 Adding crontab Entries 534 25.4 Including Standard Input Withinacron Entry 535 25.5 The at Command 535 25.6 Making Your at Jobs Quiet 536 25.7 Checking and Removing Jobs 536 25.8 Avoiding Other at and cron Jobs 537 25.9 Waiting a Little While: sleep 538 System Performance and Profiling 540 26.1 Timing Is Everything 540 26.2 Timing Programs 543 26.3 What Commands Are Running and How Long Do They Take? 544 26.4 Checking System Load: uptime 546 26.5 Know When to Be “nice” to Other Users...and When Not To 546 BSD C Shell nice 548 BSD Standalone nice 548 System V C Shell nice 549 System V Standalone nice 549 26.6 A nice Gotcha 550 26.7 Changing a Running Job’s Niceness 550 Part VI 551 Shell Interpretation 553 27.1 What the Shell Does 553 27.2 How the Shell Executes Other Commands 554 27.3 What’s a Shell, Anyway? 555 How Shells Run Other Programs 555 Interactive Use Versus Shell Scripts 555 Types of Shells 555 Shell Search Paths 556 Bourne Shell Used Here 556 Default Commands 556 27.4 Command Evaluation and Accidentally Overwriting Files 557 27.5 Output Command-Line Arguments One by One 558 27.6 Controlling Shell Command Searches 558 27.7 Wildcards Inside Aliases 560 27.8 eval: When You Need Another Chance 561 27.9 Which One Will bash Use? 563 27.10 Which One Will the C Shell Use? 564 27.11 Is It “2>&1 file” or “> file 2>&1”? Why? 566 27.12 Bourne Shell Quoting 566 Special Characters 567 How Quoting Works 567 Single Quotes Inside Single Quotes? 569 Multiline Quoting 570 27.13 Differences Between Bourne andCShellQuoting 571 Special Characters 571 How Quoting Works 572 27.14 Quoting Special Characters inFilenames 573 27.15 Verbose and Echo Settings ShowQuoting 573 27.16 Here Documents 574 27.17 “Special” Characters and Operators 575 27.18 How Many Backslashes? 580 Saving Time on the Command Line 582 28.1 What’s Special About the Unix Command Line 582 28.2 Reprinting Your Command Line withCTRL-r 583 28.3 Use Wildcards to Create Files? 584 28.4 Build Strings with {} 585 28.5 String Editing (Colon) Operators 587 28.6 Automatic Completion 589 General Example: Filename Completion 590 Menu Completion 591 Command-Specific Completion 591 Editor Functions for Completion 592 28.7 Don’t Match Useless Files in Filename Completion 593 28.8 Repeating Commands 594 28.9 Repeating and Varying Commands 594 A foreach Loop 594 A for Loop 596 28.10 Repeating a Command with Copy-and-Paste 597 28.11 Repeating a Time-Varying Command 598 28.12 Multiline Commands, Secondary Prompts 599 28.13 Here Document Example #1: Unformatted Form Letters 600 28.14 Command Substitution 601 28.15 Handling Lots of Text with TemporaryFiles 603 28.16 Separating Commands withSemicolons 603 28.17 Dealing with Too Many Arguments 605 28.18 Expect 607 Dialback 608 Automating /bin/passwd 608 Testing: A Story 610 Other Problems 610 Custom Commands 611 29.1 Creating Custom Commands 611 29.2 Introduction to Shell Aliases 611 29.3 C-Shell Aliases with Command-Line Arguments 612 29.4 Setting and Unsetting Bourne-Type Aliases 614 29.5 Korn-Shell Aliases 615 29.6 zsh Aliases 616 29.7 Sourceable Scripts 616 29.8 Avoiding C-Shell Alias Loops 618 29.9 How to Put if-then-else in a C-Shell Alias 619 29.10 Fix Quoting in csh Aliases with makealias and quote 620 29.11 Shell Function Basics 621 Simple Functions: ls with Options 621 Functions with Loops: Internet Lookup 621 Setting Current Shell Environment: TheworkFunction 622 Functions Calling Functions: Factorials 623 Conclusion 625 29.12 Shell Function Specifics 625 29.13 Propagating Shell Functions 626 Exporting bash Functions 626 FPATH Search Path 627 Korn shell 628 zsh 630 29.14 Simulated Bourne Shell Functions and Aliases 631 The Use of History 633 30.1 The Lessons of History 633 30.2 History in a Nutshell 634 30.3 My Favorite Is !$ 635 30.4 My Favorite Is !:n* 635 30.5 My Favorite Is ^^ 636 30.6 Using !$ for Safety with Wildcards 637 30.7 History by Number 637 30.8 History Substitutions 639 30.9 Repeating a Cycle of Commands 644 30.10 Running a Series of Commands onaFile 644 30.11 Check Your History First with :p 645 30.12 Picking Up Where You Left Off 646 bash, ksh, zsh 646 C Shells 647 30.13 Pass History to Another Shell 648 30.14 Shell Command-Line Editing 649 vi Editing Mode 650 Emacs Editing Mode 651 tcsh Editing 651 ksh Editing 653 bash Editing 654 zsh Editing 655 30.15 Changing History Characters withhistchars 655 30.16 Instead of Changing History Characters 656 Moving Around in a Hurry 657 31.1 Getting Around the Filesystem 657 31.2 Using Relative and Absolute Pathnames 658 31.3 What Good Is a Current Directory? 660 31.4 How Does Unix Find Your CurrentDirectory? 661 31.5 Saving Time When You Change Directories: cdpath 662 31.6 Loop Control: break and continue 663 31.7 The Shells’ pushd and popd Commands 664 31.8 Nice Aliases for pushd 666 31.9 Quick cds with Aliases 667 31.10 cd by Directory Initials 667 31.11 Finding (Anyone’s) Home Directory, Quickly 669 31.12 Marking Your Place with a Shell Variable 670 31.13 Automatic Setup When You Enter/Exit a Directory 670 Regular Expressions (PatternMatching) 673 32.1 That’s an Expression 673 32.2 Don’t Confuse Regular Expressions with Wildcards 674 32.3 Understanding Expressions 675 32.4 Using Metacharacters in Regular Expressions 677 32.5 Regular Expressions: The Anchor Characters ^ and $ 678 32.6 Regular Expressions: Matching a Character with a Character Set 679 32.7 Regular Expressions: Match Any Character with . (Dot) 680 32.8 Regular Expressions: Specifying a Range of Characters with [...] 680 32.9 Regular Expressions: Exceptions inaCharacter Set 681 32.10 Regular Expressions: Repeating Character Sets with * 681 32.11 Regular Expressions: Matching a Specific Number of Sets with \{ and \} 682 32.12 Regular Expressions: Matching Words with \ 683 32.13 Regular Expressions: Remembering Patterns with \(, \), and \1 684 32.14 Regular Expressions: PotentialProblems 684 32.15 Extended Regular Expressions 685 32.16 Getting Regular Expressions Right 686 32.17 Just What Does a Regular Expression Match? 688 32.18 Limiting the Extent of a Match 689 32.19 I Never Meta Character I Didn’t Like 690 32.20 Valid Metacharacters for Different Unix Programs 691 32.21 Pattern Matching Quick Reference with Examples 692 Examples of Searching 694 Examples of Searching and Replacing 695 Wildcards 697 33.1 File-Naming Wildcards 697 33.2 Filename Wildcards in a Nutshell 698 33.3 Who Handles Wildcards? 700 33.4 What if a Wildcard Doesn’t Match? 702 33.5 Maybe You Shouldn’t Use Wildcards in Pathnames 703 33.6 Getting a List of Matching Files withgrep –l 704 33.7 Getting a List of Nonmatching Files 704 Using grep –c 705 The vgrep Script 705 33.8 nom: List Files That Don’t Match aWildcard 706 The sed Stream Editor 708 34.1 sed Sermon^H^H^H^H^H^HSummary 708 34.2 Two Things You Must Know Aboutsed 709 34.3 Invoking sed 709 34.4 Testing and Using a sed Script: checksed, runsed 710 checksed 711 runsed 711 34.5 sed Addressing Basics 712 34.6 Order of Commands in a Script 714 34.7 One Thing at a Time 715 34.8 Delimiting a Regular Expression 715 34.9 Newlines in a sed Replacement 716 34.10 Referencing the Search String inaReplacement 717 34.11 Referencing Portions ofaSearchString 718 34.12 Search and Replacement: One Match Among Many 719 34.13 Transformations on Text 720 34.14 Hold Space: The Set-Aside Buffer 720 34.15 Transforming Part of a Line 723 34.16 Making Edits Across Line Boundaries 725 34.17 The Deliberate Scrivener 728 34.18 Searching for Patterns Split AcrossLines 730 34.19 Multiline Delete 732 34.20 Making Edits Everywhere Except... 733 34.21 The sed Test Command 735 34.22 Uses of the sed Quit Command 736 34.23 Dangers of the sed Quit Command 736 34.24 sed Newlines, Quoting, and Backslashes in a Shell Script 737 Shell Programming for the Uninitiated 738 35.1 Writing a Simple Shell Program 738 35.2 Everyone Should Learn Some ShellProgramming 740 35.3 What Environment Variables AreGood For 742 35.4 Parent-Child Relationships 745 35.5 Predefined Environment Variables 745 35.6 The PATH Environment Variable 748 35.7 PATH and path 749 35.8 The DISPLAY Environment Variable 750 35.9 Shell Variables 751 35.10 Test String Values with Bourne-Shellcase 753 35.11 Pattern Matching in case Statements 754 35.12 Exit Status of Unix Processes 755 35.13 Test Exit Status with the if Statement 756 35.14 Testing Your Success 758 35.15 Loops That Test Exit Status 759 Looping Until a Command Succeeds 759 Looping Until a Command Fails 760 35.16 Set Exit Status of a Shell (Script) 760 35.17 Trapping Exits Caused by Interrupts 761 35.18 read: Reading from the Keyboard 763 35.19 Shell Script “Wrappers” forawk,sed,etc. 764 35.20 Handling Command-Line Arguments in Shell Scripts 765 With the “$@” Parameter 765 With a Loop 766 Counting Arguments with $# 766 35.21 Handling Command-Line Arguments with a for Loop 767 35.22 Handling Arguments with while and shift 768 35.23 Loop Control: break and continue 770 35.24 Standard Command-Line Parsing 770 35.25 The Bourne Shell set Command 772 Setting Options 773 Setting (and Parsing) Parameters 773 (Avoiding?) set with No Arguments 774 Watch Your Quoting 774 Can’t Set $0 775 35.26 test: Testing Files and Strings 775 35.27 Picking a Name for a New Command 776 35.28 Finding a Program Name and Giving Your Program Multiple Names 776 35.29 Reading Files with the . and source Commands 777 35.30 Using Shell Functions in Shell Scripts 778 Shell Programming for the Initiated 781 36.1 Beyond the Basics 781 36.2 The Story of : # #! 782 36.3 Don’t Need a Shell for Your Script? Don’t Use One 783 36.4 Making #! Search the PATH 784 36.5 The exec Command 785 36.6 The Unappreciated Bourne Shell “:”Operator 786 36.7 Parameter Substitution 787 36.8 Save Disk Space and Programming: Multiple Names for a Program 788 36.9 Finding the Last Command-Line Argument 789 36.10 How to Unset All Command-Line Parameters 789 36.11 Standard Input to a for Loop 790 36.12 Making a for Loop with MultipleVariables 790 36.13 Using basename and dirname 791 Introduction to basename and dirname 791 Use with Loops 792 36.14 A while Loop with Several Loop Control Commands 793 36.15 Overview: Open Files andFileDescriptors 793 36.16 n>&m: Swap Standard Output andStandard Error 796 36.17 A Shell Can Read a Script from Its Standard Input, but... 799 36.18 Shell Scripts On-the-Fly fromStandard Input 800 36.19 Quoted hereis Document Terminators: sh Versus csh 801 36.20 Turn Off echo for “Secret” Answers 801 36.21 Quick Reference: expr 802 Syntax 802 Examples 803 36.22 Testing Characters in a String withexpr 804 36.23 Grabbing Parts of a String 804 Matching with expr 805 Using echo with awk or cut 805 Using set and IFS 806 Using sed 807 36.24 Nested Command Substitution 808 36.25 Testing Two Strings with One caseStatement 810 36.26 Outputting Text to an X Window 810 36.27 Shell Lockfile 812 Shell Script Debugging and Gotchas 815 37.1 Tips for Debugging Shell Scripts 815 Use –xv 815 Unmatched Operators 816 Exit Early 816 Missing or Extra esac, ;;, fi, etc. 816 Line Numbers Reset Inside Redirected Loops 816 37.2 Bourne Shell Debugger Shows aShellVariable 817 37.3 Stop Syntax Errors in Numeric Tests 817 37.4 Stop Syntax Errors in String Tests 818 37.5 Quoting and Command-Line Parameters 819 37.6 How Unix Keeps Time 821 37.7 Copy What You Do with script 822 37.8 Cleaning script Files 823 37.9 Making an Arbitrary-Size File forTesting 824 Part VII 825 Backing Up Files 8 By its very nature, Unix is a "power tools" environment. Even beginning Unix users quickly grasp that immense power exists in shell programming, aliases and history mechanisms, and various editing tools. Nonetheless, few users ever really master the power available to them with Unix. There is just too much to learn! Unix Power Tools, Third Edition, literally contains thousands of tips, scripts, and techniques that make using Unix easier, more effective, and even more fun. This book is organized into hundreds of short articles with plenty of references to other sections that keep you flipping from new article to new article. You'll find the book hard to put down as you uncover one interesting tip after another.With the growing popularity of Linux and the advent of Mac OS X, Unix has metamorphosed into something new and exciting. With Unix no longer perceived as a difficult operating system, more and more users are discovering its advantages for the first time. The latest edition of this best-selling favorite is loaded with advice about almost every aspect of Unix, covering all the new technologies that users need to know. In addition to vital information on Linux, Mac OS X, and BSD, Unix Power Tools, Third Edition, now offers more coverage of bash, zsh, and new shells, along with discussions about modern utilities and applications. Several sections focus on security and Internet access, and there is a new chapter on access to Unix from Windows, addressing the heterogeneous nature of systems today. You'll also find expanded coverage of software installation and packaging, as well as basic information on Perl and Python.The book's accompanying web site provides some of the best software available to Unix users, which you can download and add to your own set of power tools. Whether you are a newcomer or a Unix power user, you'll find yourself thumbing through the gold mine of information in this new edition of Unix Power Tools to add to your store of knowledge. Want to try something new? Check this book first, and you're sure to find a tip or trick that will prevent you from learning things the hard way.