Pro Bash: Learn to Script and Program the GNU/Linux Shell - Third Edition

Table of Contents 5 About the Authors 16 About the Technical Reviewer 17 Acknowledgments 18 Chapter 1: Hello, World: Your First Shell Program 19 What Is a Shell Script? 19 The Hello World Code 20 The Hello World Program File 21 Naming the Script File 21 Selecting a Directory for the Script 22 Creating the File and Running the Script 23 Choosing and Using a Text Editor 24 Building a Better “Hello, World!” 25 Summary 25 Commands 26 Concepts 26 Variables 27 Exercises 27 Chapter 2: Input, Output, and Throughput 28 Parameters and Variables 28 Positional Parameters 29 Special *@#0$?_!- Parameters 30 Variables 30 Arguments and Options 30 Why You Should Avoid echo 31 printf: Formatting and Printing Data 32 Escape Sequences 33 Format Specifiers 33 Width Specification 35 Printing to a Variable 37 Line Continuation 37 Standard Input/Output Streams and Redirection 37 Redirection: >, >>, and < 38 Reading Input 40 Pipelines 41 Command Substitution 41 Summary 42 Commands 42 Concepts 42 Exercises 43 Chapter 3: Looping and Branching 44 Exit Status 44 Testing an Expression 45 test, a.k.a. [ … ] 45 File Tests 45 Integer Tests 46 String Tests 46 [[ … ]]: Evaluate an Expression 48 Enhancements over Test 48 (( …)): Evaluate an Arithmetic Expression 48 Conditional Execution 49 if 49 Conditional Operators: && and || 51 case 52 Looping 53 while 53 until 54 for 55 break 55 continue 56 Summary 56 Commands 57 Concepts 57 Exercises 57 Chapter 4: Command-Line Parsing and Expansion 58 Quoting 59 Brace Expansion 61 Tilde Expansion 63 Parameter and Variable Expansion 64 Arithmetic Expansion 65 Command Substitution 68 Word Splitting 68 Pathname Expansion 70 Process Substitution 71 Parsing Options 72 Summary 75 Commands 75 Exercises 75 Chapter 5: Parameters and Variables 77 The Naming of Variables 77 The Scope of a Variable: Can You See It from Here? 78 Shell Variables 81 Shell Variables 81 Parameter Expansion 83 Bourne Shell 83 ${var:-default} and ${var-default}: Use Default Values 83 ${var:+alternate} and ${var+alternate}: Use Alternate Values 84 ${var:=default} and ${var=default}: Assign Default Values 86 ${var:?message} and ${var?message}: Display Error Message If Empty or Unset 86 POSIX Shell 87 ${#var}: Length of Variable’s Contents 87 ${var%PATTERN}: Remove the Shortest Match from the End 88 ${var%%PATTERN}: Remove the Longest Match from the End 89 ${var#PATTERN}: Remove the Shortest Match from the Beginning 89 ${var##PATTERN}: Remove the Longest Match from the Beginning 89 bash 90 ${var//PATTERN/STRING}: Replace All Instances of PATTERN with STRING 90 ${var:OFFSET:LENGTH}: Return a Substring of $var 91 ${!var}: Indirect Reference 92 bash-4.0 92 ${var^PATTERN}: Convert to Uppercase 92 ${var,PATTERN}: Convert to Lowercase 93 Positional Parameters 93 Arrays 94 Integer-Indexed Arrays 95 Displaying Arrays 95 Assigning Array Elements 96 Associative Arrays 97 Summary 98 Commands 98 Concepts 99 Exercises 99 Chapter 6: Shell Functions 100 Definition Syntax 100 Compound Commands 103 Getting Results 104 Set Different Exit Codes 104 Print the Result 105 Place Results in One or More Variables 106 Function Libraries 107 Using Functions from Libraries 107 Sample Script 108 Summary 110 Commands 110 Exercises 111 Chapter 7: String Manipulation 112 Concatenation 113 Repeat Character to a Given Length 114 Processing Character by Character 115 Reversal 116 Case Conversion 117 Comparing Contents Without Regard to Case 120 Check for Valid Variable Name 121 Insert One String into Another 122 Examples 123 Overlay 123 Examples 124 Trim Unwanted Characters 125 Examples 126 Index 126 Summary 128 Commands 128 Functions 128 Exercises 129 Chapter 8: File Operations and Commands 130 Reading a File 131 External Commands 132 cat 133 head 134 touch 135 ls 136 cut 137 wc 138 Regular Expressions 139 grep 139 sed 140 awk 142 File Name Expansion Options 145 nullglob 146 failglob 146 dotglob 147 extglob 147 ?(pattern-list) 148 *(pattern-list) 148 @(pattern-list) 148 +(pattern-list) 148 !(pattern-list) 149 nocaseglob 149 globstar 150 Summary 151 Shell Options 151 External Commands 151 Exercises 152 Chapter 9: Reserved Words and Built-In Commands 153 help, Display Information About Built-In Commands 153 time, Print Time Taken for Execution of a Command 155 read, Read a Line from an Input Stream 156 -r, Read Backslashes Literally 156 -e, Get Input with the readline Library 157 -a, Read Words into an Array 158 -d DELIM, Read Until DELIM Instead of a Newline 158 -n NUM, Read a Maximum of NUM Characters 159 -s, Do Not Echo Input Coming from a Terminal 159 -p PROMPT:, Output PROMPT Without a Trailing Newline 159 -t TIMEOUT, Only Wait TIMEOUT Seconds for Complete Input 159 -u FD: Read from File Descriptor FD Instead of the Standard Input 160 -i TEXT, Use TEXT As the Initial Text for Readline 161 eval, Expand Arguments and Execute the Resulting Command 162 Poor Man’s Arrays 162 Setting Multiple Variables from One Command 165 type, Display Information About Commands 166 builtin, Execute a Built-In Command 168 command, Execute a Command or Display Information About Commands 168 pwd, Print the Current Working Directory 169 unalias, Remove One or More Aliases 169 Deprecated Built-Ins 169 Dynamically Loadable Built-Ins 170 Summary 171 Commands and Reserved Words 171 Deprecated Commands 172 Exercise 172 Chapter 10: Writing Bug-Free Scripts and Debugging the Rest 173 Prevention Is Better Than Cure 174 Structure Your Programs 174 Comments 174 Initialization of Variables 175 Function Definitions 176 Runtime Configuration and Options 176 Process Information 177 Document Your Code 178 Format Your Code Consistently 178 The K.I.S.S. Principle 179 Grouping Commands 180 Test As You Go 181 Debugging a Script 183 Summary 188 Exercises 188 Chapter 11: Programming for the Command Line 189 Manipulating the Directory Stack 189 cd 190 pd 191 cdm 191 dirs Built-In Command 192 menu 193 Filesystem Functions 195 l 195 lsr 196 cp, mv 197 md 198 Miscellaneous Functions 198 pr1 198 calc 200 Managing Man Pages 200 sman 200 sus 201 k 201 Games 202 The fifteen Puzzle 203 Summary 209 Exercises 209 Chapter 12: Runtime Configuration 210 Defining Variables 210 Command-Line Options and Arguments 211 Menus 211 Q&A Dialogue 212 Configuration Files 213 Scripts with Several Names 215 Environment Variables 216 All Together Now 217 Script Information 218 Default Configuration 218 Screen Variables 219 Function Definitions 220 Function: die 220 Function: menu 220 The Upload Settings Menu 221 Function: qa 224 Function: print_config 225 Function: readline 226 Parse Command-Line Options 227 Bits and Pieces 228 Summary 230 Exercises 230 Chapter 13: Data Processing 231 Arrays 231 Holes in an Indexed Array 231 Using an Array for Sorting 233 Insertion Sort Function 235 Searching an Array 236 Reading an Array into Memory 237 Two-Dimensional Grids 240 Working with Single-String Grids 241 Function: initgrid 241 Function: gridindex 242 Function: putgrid 243 Function: getgrid 244 Function: showgrid 245 Function: rshowgrid 246 Two-Dimensional Grids Using Arrays 247 Function: initagrid 249 Function: putagrid 250 Function: getagrid 250 Function: showagrid 251 Function: rshowagrid 251 Data File Formats 252 Line-Based Records 252 Delimiter-Separated Values 253 Fixed-Length Fields 256 Block File Formats 257 Summary 259 Exercises 259 Chapter 14: Scripting the Screen 260 Teletypewriter vs. Canvas 261 Stretching the Canvas 261 Control Sequence Introducer 262 Priming the Canvas 262 Moving the Cursor 263 Changing Rendition Modes and Colors 264 Placing a Block of Text on the Screen 266 Scrolling Text 270 Rolling Dice 271 Summary 274 Exercises 274 Chapter 15: Entry-Level Programming 275 Single-Key Entry 275 Function Library, key-funcs 276 History in Scripts 284 Sanity Checking 285 Form Entry 286 Reading the Mouse 288 Summary 292 Exercises 293 Appendix A: Shell Variables 294 Index 310 Learn how to effectively utilize the Bash shell in your programming. This refreshed and expanded third edition has been updated to Bash 5.2, and many scripts have been rewritten to make them more idiomatically Bash, taking better advantage of features specific to Bash. It is easy to read, understand, and will teach you how to get to grips with Bash programming without drowning you in pages and pages of syntax. Using this book you will be able to use the shell efficiently, make scripts run faster using expansion and external commands, and understand how to overcome many common mistakes that cause scripts to fail. This book is perfect for all beginning Linux and Unix system administrators who want to be in full control of their systems, and really get to grips with Bash programming. The Bash shell is a complete programming language, not merely a glue to combine external Linux commands. By taking full advantage of Shell internals, Shell programs can perform as snappily as utilities written in C or other compiled languages. And you will see how, without assuming UNIX lore, you can write professional Bash programs through standard programming techniques. What You'll Learn Use the Bash shell to write utilities and accomplish most programming tasks Replace many external commands with shell parameter expansion making scripts very fast Avoid many common mistakes that cause scripts to fail See how Bash’s read line and history libraries can save typing when getting user input Build shell scripts that get information from the Web Who This Book Is For Developers, programmers, and open source enthusiasts who want to write scripts using Bash on multiple platforms