The Ultimate Linux Shell Scripting Guide: Automate, Optimize, and Empower tasks with Linux Shell Scripting

Embark on a dynamic journey through command shells with this hands-on manual. From the Linux staple Bash to rising stars like Z shell and PowerShell, master their power for Linux, macOS, and Unix. Cover Copyright Contributors Table of Contents Preface Chapter 1: Getting Started with the Shell Understanding Shells Finding Help with Shell Commands Understanding Manual Pages Understanding Info Pages Getting to Know the Linux Documentation Project Using Your Favorite Search Engine Using a Text Editor to Create Shell Scripts Text-mode Editors GUI Text Editors Understanding Compiled versus Interpreted Programming Understanding root and sudo Privileges Summary Questions Further Reading Answers Chapter 2: Interpreting Commands Understanding the Structure of a Command Using Command Options Hands-on Lab – Practice With Command Options Using Command Arguments Executing Multiple Commands at Once Running Commands Interactively Using Command Sequences Chaining Commands with a Semi-Colon Conditional Command Execution with Double Ampersands Conditional Command Execution with Double Pipes Using the find Utility Performing Multiple Actions with find Hands-on Lab – Using find to Perform Other Commands Running Commands Recursively Hands-on Lab – Using Commands with Recursion Understanding the Command History Escaping and Quoting Escaping Metacharacters Quoting Summary Questions Further Reading Answers Chapter 3: Understanding Variables and Pipelines Understanding Environmental Variables Understanding Programming Variables Understanding Pipelines Summary Questions Further Reading Answers Chapter 4: Understanding Input/Output Redirection Introduction to Input/Output Redirection Understanding stdout Preventing File Overwrites Using the File Descriptor Understanding stdin Understanding stderr Understanding tee Hands-on Lab – Pipes, Redirectors, and find Summary Questions Further Reading Answers Chapter 5: Customizing the Environment Technical Requirements Reviewing the Environmental Variables Understanding Shell Sessions Understanding the Configuration Files bash Global Configuration Files on Fedora Users’ Configuration Files on Fedora bash Global Configuration Files on Debian Users’ Configuration Files on Debian Setting the Default Editor on Debian Setting Shell Options from the Command-line Understanding Aliases Summary Questions Further Reading Answers Chapter 6: Text-Stream Filters – Part 1 Technical Requirements Introduction to Text-Stream Filters Using cat Using tac Using cut Using paste Using join Using sort Summary Questions Further Reading Answers Chapter 7: Text Stream Filters – Part 2 Technical Requirements Using expand Using unexpand Using nl Using head Using tail Using Head And Tail Together Using od Using uniq Using wc Using fmt Using split Using tr Using xargs Using pr Printing from the Command-line Summary Questions Further Reading Answers Chapter 8: Basic Shell Script Construction Technical Requirements Understanding Basic Shell Script Construction Hands-on Lab – Counting Logged-in Users Performing Tests Using the test Keyword Enclosing a test Condition Within Square Brackets Using an if. . .then Construct Using Other Types of Tests Understanding Subshells Hands-on Lab – Testing Conditions Understanding Scripting Variables Creating and Deleting Variables Understanding Variables and Shell Levels Understanding Case Sensitivity Understanding Read-Only Variables Understanding Array Variables Hands-on Lab – Using Arrays Understanding Variable Expansion Substituting a Value for an Unset Variable Substituting a Value for a Set Variable Assigning a Value to a Variable Displaying an Error Message Using Variable Offsets Matching Patterns Understanding Command Substitution Understanding Decisions and Loops The if. .then Construct The do. . while construct The for..in Construct The for Construct Using break Using continue The until Construct The case Construct Using Positional Parameters Understanding Exit Codes Standard Shell Exit Codes User-defined Exit Codes More Information About echo Looking at Some Real-World Examples Hands-on Lab: Using if..then Hands-on Lab – Parsing an Apache Access Log Hands-on Lab – Beta Testing a Hard Drive Summary Questions Further Reading Answers Chapter 9: Filtering Text with grep, sed, and Regular Expressions Technical Requirements Understanding Regular Expressions Literals and Metacharacters Understanding sed Understanding sed Portability Issues Installing gsed on FreeBSD Installing gsed on macOS Installing gsed on OpenIndiana Substitution with sed Example 1: Modifying an Office Memo Example 2: Modifying a List of Hollywood Actors Example 3: Modifying Lists of Cars Example 4: Performing a Whole-Word Substitution Deletion with sed Example 1: Deleting Items from a List Example 2: Deleting Blank Lines Appending and Inserting with sed Example 1: Appending Lines of Text Example 2: Performing Multiple Operations at Once Example 3: Inserting Lines of Text Changing with sed Example 1: Changing Edsel to Studebaker Example 2: Changing Entire Lines of Text Other Miscellaneous sed tricks Example 1: Using the q Command Example 2: Using the w Command Example 3: Using the r Command Using sed program files Example 1: Appending Lines in a Text File Example 2: Changing Lines in a Text File Example 3: Substituting Text Example 4: Copying Lines from One File to Another Compound Scripts in sed Program Files Using sed in Shell Scripts Understanding grep Basic Searches with grep More Advanced Searches with grep Example 1: Searching for Whole Words Even More Advanced Searches with grep Example 1: Auditing Source Code Files Example 2: Searching for Social Security Numbers Example 3: Using the ^ Metacharacter Using Extended Regular Expressions with grep Example 1: Basic Search with Extended Syntax Example 2: Searching for Consecutive Duplicate Words Example 3: Searching for Words that Begin with a Certain Letter Example 4: Searching for Words with Digits Using Fixed-strings Regular Expressions with grep Using RegEx Helper Programs RegexBuddy and RegexMagic Regex101 Looking at Some Real-World Examples Modifying Multiple Files at Once Searching Through Apache Webserver Logs for Cross-site Scripting Attacks Automating Third-party Repository Installations Filling Empty Fields in a .csv File Summary Questions Further Reading Answers Chapter 10: Understanding Functions Technical Requirements Introduction to Functions Defining a Function Using Functions in Shell Scripts Creating and Calling Functions Passing Positional Parameters to Functions Passing Values from a Function Creating Function Libraries Looking at Some Real-World Examples Checking Network Connectivity Using the CoinGecko API Hands-on Lab – Creating the coingecko.sh Script Summary Questions Further Reading Answers Chapter 11: Performing Mathematical Operations Technical Requirements Performing Integer Math with Expressions Using the expr Command Using echo with Math Expressions Performing Integer Math with Integer Variables Performing Floating Point Math with bc Using bc in Interactive Mode Using bc Program Files Using bc in Shell Scripts Summary Questions Further Reading Answers Chapter 12: Automating Scripts with here Documents and expect Technical Requirements Using here Documents Creating here Documents with Static Data Creating here documents with Dynamic Data Using Functions in here Documents Automating Responses with expect Security Implications with expect Summary Questions Further Reading Answers Chapter 13: Scripting with ImageMagick Technical Requirements Converting Non-standard Filename Extensions Installing ImageMagick Displaying Images Viewing Image Properties Resizing and Customizing Images Batch-processing Image Files Using Fred’s ImageMagick Scripts Summary Questions Further Reading Answers Chapter 14: Using awk – Part 1 Introducing awk Understanding Patterns and Actions Obtaining Input from Text Files Looking for Human Users Parsing Webserver Access Logs Using Regular Expressions Obtaining Input from Commands Summary Questions Further Reading Answers Chapter 15: Using awk – Part 2 Technical Requirements Basic awk Script Construction Using Conditional Statements Using a while Construct and Setting Variables Summing Numbers in a Line Finding the CPU Generation Using for loops and Arrays Using Floating Point Math and printf Working with Multi-Line Records Summary Questions Further Reading Answers Chapter 16: Creating User Interfaces with yad, dialog, and xdialog Technical Requirements Creating a Graphical User Interface with yad The yad Basics Creating Data Entry Forms Creating a Drop-down List Using the yad File Manager Creating a File Checksum Utility Creating a GUI Front-end for ImageMagick Programming Form Buttons Some Final Thoughts about yad Creating User Interfaces with dialog and xdialog The dialog Basics The xdialog Basics Automatically Choosing Either dialog or xdialog Adding Widgets Creating an SSH Login Interface Summary Questions Further Reading Answers Chapter 17: Using Shell Script Options with getops Technical Requirements Understanding the Need for getopts Understanding getopt versus getopts Using getopts Looking at Real-world Examples The Modified Coingecko Script The Tecmint Monitor Script Summary Questions Further Reading Answers Chapter 18: Shell Scripting for Security Professionals Technical Requirements Simple Scripts for Auditing Identifying an Operating System A Simple Port-scanning Script Auditing the root User Account Creating the root Account Auditing Script for Linux and OpenIndiana Modifying the root Account Auditing Script for Use on FreeBSD Creating a User Activity Monitoring Script Creating Simple Firewall Scripts Creating an IP Address Blocking Script for Red Hat Distros Hands-on Lab: Create the Script with an Array and a for loop Hands-on Lab: Creating the Script with xargs Searching for Existing Security-related Scripts Summary Questions Further Reading Answers Chapter 19: Shell Script Portability Technical Requirements Running bash on Non-Linux Systems Using env to Set the bash Environment Creating a Symbolic Link to bash Understanding POSIX compliance Understanding the Differences Between Shells Understanding Bashisms Using Portable Tests Making Portable Arrays Understanding Portability Problems with echo Testing Scripts for POSIX Compliance Creating Scripts on a POSIX-compliant Shell Using checkbashisms Using shellcheck Specifying a Shell with the -s Option Hands-on Lab – Using -s to Scan Function Libraries Using shall Summary Questions Further Reading Answers Chapter 20: Shell Script Security Technical Requirements Controlling Access to Your Scripts Assigning sudo Privileges Hands-on Lab ­– Configuring sudo Using an Access Control List Hands-on Lab – Setting an ACL for Horatio on Linux Hands-on Lab – Setting an ACL for Horatio on FreeBSD 14 Hands-on Lab – Setting an ACL for Horatio on OpenIndiana Obfuscating Plain-Text Scripts Installing shc Hands-on Lab – Using shc Hands-on Lab – Creating Untraceable Executables Decrypting shc Binaries Understanding SUID and SGID Considerations Avoiding Sensitive Data Leakage Securing Temporary Files Understanding the /tmp/ Directory The Wrong Way to Create Temporary Files The Right Way to Create Temporary Files Using Passwords in Shell Scripts Hands-on Lab – Encrypting Passwords Understanding Command Injection with eval Using eval on the Command-line Using eval Safely Using eval Dangerously Using Alternatives to eval Using Command Substitution Evaluating if eval is Necessary Understanding Path Security Attack Scenario 1: Compromising the User’s Account Attack Scenario 2: Social Engineering Summary Questions Further Reading Answers Chapter 21: Debugging Shell Scripts Technical Requirements Understanding Common Scripting Errors Not Enough Quoting Filenames with Blank Spaces Problems with Unset Variables Creating a Wild Loop Using Shell Script Debugging Tools and Techniques Using echo Statements Using xtrace for Debugging Checking for Undefined Variables Checking for Errors with the -e Option Understanding the Problems with set -e and -e Using bash Debugger Installing bashdb on Linux Installing bashdb on FreeBSD Installing on macOS Debugging a Script with bashdb Getting Help with bashdb Summary Questions Further Reading Answers Chapter 22: Introduction to Z Shell Scripting Technical Requirements Introducing zsh Installing zsh Understanding the Unique Features of zsh Scripting Differences in Variable Expansion Substituting Values Substituting Substrings Translating Between Upper and Lower Case Extended File Globbing Understanding zsh Arrays Enhanced Math Capabilities Using zsh Modules Using the mathfunc Module The datetime Module Summary Questions Further Reading Answers Chapter 23: Using PowerShell on Linux Technical Requirements Installing PowerShell on Linux and macOS Installing PowerShell on Linux via a snap Package Installing PowerShell on Fedora Installing PowerShell on macOS Invoking PowerShell Reasons for Linux and Mac Admins to Learn PowerShell Working with Mixed Operating System Environments PowerShell Commands Can Be Simpler Enhanced Builtin Math Capabilities Differences Between PowerShell Scripting and Traditional Linux/Unix Scripting Using Filename Extensions and the Executable Permission PowerShell is Object-oriented PowerShell Uses Cmdlets Using Aliases on PowerShell Viewing the Available PowerShell Commands Getting Help with PowerShell Commands Real-World Cross-Platform PowerShell Scripts The write-marquee.ps1 Script The check-cpu.ps1 Script Summary Further Reading Packt page Other Books You May Enjoy Index