CI/CD Pipeline Using Jenkins Unleashed : Solutions While Setting Up CI/CD Processes

Table of Contents About the Author About the Technical Reviewer Acknowledgments Introduction Chapter 1: Understanding CI/CD The Development Workflow Running Unit Tests Locally Pushing and Merging Code to the Central Repository Compiling Code after the Merge Running Tests on the Merged Code Deploying an Artifact Continuous Delivery/Continuous Deployment CI/CD Workflow Case Study Pulling the Application’s Latest Code Developing and Running Unit Test Cases Code Development Rerunning the Unit Test Case Pushing and Merging the Code to the Central Repository Compiling the Code after the Merge Running Tests on the Merged Code Deploying an Artifact Run e-e Tests on a Deployed Application Summary Chapter 2: Introducing Jenkins What Is Jenkins? A History of Jenkins Implementing CI/CD Using Jenkins The Jenkins Architecture Summary Chapter 3: Installing Jenkins Installing Jenkins on Windows Hardware/Software Requirements Different Ways to Install Jenkins on Windows Understanding the Configuration Files and Directory Structure of Jenkins Understanding Important Settings in Jenkins.xml Summary Chapter 4: Configuring Jenkins Configuring Global Settings and Paths Signing into Jenkins Starting the Jenkins Server Starting the Jenkins Service on Linux Opening the Browser and Signing In Understanding the Configure System Options Resetting the Username and Password in Jenkins Adding a New User Summary Chapter 5: Managing Plugins in Jenkins What Are Plugins Commonly Used Jenkins Plugins Installing the Plugins in Jenkins Understanding the Plugin Manager The Updates Tab The Available Tab The Installed Tab The Advanced Tab Troubleshooting Installation Problems Problem 1 Problem 2 Summary Chapter 6: Understanding the Global Tool Configuration Page Global Tool Configuration Settings Understanding the Global Tool Configuration Settings Maven Configuration Java Configuration Summary Chapter 7: Managing Security with Jenkins Configuring Global Security in Jenkins Configuring LDAP with Jenkins Why We Need to Configure LDAP with Jenkins How to Configure LDAP with Jenkins Summary Chapter 8: Managing Credentials Understanding Credentials in Jenkins Creating Credential Entries in Jenkins Understanding Scope and Domains Creating Credential Entries in Jenkins Creating a Credentials Entry in a Global Domain (Default Domain) and a Global Scope Updating a Credentials Entry Creating a Credentials Entry in a Particular Domain Configuring a Credentials Provider Summary Chapter 9: Managing Users Creating Users in Jenkins Assigning Roles to Users in Jenkins Installing the Role-Based Authorization Strategy Plugin Enabling Role-Based Strategy in Jenkins Creating User Roles in Jenkins Assigning Roles to Users in Jenkins Checking the Assignment of a Role to a User Creating Project-Based Roles in Jenkins Assigning Project-Based Roles to Users Verifying the Assignment of the Project-Based Role to the User Understanding Matrix-Based Security in Jenkins Understanding the Project-Based Matrix Authorization Strategy in Jenkins Summary Chapter 10: Understanding Jobs in Jenkins What Is a Job in Jenkins? What Is a Build in Jenkins? What Is a Free-Style Job in Jenkins? How to Create a Job in Jenkins How to Configure a Job in Jenkins This Project Is Parameterized Disabling a Project Executing Concurrent Builds Quiet Period Block Build When Upstream Project Is Building Block Build When Downstream Project Is Building Use Custom Workspace Display Name Keep the Build Logs of Dependencies Source Code Management Branches to Build Build Triggers Trigger Build Remotely (e.g. from Scripts) Build After Other Projects Are Built Build Periodically Poll SCM Build Step Post-Build Actions How to Run a Job in Jenkins and Check Its Output How to Edit a Job in Jenkins How to View a Job’s Workspace How to Clear a Job’s Workspace How to Delete a Job Summary Chapter 11: Preparing a Java API Project Using Maven Understanding the Maven Build Tool Typical Development Flow of a Java API Project Downloading Third-Party Libraries Adding Downloaded Libraries to the Project Build Path Coding and Writing Unit Test Cases Compiling the Application and Unit Test Cases Code Running Unit Test Cases Bundling/Packaging the Application Releasing it on the Artifact Repository How the Build Tool Helps Developers How to Use the Maven Build Tool to Build a Java API Project Step 1: Creating a Workspace in Eclipse Step 2: Creating a Maven Project Understanding the Maven Project Directory Structure src/main/java src/test/java Understanding Java API Project Code Files Understanding the pom.xml File in the Java API Project Build Lifecycle Phases and Their Order How to Use Maven from the CLI Setting Up Maven Using Maven CLI Commands Understanding Maven’s settings.xml File Summary Chapter 12: Integrating Maven with the Nexus Repository and Creating Free-Style Jobs to Release the Java API on the Nexus Repository Understanding Git Installing Git Understanding GitHub/GitLab Understanding End-End Use of Git for the API Project Step 1: Creating a Local Repository Step 2: Creating a Central Repository on GitLab Step 3: Committing Code to the Local Repository Step 4: Pushing the Code from the Local Repository to the Central Repository on GitLab Step 5: Creating a Master Branch in the Central Repository on GitLab Understanding the Nexus Repository What is an Artifact? What is the Nexus Repository? Installing the Nexus Repository How to Start the Nexus Repository System Installing Nexus as a Service Accessing the Nexus Repository Manager Creating a Hosted Repository to Release the Artifacts Integrating Maven and the Nexus Repository Releasing the CalculatorAPI.jar File in the Nexus Repository Creating a Jenkins Free-style Job to Release the CalculatorAPI.jar in the Nexus Repository Step 1: Setting Up Maven in Jenkins Step 2: Adding Git Repository Credentials to Jenkins Step 3: Creating a Free-Style Job from the Jenkins Dashboard Step 4: Add a Subtraction Function and Unit Test Cases to the API Project Step 5: Commit Changes in the Local Repository and Push them to the Central GitLab Repository Step 6: Merge the SubtractionFunction Branch with the Master Branch on the Central GitLab Repository Running a Jenkins Free-Style Job to Perform a Release of the CalculatorAPI.jar in the Nexus Repository Summary Chapter 13: Creating an Auto-Trigger Free-Style Job to Manage Java API Releases How to Add a New Code Contributor to a Private GitLab Repository How to Invite a Team Member to the Code Repository Understanding SSH Authentication Why You Need SSH Authentication How SSH Authentication Works with GitLab Applying SSH Authentication to the Java API Code Repository Step 1: Generating the Public and Private Keys Step 2: Adding the Public Key to the JenkinsBookCalculatorAPI GitLab Repository Accessing the JenkinsBookCalculatorAPI Repository Using SSH URL and Adding new Arithmetic Functions to the CalculatorAPI Step 1: Cloning the JenkinsBookCalculatorAPI Repository Step 2: Adding a Multiplication Function to the Calculator Class Step 3: Adding Unit Test Cases for the Multiplication Function Step 4: Changing the Version Number to 3.0 in pom.xml Step 5: Unit Testing the Recent Multiplication Function and Regression Testing for the New Functionalities Step 6: Committing New Functionality Changes in a Branch Named Multiplication Function in the Local Repository Step 7: Pushing the MultiplicationFunction Branch to the Remote Repository Step 8: Creating a Merge Request for this New Branch Creating an Auto-Trigger Jenkins Job with Email Notification Step 1: Creating a Free-Style Job from Jenkins Dashboard: Click the New Item link on Jenkins Dashboard Setting Up Jenkins to Send Email Notifications Triggering the New Jenkins Job Understanding the Execution of Jobs with SCM Set Failing the Build to See the Email Notification Summary Chapter 14: Understanding the Jenkins Pipeline What Is a Jenkins Pipeline? Why Use a Jenkins Pipeline? Understanding Different Pipeline Concepts Pipeline Syntax Overview Fundamentals of Declarative Pipelines Fundamentals of Scripted Pipelines Defining a Simple Pipeline in Jenkins UI Pipeline Speed/Durability Override Resolving java.lang.IllegalArgumentException: Unsupported Class File Major Version Error Understanding String Interpolation in the in Jenkins Pipeline String Interpolation Example Creating a Pipeline Job to Release the Java API Running a Pipeline Job and Release the Calculator API Summary Chapter 15: Creating Jenkins Jobs to Manage a Web Application Project Understanding the Calculator Web Application Source Code Building the Calculator Web Application Deploying the Calculator Web Application Accessing the Calculator Web Application Understanding E-E Testing the Calculator Web Application Using the Selenium WebDriver What Is UI Automation Testing How UI Automation Tools Work What Is a Selenium WebDriver Understanding the Selenium Tests Written in Python EnvVars.csv CalculatorPage.csv AutomationFramework Package CalculatorWebApp_Pages Package CalculatorWebApp_TestCases Package Software Setup to Run these Test Cases Running Selenium Python Tests Step 1: Open the Command Prompt and go to the TestCalculatorWebApplicationUsing SeleniumPython Project Directory Step 2: Create a Virtual Python Environment and Activate It Step 3: Install the Required Python Packages Pushing the WebApplication and Automation Project to the GitHub Repository Step 1: Sign up on Github.com Step 2: Creating a New Repository Step 3: Creating a New SSH Key Pair for the GitHub Repository Step 4: Include the SSH Private Key File Path in the $(user.name)\.ssh\config File Step 5: Adding an SSH Public Key to the GitHub Repository Step 6: Committing the Calculator Web Application to the Master Branch in the Local Repository Step 7: Pushing the Master Branch to the Remote Repository Pushing the Selenium Python Automation Project to GitHub Creating Parameterized Auto-Trigger Free-Style Jenkins Jobs Step 1: Installing the Parameterized Trigger Plugin Step 2: Creating a BuildAndDeployCalculator WebApplication Job Step 3: Creating the TestCalculatorWeb Application Job Configuring an Extended Email Notification Step 1: Installing the Email Extension Plugin Step 2: Configuring Extended Email Extension settings Step 3: Adding an Email Notification Step in the Post-Build Section of the TestCalculatorWeb Application Job Running a Parameterized Free-Style Job Manually Auto-Triggering a Parameterized Free-Style Job Creating a Parameterized Pipeline Job Creating a Build and Deploying the Calculator Web Application Parameterized Pipeline Job Creating a TestCalculatorWebApplication Pipeline Job Running the Parameterized Pipeline Job Manually Automatically Triggering the Parameterized Pipeline Job Summary Chapter 16: Understanding Pipeline as Code What Is API Authentication How to Apply API Authentication to GitHub Repositories How to Use an API Token to Access a Private GitHub Repository Creating an API Access Token in GitLab How to Use the API Token to Access a Private GitLab Repository How to Access a GitHub/GitLab Repository Using an API Token in Jenkins Step 1: Setting Up Maven in Jenkins Step 2: Creating a Free-Style Job from the Jenkins Dashboard Step 3: Change the Version in pom.xml of the Cloned API Project Step 4: Commit Changes in the Local Repository and Push them to the Central GitLab Repository Running Free-Style Jobs Accessing the GitLab Repository with the API Token Understanding Pipeline as Code (Jenkinsfile) What Is a Jenkinsfile Advantages of Pipeline as Code Writing a Pipeline in a Jenkinsfile Scripted vs Declarative Syntax Creating a Jenkins Job to Run a Jenkinsfile Saving the Pipeline Code in a Jenkinsfile and Pushing it to the GitLab Repository Triggering a Jenkins Job Using a GitLab Webhook Creating a Pipeline Job to Trigger Using GitHub Webhooks Triggering Pipeline Jobs Using GitHub Webhook Summary Chapter 17: Jenkins Distributed Builds Jenkins Distributed Architecture Ways to Connect the Master and Slaves Understanding the Configuration to Connect the Master to the Agent Using SSH Step 1: Install the SSH Build Agent's Plugin Step 2: Install Java on the Slave Node Step 3: Create an SSH Public-Private Key Pair Step 4: Add a Credentials Entry with a Private Key to the Master Machine Step 5: Add a Node Entry to the Jenkins Master Step 6: Add a Public Key to the authorized_keys File on the Slave Machine Step 7: Change the Permissions of the authorized_keys File on the Slave Machine Step 8: Restart the sshd Service Step 9: Check the Connection to the Agent Machine from the Master Step 10: Launch the New Node from the Master Machine Creating a Free-Style Job to Run on the Node1 Agent Running the New Free-Style Job on the Node1 Agent Understanding the Configuration to Connect the Agent to the Master Using JNLP Step 1: Configure the Jenkins Master to Receive JNLP Agent Connection Requests Step 2: Set the Jenkins URL Step 3: Create a New Node Entry from the Agent (Slave) Machine Creating a Job to Run on the JNLP Node Summary Chapter 18: Integrating Jenkins with AWS Understanding an EC2 Instance on AWS Creating an EC2 Instance on AWS Step 1: Sign Up on AWS Step 2: Sign in to AWS Step 3: Create an EC2 Instance Step 4: Start an EC2 Instance Configuring the EC2 Instance to Deploy the Calculator Web Application Step 1: Install the IIS Web Server Step 2: Configure the IIS Web Server Step 3: Configure the SSH Connection Between the Local Computer and the EC2 Instance Copying the Calculator.html File to the EC2 Instance Accessing Calculator.html Copied to the EC2 Instance from a Browser on a Local Computer Creating a Jenkins Job to Deploy the Calculator Web Application on an EC2 instance Changing the Calculator Application URL in Selenium Framework Running the BuildAndDeployCalculator WebApplication OnEC2Instance Job Summary Chapter 19: Miscellaneous Topics Part 1 Understanding the Jenkins CLI How to Interact with Jenkins Using its CLI How to Create a Job Using the Jenkins CLI Authenticating Users Using Basic Authentication (Username-Password/API Token) Authenticating Users Through SSH While Using CLI Commands Configuring an SSH Server in Jenkins Configuring an Inbound Rule in a Firewall How to Build Jobs with the Jenkins CLI Using SSH Authentication Using the OpenSSH Client to Run Jenkins CLI Commands Using the jenkins-cli.jar Client to Run Jenkins CLI Commands with SSH How to Export All Jobs How to Import All Jobs Summary Chapter 20: Miscellaneous Topics Part 2 Understanding Jenkins Remote Access API Using Jenkins Remote Access API Getting the Configuration of Existing Jenkins Jobs Using Jenkins Remote API Creating New Jenkins Jobs Using Jenkins Remote API Triggering Parameterized Jenkins Job Using Jenkins Remote API Triggering Normal (Non-Parameterized) Jenkins Job Using Jenkins Remote API Working with the Jenkins Server Using Python-Jenkins Using Python-Jenkins Package Libraries How to Use Shared Libraries in the Jenkins Pipeline Step 1: Creating a Shared Library in the .Groovy File Step 2: Push the Created Shared Library File to the GitLab Repository Step 3: Configure the Shared Library in Jenkins Step 4: Create a Pipeline Job to Use the Shared Library Step 5: Running the Pipeline Job Summary Index Understand continuous integration (CI), continuous delivery, and continuous deployment (CD) with Jenkins. These processes allow users as well as administrators to catch problems as soon as they get injected into software systems. This book starts with an introduction to Jenkins and covers its architecture and role in CI/CD. The basics are covered, including installing and configuring Jenkins. Tool configuration and plugins are discussed as well as available security measures such as credentials. You will learn what is meant by Job in Jenkins, its types, sections, and much more. You will look at Java API: projects, jobs, configuration. The concluding chapters take you through creating pipelines, their role in managing web apps, and distributed pipelines. The book also covers unit testing using TestNG as well as end-to-end testing using Selenium Python as a part of building a life cycle and setting up Jenkins on different physical and Docker environments as well as Jenkins integration with cloud environments such as AWS. And you will learn how to create reusable libraries for use in Jenkins Pipeline and control Jenkins servers using Jenkins CLI and REST APIs. The new Jenkins Blue Ocean also is covered. The book helps you understand CI/CD implementation using Jenkins from scratch in your projects and prepare for end-to-end DevOps practices. What You Will Learn Apply Jenkins to create end-to-end pipelines Integrate Jenkins with AWS, Docker, Git, and many more tools Use Selenium automation for end-to-end testing Create distributed pipelines Who Is This Book For Developers and test automation professionals who are involved in creating CI/CD pipelines as well as prospective DevOps aspirants who want to make their way ahead as professionals