Pro Google Kubernetes Engine : Network, Security, Monitoring, and Automation Configuration

Discover methodologies and best practices for getting started with Google Kubernetes Engine (GKE). This book helps you understand how GKE provides a fully managed environment to deploy and operate containerized applications on Google Cloud infrastructure. You will see how Kubernetes makes it easier for users to manage clusters and the container ecosystem. And you will get detailed guidance on deploying and managing applications, handling administration of container clusters, managing policies, and monitoring cluster resources. You will learn how to operate the GKE environment through the GUI-based Google Cloud console and the "gcloud" command line interface. The book starts with an introduction to GKE and associated services. The authors provide hands-on examples to set up Container Registry and GKE Cluster, and you will follow through an application deployment on GKE. Later chapters focus on securing your GCP GKE environment, GKE monitoring and dashboarding, and CI/CD automation. All of the code presented in the book is provided in the form of scripts, which allow you to try out the examples and extend them in interesting ways. What You Will Learn Understand the main container services in GCP (Google Container Registry, Google Kubernetes Engine, Kubernetes Engine, Management Services) Perform hands-on steps to deploy, secure, scale, monitor, and automate your containerized environment Deploy a sample microservices application on GKE Deploy monitoring for your GKE environment Use DevOps automation in the CI/CD pipeline and integrate it with GKE Who This Book Is For Architects, developers, and DevOps engineers who want to learn Google Kubernetes Engine Table of Contents 4 About the Authors 10 About the Technical Reviewers 12 Acknowledgments 13 Chapter 1: Introduction to GKE 15 Introduction to Docker 15 Introduction to Kubernetes 18 Master Node 19 Node (Worker) Component 20 Managing Kubernetes 21 Kubernetes Workload 21 Introduction to GCP 22 GCP Container Solutions for Container Ecosystems 23 Google Kubernetes Engine 26 Comparing EKS, AKS, and GKE 28 Amazon Elastic Kubernetes Service 28 Azure Kubernetes Service 28 Google Kubernetes Engine 29 GKE Architecture 30 Cluster Master 31 Cluster Master and the Kubernetes API 31 Nodes 32 Types of Node Machines 32 Container Registry 33 Network 34 Cloud Run 35 Key Features 35 Anthos 35 Components of Anthos 36 Anthos GKE 36 Anthos Config Management 36 Anthos Service Mesh 37 Summary 37 Chapter 2: Setting Up an Environment on GCP 38 Signing Up for Google Cloud 38 Setting Up an Environment for Google GKE 46 Setting Up the GCP CLI 46 GCP Cloud Shell 48 Creating a Project 48 Launching Cloud Shell 50 Creating a VM Instance in Compute Engine 53 Setting Up CLI on the Created VM 57 Configuring Google Cloud CLI on Local Machine 63 How to Get Project ID 64 Supporting Services for Containers 68 Summary 77 Chapter 3: Container Image Management Using Google Container Registry 78 Introduction to Google Container Registry 78 Setting Up Google Container Registry 80 Pushing a Docker Image into the GCP Container Registry 87 Managing and Securing the GCP Container Registry 94 Copying Images to a New Registry 94 Deleting Images from the Container Registry 100 Vulnerability Scanning an Image 101 Setting Up Vulnerability Scanning 102 Summary 109 Chapter 4: GKE Networking 110 Introduction to Google Kubernetes Engine (GKE) 111 Kubernetes on Google Cloud 111 GKE Network Terminologies 115 Network Namespace 115 Virtual Ethernet Devices 115 Bridges 115 Iptables 115 Services 116 Kube-proxy 116 HTTP(S) Load Balancer 117 NodePort 117 GKE IP Address Allocation 117 Communication Among Containers 119 Communication Between Pods on the Same Node 119 Communication Between PODs on Different Nodes 121 Communication of PODs with Services 123 Communication of Pods with the Internet (Outside World) 125 Load Balancing for Services 126 External Load Balancer 126 Internal Load Balancer (TCP/UDP) Load Balancer 127 HTTP(s) Load Balancer (at Layer 7) 127 Communication from an External Load Balancer to a Service 127 Internal Load Balancer 128 Communication from an Internal Load Balancer to a Service 129 HTTP(S) Load Balancer 130 Communication from an HTTP(S) Load Balancer to a Service 131 Deployment of the GKE Cluster 133 Cluster Basics 135 Node Pool 138 Node Metadata 145 Automation 146 Networking 148 Advance Networking Options 149 Security 151 Metadata 154 Features 154 Cluster Management Dashboard 156 Summary 157 Chapter 5: Deploying Containerized Applications with Google GKE 158 Introduction 159 Simple Application Architecture Overview 159 Advantages of Microservice Architecture 159 Introduction to the Sock Shop Microservice Application 160 Services 162 Application Deployment on GKE 164 complete-demo.yaml: A Brief Explanation 169 Deleting the Cluster 173 Summary 175 Chapter 6: GKE Security 176 Introduction 176 Google Cloud Shared Responsibility Model (GCSRM) 177 Infrastructure Security 180 Identity and Access Management 183 The GCP Console 183 GCP Console Features 184 Accessing the GCP Console 184 Cloud SDK 186 gcloud 186 kubectl 186 Install Cloud SDK 186 Cloud IAM Policies 190 Types of Cloud IAM Roles 191 Predefined GKE Role 191 Primitive Roles 195 Service Account User Role 196 Custom Roles 199 Role-Based Access Control 199 Audit Logging 202 Viewing the Admin Activity Log Through the Google Cloud Console 203 Network Security 207 Enabling Authorized Networks for the Kubernetes Master 207 Node Firewall 207 Enabling Network Policy for Pods’ Secure Communication 210 Enabling Network Policy Enforcement by the gcloud Command 211 Enabling Network Policy Through the Google Console 212 Enabling Network Policy Enforcement for an Existing Cluster Through the Google Console 214 Creating a Network Policy 216 Filtering Traffic Through a Load Balancer 219 Creating a Private Kubernetes Cluster in GKE 220 Compliance and Minimal Host OS 221 Auto-upgrade Components 222 Enable Automatic Upgrades Through the Cloud Console 222 Enabling Automatic Node Repair 226 Enabling Automatic Node Repair Through gcloud 227 Enabling Automatic Node Repair Through the Cloud Console 227 Encryption Keys and Secrets Encryption 229 Workload Identity 229 Defining Privileges and Access Control for Pods 229 Managed SSL Certificates 237 Application Development and Release Security 237 Development Phase 239 Build Phase 240 Development and Release Phase 242 Summary 244 Chapter 7: GKE Dashboarding Using Stackdriver (Google Operations) and Grafana 245 Introduction to Google Stackdriver (Google Operations) 245 Setting Up Google Stackdriver (Google Operations) with Grafana 246 Setting Up the Environment 246 Deploying and Validating Application Deployment on the GKE Cluster 249 Deploying Grafana 250 Connecting with the Grafana Installation 254 Data Source Configuration 258 Setting Up a Grafana Dashboard 264 Summary 267 Chapter 8: Monitoring GKE Using Sysdig 268 Introduction to the Sysdig Monitoring Solution 268 Container Application Monitoring 270 Open Port 6443 for Agent Ingress and Egress 276 Installing Sysdig Agent on GKE 282 Navigating GKE Monitoring Reports 290 Summary 309 Chapter 9: GKE Monitoring Using Prometheus 310 Introduction 310 Overview of Prometheus 311 Prometheus Architecture 311 Prometheus Server 312 Prometheus Pushgateway 313 Exporters 313 Alertmanager 313 Web-UI 313 Key Container Monitoring Features 314 Multidimensional Data Model 314 Accessible Format and Protocols 314 Service Discovery 314 Modular and Highly Available Components 314 Native Query Language Support 315 Support for Dashboarding and Reporting 315 Prometheus on Google Kubernetes Engine 315 Setting Up Prometheus on a Kubernetes Cluster 317 Installing and Setting Up Prometheus on GKE 318 Cloning Prometheus Code from GitHub 318 Creating the Namespace 321 Cluster Role YAML File 322 Cluster Role Section 322 apiVersion 322 kind 323 metadata 323 rules 323 Cluster Role Binding Section 324 apiVersion 324 kind 325 metadata 325 Creating a ConfigMap 327 Prometheus Deployment 331 Exporters 338 Node Exporter 338 Installation of Node Exporter in Prometheus Using a Helm Chart 340 Blackbox Exporter 346 Summary 351 Chapter 10: Automation of GKE Cluster, Application, and Monitoring Deployments 352 Introduction 353 Cleaning Up the GKE Environment Namespace 357 Environment Setup 358 Setting Up the Docker CE 362 Setting Up kubectl 368 Installing the Java Development Kit (JDK) 370 Installing Jenkins 371 Jenkins Slave Setup 381 Creating a Service for the Jenkins Slave 387 GKE Provisioning, Application Deployment, and Sysdig Agent, Using a Jenkins Pipeline 390 Deleting the GKE Cluster from the Jenkins Pipeline 409 Summary 416 Index 417