Cloud Native Applications with Docker and Kubernetes : Design and Build Cloud Architecture and Applications with Microservices, EMQ, and Multi-Site Configurations

This book takes developers on a journey into the cloud with Docker and Kubernetes. It walks you through the basics of Docker containers, how they are built, run, and published, and how the Kubernetes system allows you to use containers to better manage a cloud native application. Additionally, it walks you through various issues in cloud architecture, and how to design a cloud architecture that will work with your application and your team. The book takes a unique approach, getting you immersed in each subject with tutorials, then building up your technical knowledge, and finally backing up and thinking about more big-picture issues. Part one introduces Docker, building and working with Docker images, and covering best practices for Docker Containers. Part two covers the practicalities of "cloud native” and managing a Kubernetes application, including a full working example. The last part covers the design of cloud and microservice architectures, including the use of enterprise message queues, multi-site configurations and the common values that such architectures follow. This approach accelerates learning and keeps you moving forward without leaving you behind. The appendices also contain a wealth of worthwhile reference material for routine cloud application management. What You Will Learn Understand Docker and containerization Gain insight into what Kubernetes is Master essential cloud architecture design principles Design and implement notes for building cloud architectures Who This Book Is For Primarily developers who are moving to the cloud and want to get a sense of the environment they are getting into, and developers who want to move into a larger role of cloud architecture. Table of Contents 4 About the Author 11 About the Technical Reviewer 13 Source Code 14 Acknowledgments 15 Chapter 1: Introduction 16 1.1 A Brief History of Web Service Hosting 16 1.1.1 The Old Way 16 1.1.2 The Virtual Private Server 17 1.1.3 From Virtual Private Servers to Containers 19 1.1.4 Cloud Native Infrastructure 20 1.2 An Overview of This Book 22 1.3 Prerequisites 22 1.4 Typographical Conventions 23 Part I: An Introduction to Containers 24 Chapter 2: Containers Under the Hood 25 2.1 Answering Basic Questions About Containers 26 2.1.1 What Are Containers? 26 2.1.2 What Problems Do Containers Solve? 26 2.1.3 Does It Use a Lot of Disk Space? 27 2.1.4 What Is the Relationship Between “Docker” and “Containers”? 28 2.2 A Short History of Container Technology 28 2.2.1 From Emulators to Virtual Machines 29 2.2.2 Increasing Isolation Inside the Operating System 30 2.2.3 The Birth of Containers 30 2.2.4 The Union Filesystem 31 2.2.5 The Rise of Docker 32 2.3 Summary 34 Chapter 3: A Docker Interactive Tutorial 35 3.1 Registries, Repositories, and Tags 35 3.2 Running Your First Container 37 3.3 Running a Docker Service 39 3.4 Running a Whole Operating System 41 3.5 Copying Files to and from the Container 41 3.6 Creating a New Docker Image 42 3.7 Creating Docker Images Using a Recipe 43 3.8 Pushing the Image to Docker Hub 44 3.9 Logging into a Running Docker Container 45 3.10 Summary 45 Chapter 4: Best Practices for Building Containers 46 4.1 How Not to Build a Container 47 4.1.1 Don’t Make a Container Perform Multiple Tasks 47 4.1.2 Don’t Include an Entire Operating System 48 4.2 Base Images 50 4.3 Alpine Distributions 51 4.4 Avoid Bloated Images from Deleted Files 52 4.5 Make Your Containers Configurable 53 4.6 Be Clear About Your Statefulness 54 4.7 Final Tips 55 4.8 Summary 57 Part II: Introducing Kubernetes 58 Chapter 5: The Cloud Native Philosophy 59 5.1 Cloud-Level Operating System 59 5.2 Declarative Infrastructure 60 5.3 Containers Are “Cattle” Not “Pets” 62 5.4 A Note on Costs 63 5.5 Summary 64 Chapter 6: Getting Started with Kubernetes 65 6.1 Setting Up Your First Cluster 65 6.2 Deploying Your First Application 67 6.3 Looking Around the Kubernetes Dashboard 70 6.4 Summary 72 Chapter 7: Managing Kubernetes with kubectl 73 7.1 Setting Up Your Connection 73 7.2 Basic kubectl Commands 74 7.3 Playing with kubectl 76 7.4 Creating Another Deployment with kubectl 77 7.5 Accessing Multiple Clusters 78 7.6 Summary 79 Chapter 8: An Overview of the Kubernetes Environment 80 8.1 Basic Kubernetes Components 80 8.1.1 The Control Plane 81 8.1.2 Nodes and Pods 82 8.1.3 Workloads 83 8.1.4 Kubernetes Services 83 8.1.5 CoreDNS 85 8.1.6 The Structure of a Pod 86 8.2 Kubernetes Storage 88 8.3 Configuring a Kubernetes Cluster 91 8.4 Application Interaction with Kubernetes 92 8.5 Summary 92 Chapter 9: Basic Kubernetes Management 94 9.1 Infrastructure as Code 94 9.2 A Short Introduction to YAML 96 9.3 Defining Kubernetes Objects in YAML Manifest Files 98 9.4 Kubernetes Files for Our Walkthrough Deployment 101 9.4.1 Organizing Your Kubernetes Files 103 9.5 Deleting Kubernetes Objects 105 9.6 Summary 106 Chapter 10: A Full Kubernetes Cloud Example 107 10.1 The Application Code 109 10.2 The Memcache Service 113 10.3 The Database Service 115 10.4 The Front-End Service 119 10.5 The Message Board API Server 122 10.6 Configurations and Secrets 126 10.7 Making an Ingress 129 10.8 Final Thoughts 132 10.9 Summary 134 Chapter 11: Going Further in Kubernetes 135 11.1 Cluster Namespaces 135 11.2 Helm 137 11.3 Capacity Management and Autoscaling 138 11.4 DaemonSets 142 11.5 Jobs 143 11.6 Cluster Security 145 11.7 Customizing Kubernetes 149 11.8 Additional Information 150 11.9 Summary 151 Part III: Architecting for the Cloud 152 Chapter 12: A Cloud Architecture Introduction 153 12.1 What Is Cloud Architecture? 153 12.2 Architectural Diagrams 154 12.3 Application Policies 156 12.4 Summary 158 Chapter 13: Basic Cloud Architectures 160 13.1 A Basic Load-Balanced Architecture 160 13.2 Adding Caching to the Architecture 161 13.3 An Ingress to Multiple Load-Balanced Systems 163 13.4 Summary 165 Chapter 14: A Basic Microservice Architecture 166 14.1 What Is a Microservice Architecture? 166 14.2 Authentication and Authorization 167 14.3 Choosing Where to Cut the Application 169 14.4 URL Path Layout 169 14.5 Quickly Transitioning to a Microservice Architecture 171 14.6 Summary 173 Chapter 15: Enterprise Message Queues 174 15.1 Basic Message Queue Components and Terminology 175 15.2 Working with Work Queues 176 15.3 Decoupling Systems with Publish/Subscribe 178 15.4 Event Bus Architecture 180 15.5 Message Streaming for Permanent Replayability 182 15.6 Two-Way Messaging 182 15.7 The Job of the Cloud Architect 184 15.8 Getting Started with Message Queueing 186 15.9 Summary 188 Chapter 16: Architecting Data Stores 190 16.1 Types of Data Stores 190 16.1.1 Relational Databases 191 16.1.2 Key/Value Stores 192 16.1.3 Document Databases 193 16.1.4 Full-Text Databases 193 16.1.5 Object/File Stores 194 16.1.6 Specialized Persistent Stores 194 16.2 Implementing Stores Using a Relational Database 194 16.2.1 Document Databases 195 16.2.2 Full-Text Search 195 16.2.3 Object/File Stores 197 16.2.4 Key/Value Stores 198 16.3 Database Topologies 198 16.4 Database Partitioning 200 16.5 Using UUIDs for Primary Keys 200 16.6 Thinking About Sources of Truth 202 16.7 Summary 203 Chapter 17: Extended Cloud Topologies 204 17.1 Terminology 204 17.2 How Kubernetes Handles Extended Topologies 206 17.3 Considerations for Multiregion Deployments 207 17.4 Summary 209 Chapter 18: Architecture Values 211 18.1 Scalability 211 18.2 Observability 213 18.3 Traceability 213 18.4 Testability 215 18.5 Predictability 216 18.6 Repeatability 217 18.7 Integrity and Security 218 18.8 Summary 221 Chapter 19: Conclusion 223 Appendix A: Navigating the Linux Command-Line Shell 224 A.1 General Command Format 224 A.2 Package Management Commands 225 A.2.1 Debian Package Management 226 A.2.2 Alpine Package Management 226 A.2.3 CentOS/Fedora/RHEL Package Management 227 A.3 Basic Linux File Commands 227 A.4 System Diagnostic Commands 229 A.5 Network Diagnostic Commands 230 A.6 Application-Level Diagnostic Commands 232 A.7 Permission-Based Commands 232 A.8 Basic Linux Filesystem Directories 233 Appendix B: Installing Applications 236 B.1 Installing Docker 236 B.2 Using Podman 237 B.3 Using Apple Silicon 238 B.4 Installing kubectl 239 B.5 Installing Helm 239 Appendix C: Common kubectl Commands 240 C.1 List Kubernetes Resources and What API Version Defines Them 240 C.2 Fire Up a Shell on a Pod 241 C.3 Run a Custom Image (and Shell) on a New Pod 242 C.4 Attaching to a Pod Container 242 C.5 Getting Logs for a Pod 243 C.6 Reboot All Pods in a Deployment 243 C.7 Forwarding a Local Port to a Port in the Cluster 243 C.8 Show Node Health 244 C.9 Show All Kubernetes Events 244 C.10 Show a Secret 244 C.11 Force-Terminate a Pod 244 C.12 Some Handy Switches to Remember 245 C.13 Accessing the Underlying Node 245 Appendix D: More on Kubernetes Storage 247 D.1 StorageClasses 247 D.2 File System Expansion 248 D.3 Preprovisioned Storage 249 D.4 StatefulSets 250 D.5 A Few Additional Details 253 Appendix E: More on Pod Scheduling 254 E.1 Readiness Gates 254 E.2 Pod Topology: Affinities and Anti-affinities 256 E.3 Node Tainting and Cordoning 256 Appendix F: Troubleshooting Kubernetes Clusters 259 F.1 Connecting to the Cluster 260 F.2 Pods Are Not Starting 261 F.3 Pods Not Behaving as Expected 262 F.4 Pods Can’t Reach Service 263 F.5 Ingress Is Not Getting an External IP Address 263 F.6 Going Forward 264 Index 265