Docker in Practice, Second Edition

Docker is impossible to ignore. This lightweight container system is easier to deploy and more flexible than traditional VMs. Built for simplicity and speed, it radically reduces your reliance on manual system administration for tasks like configuring servers, creating disposable (and portable!) development environments, and predictably rolling out applications on unknown systems. While the idea behind Docker is simple, it can have a major impact on how you develop and deploy software. Docker in Practice is a hands-on guide to over 100 specific techniques you can use to get the most out of Docker in your daily work. Following a cookbook-style Problem/Solution/Discussion format, this practical handbook gives you instantly-useful solutions for important areas like effortless server maintenance and configuration, deploying microservices, creating safe environments for experimentation, and much more. As you read, you'll graduate from Docker basics into must-have practices like integrating Docker with your Continuous Integration process, automating complex container creation with Chef, and orchestration with Kubernetes. Docker in Practice, Second Edition......Page 1 contents......Page 6 preface......Page 14 acknowledgments......Page 16 about this book......Page 17 Roadmap......Page 18 Book Forum......Page 19 about the cover illustration......Page 20 Part 1 Docker fundamentals......Page 22 1 Discovering Docker......Page 24 1.1 The what and why of Docker......Page 25 1.1.1 What is Docker?......Page 26 1.1.2 What is Docker good for?......Page 28 1.1.3 Key concepts......Page 29 1.2 Building a Docker application......Page 31 1.2.1 Ways to create a new Docker image......Page 32 1.2.2 Writing a Dockerfile......Page 33 1.2.3 Building a Docker image......Page 34 1.2.4 Running a Docker container......Page 35 1.2.5 Docker layering......Page 37 2 Understanding Docker: Inside the engine room......Page 40 2.1 Docker?s architecture......Page 41 2.2 The Docker daemon......Page 42 Technique 1 Open your Docker daemon to the world......Page 43 Technique 2 Running containers as daemons......Page 45 Technique 3 Moving Docker to a different partition......Page 48 Technique 4 Using socat to monitor Docker API traffic......Page 49 Technique 5 Using Docker in your browser......Page 53 Technique 6 Using ports to connect to containers......Page 55 Technique 7 Allowing container communication......Page 56 Technique 8 Linking containers for port isolation......Page 58 2.4 Docker registries......Page 60 Technique 9 Setting up a local Docker registry......Page 61 2.5 The Docker Hub......Page 62 Technique 10 Finding and running a Docker image......Page 63 Part 2 Docker and development......Page 66 3 Using Docker as a lightweight virtual machine......Page 68 Technique 11 Converting your VM to a container......Page 69 Technique 12 A host-like container......Page 72 Technique 13 Splitting a system into microservice containers......Page 75 Technique 14 Managing the startup of your container?s services......Page 78 3.2 Saving and restoring your work......Page 80 Technique 15 The ?save game? approach: Cheap source control......Page 81 Technique 16 Docker tagging......Page 83 Technique 17 Sharing images on the Docker Hub......Page 86 Technique 18 Referring to a specific image in builds......Page 88 Technique 19 The ?save game? approach: Winning at 2048......Page 89 4.1 Building images......Page 92 Technique 20 Injecting files into your image using ADD......Page 93 Technique 21 Rebuilding without the cache......Page 96 Technique 22 Busting the cache......Page 97 Technique 23 Intelligent cache-busting using build-args......Page 99 Technique 24 Intelligent cache-busting using the ADD directive......Page 102 Technique 25 Setting the right time zone in your containers......Page 105 Technique 26 Locale management......Page 107 Technique 27 Stepping through layers with the image-stepper......Page 111 Technique 28 Onbuild and golang......Page 115 5.1 Running containers......Page 118 Technique 29 Running GUIs within Docker......Page 119 Technique 30 Inspecting containers......Page 121 Technique 31 Cleanly killing containers......Page 122 Technique 32 Using Docker Machine to provision Docker hosts......Page 124 Technique 33 Wildcard DNS......Page 128 Technique 34 Docker volumes: Problems of persistence......Page 129 Technique 35 Distributed volumes with Resilio Sync......Page 131 Technique 36 Retaining your container?s bash history......Page 133 Technique 37 Data containers......Page 135 Technique 38 Remote volume mounting using SSHFS......Page 137 Technique 39 Sharing data over NFS......Page 140 Technique 40 Dev tools container......Page 142 6.1 Staying ship-shape......Page 145 Technique 42 Housekeeping containers......Page 146 Technique 43 Housekeeping volumes......Page 148 Technique 44 Detaching containers without stopping them......Page 150 Technique 45 Using Portainer to manage your Docker daemon......Page 151 Technique 46 Generating a dependency graph of your Docker images......Page 152 Technique 47 Direct action: Executing commands on your container......Page 154 Technique 48 Are you in a Docker container?......Page 155 7 Configuration management: Getting your house in order......Page 158 Technique 49 Creating reliable bespoke tools with ENTRYPOINT......Page 159 Technique 50 Avoiding package drift by specifying versions......Page 161 Technique 51 Replacing text with perl -p -i -e......Page 163 Technique 52 Flattening images......Page 164 Technique 53 Managing foreign packages with Alien......Page 166 Technique 54 Traditional: Using make with Docker......Page 169 Technique 55 Building images with Chef Solo......Page 172 Technique 56 Tricks for making an image smaller......Page 176 Technique 57 Tiny Docker images with BusyBox and Alpine......Page 178 Technique 58 The Go model of minimal containers......Page 180 Technique 59 Using inotifywait to slim containers......Page 183 Technique 60 Big can be beautiful......Page 185 Part 3 Docker and DevOps......Page 188 8 Continuous integration: Speeding up your development pipeline......Page 190 Technique 61 Using the Docker Hub workflow......Page 191 Technique 62 Speeding up I/O-intensive builds with eatmydata......Page 194 Technique 63 Setting up a package cache for faster builds......Page 196 Technique 64 Headless Chrome in a container......Page 199 Technique 65 Running Selenium tests inside Docker......Page 201 Technique 66 Running the Jenkins master within a Docker container......Page 206 Technique 67 Containing a complex development environment......Page 209 Technique 68 Scaling your CI with Jenkins? Swarm plugin......Page 213 Technique 69 Upgrading your containerized Jenkins server safely......Page 216 9 Continuous delivery: A perfect fit for Docker principles......Page 221 Technique 70 The Docker contract: Reducing friction......Page 222 Technique 71 Manually mirroring registry images......Page 225 Technique 72 Delivering images over constrained connections......Page 227 Technique 73 Sharing Docker objects as TAR files......Page 229 Technique 74 Informing your containers with etcd......Page 231 Technique 75 Using confd to enable zero-downtime switchovers......Page 235 10 Network simulation: Realistic environment testing without the pain......Page 241 Technique 76 A simple Docker Compose cluster......Page 242 Technique 77 A SQLite server using Docker Compose......Page 246 10.2 Using Docker to simulate real-world networking......Page 250 Technique 78 Simulating troublesome networks with Comcast......Page 251 Technique 79 Simulating troublesome networks with Blockade......Page 254 10.3 Docker and virtual networks......Page 258 Technique 80 Creating another Docker virtual network......Page 259 Technique 81 Setting up a substrate network with Weave......Page 262 Part 4 Orchestration from a single machine to the cloud......Page 268 11 A primer on container orchestration......Page 270 Technique 82 Managing your host?s containers with systemd......Page 272 Technique 83 Orchestrating the startup of your host?s containers......Page 275 Technique 84 Manual multi-host Docker with Helios......Page 278 11.3 Service discovery: What have we here?......Page 284 Technique 85 Using Consul to discover services......Page 285 Technique 86 Automatic service registration with Registrator......Page 292 12 The data center as an OS with Docker......Page 296 Technique 87 A seamless Docker cluster with swarm mode......Page 297 Technique 88 Using a Kubernetes cluster......Page 301 Technique 89 Accessing the Kubernetes API from within a pod......Page 306 Technique 90 Using OpenShift to run AWS APIs locally......Page 310 Technique 91 Building a framework on Mesos......Page 315 Technique 92 Micromanaging Mesos with Marathon......Page 323 13 Docker platforms......Page 327 13.1 Organizational choice factors......Page 328 13.1.2 Buy vs. build......Page 331 13.1.3 Monolithic vs. piecemeal......Page 332 13.1.5 Security stance......Page 333 13.1.8 Organizational structure......Page 334 13.1.10......Page 335 13.2.1 Security and control......Page 336 13.2.2 Building and shipping images......Page 342 13.2.3 Running containers......Page 345 13.3.1 The Cloud Native Computing Foundation (CNCF)......Page 348 13.3.2 Docker, Inc.......Page 349 13.3.4 Microsoft......Page 350 13.3.6 Red Hat......Page 351 Part 5 Docker in production......Page 354 14 Docker and security......Page 356 14.1.1 Do you care?......Page 357 Technique 93 Constraining capabilities......Page 358 Technique 94 A ?bad? Docker image to scan......Page 362 Technique 95 HTTP auth on your Docker instance......Page 364 Technique 96 Securing your Docker API......Page 367 Technique 97 Reducing a container?s attack surface with DockerSlim......Page 371 Technique 98 Removing secrets added during a build......Page 377 Technique 99 OpenShift: An application platform as a service......Page 381 Technique 100 Using security options......Page 389 15 Plain sailing: Running Docker in production......Page 397 Technique 101 Logging your containers to the host?s syslog......Page 398 Technique 102 Logging your Docker logs output......Page 401 Technique 103 Monitoring containers with cAdvisor......Page 403 Technique 104 Restricting the cores a container can execute on......Page 404 Technique 105 Giving important containers more CPU......Page 405 Technique 106 Limiting the memory usage of a container......Page 407 Technique 107 Using Docker to run cron jobs......Page 409 Technique 108 The ?save game? approach to backups......Page 412 16 Docker in production: Dealing with challenges......Page 415 Technique 109 Accessing host resources from the container......Page 416 Technique 110 Disabling the OOM killer......Page 420 Technique 111 Debugging a container?s network with nsenter......Page 422 Technique 112 Using tcpflow to debug in flight without reconfiguring......Page 425 Technique 113 Debugging containers that fail on specific hosts......Page 426 Technique 114 Extracting a file from an image......Page 430 Appendix A Installing and using Docker......Page 434 Native Docker client and virtual machine......Page 435 Docker on Windows......Page 436 Getting help......Page 438 Configuring Docker......Page 439 Restarting with systemctl......Page 440 Restarting with service......Page 441 GUIs......Page 442 Memory......Page 443 B......Page 444 C......Page 445 D......Page 447 G......Page 448 I......Page 449 M......Page 450 P......Page 451 S......Page 452 W......Page 454 Z......Page 455 SummaryDocker in Practice, Second Edition presents over 100 practical techniques, hand-picked to help you get the most out of Docker. Following a Problem/Solution/Discussion format, you'll walk through specific examples that you can use immediately, and you'll get expert guidance on techniques that you can apply to a whole range of scenarios.Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.About the TechnologyDocker's simple idea-wrapping an application and its dependencies into a single deployable container-created a buzz in the software industry. Now, containers are essential to enterprise infrastructure, and Docker is the undisputed industry standard. So what do you do after you've mastered the basics? To really streamline your applications and transform your dev process, you need relevant examples and experts who can walk you through them. You need this book. About the BookDocker in Practice, Second Edition teaches you rock-solid, tested Docker techniques, such as replacing VMs, enabling microservices architecture, efficient network modeling, offline productivity, and establishing a container-driven continuous delivery process. Following a cookbook-style problem/solution format, you'll explore real-world use cases and learn how to apply the lessons to your own dev projects.What's insideContinuous integration and deliveryThe Kubernetes orchestration toolStreamlining your cloud workflowDocker in swarm modeEmerging best practices and techniquesAbout the ReaderWritten for developers and engineers using Docker in production.About the AuthorIan Miell and Aidan Hobson Sayers are seasoned infrastructure architects working in the UK. Together, they used Docker to transform DevOps at one of the UK's largest gaming companies.Table of ContentsPART 1 - DOCKER FUNDAMENTALSDiscovering DockerUnderstanding Docker: Inside the engine roomPART 2 - DOCKER AND DEVELOPMENTUsing Docker as a lightweight virtual machineBuilding imagesRunning containersDay-to-day DockerConfiguration management: Getting your house in orderPART 3 - DOCKER AND DEVOPSContinuous integration: Speeding up your development pipeline Continuous delivery: A perfect fit for Docker principlesNetwork simulation: Realistic environment testing without the painPART 4 - ORCHESTRATION FROM A SINGLE MACHINE TO THE CLOUDA primer on container orchestrationThe data center as an OS with DockerDocker platformsPART 5 - DOCKER IN PRODUCTIONDocker and securityPlain sailing: Running Docker in productionDocker in production: Dealing with challenges