Docker in Practice, Second Edition

Docker is impossible to ignore. This lightweight container system is easier to deploy and more flexible than traditional VMs. Built for simplicity and speed, it radically reduces your reliance on manual system administration for tasks like configuring servers, creating disposable (and portable!) development environments, and predictably rolling out applications on unknown systems. While the idea behind Docker is simple, it can have a major impact on how you develop and deploy software. Docker in Practice is a hands-on guide to over 100 specific techniques you can use to get the most out of Docker in your daily work. Following a cookbook-style Problem/Solution/Discussion format, this practical handbook gives you instantly-useful solutions for important areas like effortless server maintenance and configuration, deploying microservices, creating safe environments for experimentation, and much more. As you read, you'll graduate from Docker basics into must-have practices like integrating Docker with your Continuous Integration process, automating complex container creation with Chef, and orchestration with Kubernetes. Docker in Practice, Second Edition contents preface acknowledgments about this book Roadmap About the code Book Forum about the cover illustration Part 1 Docker fundamentals 1 Discovering Docker 1.1 The what and why of Docker 1.1.1 What is Docker? 1.1.2 What is Docker good for? 1.1.3 Key concepts 1.2 Building a Docker application 1.2.1 Ways to create a new Docker image 1.2.2 Writing a Dockerfile 1.2.3 Building a Docker image 1.2.4 Running a Docker container 1.2.5 Docker layering 2 Understanding Docker: Inside the engine room 2.1 Docker?s architecture 2.2 The Docker daemon Technique 1 Open your Docker daemon to the world Technique 2 Running containers as daemons Technique 3 Moving Docker to a different partition 2.3 The Docker client Technique 4 Using socat to monitor Docker API traffic Technique 5 Using Docker in your browser Technique 6 Using ports to connect to containers Technique 7 Allowing container communication Technique 8 Linking containers for port isolation 2.4 Docker registries Technique 9 Setting up a local Docker registry 2.5 The Docker Hub Technique 10 Finding and running a Docker image Part 2 Docker and development 3 Using Docker as a lightweight virtual machine 3.1 From VM to container Technique 11 Converting your VM to a container Technique 12 A host-like container Technique 13 Splitting a system into microservice containers Technique 14 Managing the startup of your container?s services 3.2 Saving and restoring your work Technique 15 The ?save game? approach: Cheap source control Technique 16 Docker tagging Technique 17 Sharing images on the Docker Hub Technique 18 Referring to a specific image in builds 3.3 Environments as processes Technique 19 The ?save game? approach: Winning at 2048 4 Building images 4.1 Building images Technique 20 Injecting files into your image using ADD Technique 21 Rebuilding without the cache Technique 22 Busting the cache Technique 23 Intelligent cache-busting using build-args Technique 24 Intelligent cache-busting using the ADD directive Technique 25 Setting the right time zone in your containers Technique 26 Locale management Technique 27 Stepping through layers with the image-stepper Technique 28 Onbuild and golang 5 Running containers 5.1 Running containers Technique 29 Running GUIs within Docker Technique 30 Inspecting containers Technique 31 Cleanly killing containers Technique 32 Using Docker Machine to provision Docker hosts Technique 33 Wildcard DNS 5.2 Volumes?a persistent problem Technique 34 Docker volumes: Problems of persistence Technique 35 Distributed volumes with Resilio Sync Technique 36 Retaining your container?s bash history Technique 37 Data containers Technique 38 Remote volume mounting using SSHFS Technique 39 Sharing data over NFS Technique 40 Dev tools container 6 Day-to-day Docker 6.1 Staying ship-shape Technique 41 Running Docker without sudo Technique 42 Housekeeping containers Technique 43 Housekeeping volumes Technique 44 Detaching containers without stopping them Technique 45 Using Portainer to manage your Docker daemon Technique 46 Generating a dependency graph of your Docker images Technique 47 Direct action: Executing commands on your container Technique 48 Are you in a Docker container? 7 Configuration management: Getting your house in order 7.1 Configuration management and Dockerfiles Technique 49 Creating reliable bespoke tools with ENTRYPOINT Technique 50 Avoiding package drift by specifying versions Technique 51 Replacing text with perl -p -i -e Technique 52 Flattening images Technique 53 Managing foreign packages with Alien 7.2 Traditional configuration management tools with Docker Technique 54 Traditional: Using make with Docker Technique 55 Building images with Chef Solo 7.3 Small is beautiful Technique 56 Tricks for making an image smaller Technique 57 Tiny Docker images with BusyBox and Alpine Technique 58 The Go model of minimal containers Technique 59 Using inotifywait to slim containers Technique 60 Big can be beautiful Part 3 Docker and DevOps 8 Continuous integration: Speeding up your development pipeline 8.1 Docker Hub automated builds Technique 61 Using the Docker Hub workflow 8.2 More efficient builds Technique 62 Speeding up I/O-intensive builds with eatmydata Technique 63 Setting up a package cache for faster builds Technique 64 Headless Chrome in a container Technique 65 Running Selenium tests inside Docker 8.3 Containerizing your CI process Technique 66 Running the Jenkins master within a Docker container Technique 67 Containing a complex development environment Technique 68 Scaling your CI with Jenkins? Swarm plugin Technique 69 Upgrading your containerized Jenkins server safely 9 Continuous delivery: A perfect fit for Docker principles 9.1 Interacting with other teams in the CD pipeline Technique 70 The Docker contract: Reducing friction 9.2 Facilitating deployment of Docker images Technique 71 Manually mirroring registry images Technique 72 Delivering images over constrained connections Technique 73 Sharing Docker objects as TAR files 9.3 Configuring your images for environments Technique 74 Informing your containers with etcd 9.4 Upgrading running containers Technique 75 Using confd to enable zero-downtime switchovers 10 Network simulation: Realistic environment testing without the pain 10.1 Container communication: Beyond manual linking Technique 76 A simple Docker Compose cluster Technique 77 A SQLite server using Docker Compose 10.2 Using Docker to simulate real-world networking Technique 78 Simulating troublesome networks with Comcast Technique 79 Simulating troublesome networks with Blockade 10.3 Docker and virtual networks Technique 80 Creating another Docker virtual network Technique 81 Setting up a substrate network with Weave Part 4 Orchestration from a single machine to the cloud 11 A primer on container orchestration 11.1 Simple single-host Docker Technique 82 Managing your host?s containers with systemd Technique 83 Orchestrating the startup of your host?s containers 11.2 Manual multi-host Docker Technique 84 Manual multi-host Docker with Helios 11.3 Service discovery: What have we here? Technique 85 Using Consul to discover services Technique 86 Automatic service registration with Registrator 12 The data center as an OS with Docker 12.1 Multi-host Docker Technique 87 A seamless Docker cluster with swarm mode Technique 88 Using a Kubernetes cluster Technique 89 Accessing the Kubernetes API from within a pod Technique 90 Using OpenShift to run AWS APIs locally Technique 91 Building a framework on Mesos Technique 92 Micromanaging Mesos with Marathon 13 Docker platforms 13.1 Organizational choice factors 13.1.1 Time to market 13.1.2 Buy vs. build 13.1.3 Monolithic vs. piecemeal 13.1.4 Open source vs. licensed 13.1.5 Security stance 13.1.6 Consumer independence 13.1.7 Cloud strategy 13.1.8 Organizational structure 13.1.9 Multiple platforms? Organizational factors conclusion 13.2 Areas to consider when adopting Docker 13.2.1 Security and control 13.2.2 Building and shipping images 13.2.3 Running containers 13.3 Vendors, organizations, and products 13.3.1 The Cloud Native Computing Foundation (CNCF) 13.3.2 Docker, Inc. 13.3.3 Google 13.3.4 Microsoft 13.3.5 Amazon 13.3.6 Red Hat 13.1.10 Part 5 Docker in production 14 Docker and security 14.1 Docker access and what it means 14.1.1 Do you care? 14.2 Security measures in Docker Technique 93 Constraining capabilities Technique 94 A ?bad? Docker image to scan 14.3 Securing access to Docker Technique 95 HTTP auth on your Docker instance Technique 96 Securing your Docker API 14.4 Security from outside Docker Technique 97 Reducing a container?s attack surface with DockerSlim Technique 98 Removing secrets added during a build Technique 99 OpenShift: An application platform as a service Technique 100 Using security options 15 Plain sailing: Running Docker in production 15.1 Monitoring Technique 101 Logging your containers to the host?s syslog Technique 102 Logging your Docker logs output Technique 103 Monitoring containers with cAdvisor 15.2 Resource control Technique 104 Restricting the cores a container can execute on Technique 105 Giving important containers more CPU Technique 106 Limiting the memory usage of a container 15.3 Sysadmin use cases for Docker Technique 107 Using Docker to run cron jobs Technique 108 The ?save game? approach to backups 16 Docker in production: Dealing with challenges 16.1 Performance: You can?t ignore the tin Technique 109 Accessing host resources from the container Technique 110 Disabling the OOM killer 16.2 When containers leak?debugging Docker Technique 111 Debugging a container?s network with nsenter Technique 112 Using tcpflow to debug in flight without reconfiguring Technique 113 Debugging containers that fail on specific hosts Technique 114 Extracting a file from an image Appendix A Installing and using Docker The virtual machine approach Docker client connected to an external Docker server Native Docker client and virtual machine Docker on Windows Getting help Appendix B Docker configuration Configuring Docker Restarting Docker Restarting with systemctl Restarting with service Appendix C Vagrant Setting up GUIs Memory index Symbols A B C D E F G H I J K L M N O P Q R S T U V W X Y Z SummaryDocker in Practice, Second Edition presents over 100 practical techniques, hand-picked to help you get the most out of Docker. Following a Problem/Solution/Discussion format, you'll walk through specific examples that you can use immediately, and you'll get expert guidance on techniques that you can apply to a whole range of scenarios.Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.About the TechnologyDocker's simple idea-wrapping an application and its dependencies into a single deployable container-created a buzz in the software industry. Now, containers are essential to enterprise infrastructure, and Docker is the undisputed industry standard. So what do you do after you've mastered the basics? To really streamline your applications and transform your dev process, you need relevant examples and experts who can walk you through them. You need this book. About the BookDocker in Practice, Second Edition teaches you rock-solid, tested Docker techniques, such as replacing VMs, enabling microservices architecture, efficient network modeling, offline productivity, and establishing a container-driven continuous delivery process. Following a cookbook-style problem/solution format, you'll explore real-world use cases and learn how to apply the lessons to your own dev projects.What's insideContinuous integration and deliveryThe Kubernetes orchestration toolStreamlining your cloud workflowDocker in swarm modeEmerging best practices and techniquesAbout the ReaderWritten for developers and engineers using Docker in production.About the AuthorIan Miell and Aidan Hobson Sayers are seasoned infrastructure architects working in the UK. Together, they used Docker to transform DevOps at one of the UK's largest gaming companies.Table of ContentsPART 1 - DOCKER FUNDAMENTALSDiscovering DockerUnderstanding Docker: Inside the engine roomPART 2 - DOCKER AND DEVELOPMENTUsing Docker as a lightweight virtual machineBuilding imagesRunning containersDay-to-day DockerConfiguration management: Getting your house in orderPART 3 - DOCKER AND DEVOPSContinuous integration: Speeding up your development pipeline Continuous delivery: A perfect fit for Docker principlesNetwork simulation: Realistic environment testing without the painPART 4 - ORCHESTRATION FROM A SINGLE MACHINE TO THE CLOUDA primer on container orchestrationThe data center as an OS with DockerDocker platformsPART 5 - DOCKER IN PRODUCTIONDocker and securityPlain sailing: Running Docker in productionDocker in production: Dealing with challenges