DevOps for SharePoint : With Packer, Terraform, Ansible, and Vagrant

Deploy a SharePoint farm in a repeatable, predictable, and reliable fashion using Infrastructure as Code (IaC) techniques to automate provisioning. Savvy IT pros will learn how to use DevOps practices and open source tools to greatly reduce costs, and streamline management operations for SharePoint farms deployed via Amazon Web Services (AWS), Azure, or on premise. __DevOps for SharePoint__ will help you navigate the complex challenges of deploying and managing SharePoint Server farms. You will learn how to reduce time-consuming tasks and errors when generating development, testing, or production environments. And you will benefit from learning proven methods to apply Microsoft updates with minimal downtime and productivity loss. Whether you are a SharePoint architect, IT pro, or developer helping customers with the SharePoint platform, this book will teach you the most useful DevOps practices to tackle those issues and broaden your skill set. **What You’ll Learn** * Understand the basics of the most popular open source tools—Vagrant, Packer, Terraform, and Ansible—and how to use them in the context of deploying and scaling a SharePoint farm * Use Vagrant to build SharePoint development environments in less than an hour, and add automated testing * Use Packer to create a “golden image” with preconfigured settings, and then use it as the base image in your Terraform configuration for both AWS and Azure farms * Use Terraform to scale your SharePoint farm topology * Use Red Hat’s Ansible Playbooks to perform configuration management on your farm * Use Terraform to deploy immutable infrastructure environments using IaC (Infrastructure as Code) * Use InSpec 2.0 to stay in compliance by testing your cloud infrastructure * Use Ansible to apply Microsoft updates and patches **Who This Book Is For** IT pros and developers who are looking to expand their knowledge and take a modern approach by using open source technologies to work with Microsoft products. Experience installing SharePoint, and a basic understanding of either Azure or AWS, is helpful. Table of Contents 5 About the Authors 9 About the Technical Reviewers 10 Acknowledgments 11 Chapter 1: Introduction 12 What Is DevOps? 12 DevOps Core Practices 14 Continuous Integration 15 Continuous Delivery 15 Continuous Deployment 16 DevOps Core Values 16 Culture 16 Automation 16 Measurement 16 Sharing 17 DevOps Tools of the Trade 17 Infrastructure as Code 17 Introducing Terraform 18 Idempotency 20 Configuration Management 20 Virtualization 22 Machine Imaging (Prebaked Images) 26 Introducing Packer 26 Benefits of DevOps Practices 27 Why DevOps Practices Matter When It Comes to SharePoint 28 Applying DevOps Practices 29 Use Infrastructure as Code to Provision SharePoint Dev and Test Environments 29 Continuous Integration/Continuous Deployment 30 Configuration Management 30 Summary 31 Chapter 2: Getting up and Running: Set up Your Environment 32 Our Environment 32 Our Project Workflow 33 Install an Open Source Toolchain 34 Install Git 34 Install posh-git Using Chocolatey on Windows 34 Install Terraform 35 Install Using Homebrew on Mac OS X 35 Verifying Installation 35 Install Using Terraform Version Manager 35 Install Using Other Methods 36 Install Packer 36 Install Using Homebrew on Mac OS X 36 Verifying Installation 36 Install Vagrant 36 Install Using Homebrew on Mac OS X 37 Verifying Installation 38 Install Ansible 38 Install Using Homebrew on Mac OS X 38 Install Using PIP (Python Package Manager) on Mac OS X 39 Use an Ansible-Provided Vagrant Machine as a Control Machine 39 About the GitHub Repository 41 Getting Started 41 Summary 41 Chapter 3: Build a Dev SharePoint Farm with Vagrant and ServerSpec 42 GitHub Repository Project Structure 43 Creating a Windows-Based Image Using Packer 44 Disabling Windows Updates 45 The Packer Template 45 Builders 49 Provisioners 49 Post Processors 49 Variables 49 Building the Box File 50 Building the Vagrant SharePoint Test Environment 50 Using SPAutoInstaller 51 A Look at the Vagrant Multimachine Environment 51 A Closer Look at the Vagrantfile 52 Building the Ansible Inventory Dynamically 56 Performing Ansible Provisioning Tasks Based on SharePoint Role 56 Bring up the SharePoint Vagrant Environment 62 Bring up the Domain Controller 63 The Ansible Tasks 63 Using Ansible to Execute Playbooks or Specific Tasks 68 Bring up the Database Server 76 The Ansible Tasks 77 The mountimg.yml Task 78 Bring up the WFE and App Server 82 The join-to-domain.yml Task 84 The mount-sp-img.yml Task 85 Using ServerSpec to Test SharePoint Server Role Config 88 About ServerSpec 89 About Test-Driven Infrastructure 89 Running Tests on Provisioning VM 89 Running Tests Manually 91 Testing Additional Server Roles 93 Push Vagrantfile to GitHub 94 Summary 94 Chapter 4: Provisioning the SharePoint Farm to Azure Using Terraform 95 About the Solution Architecture 95 Terraform Folder Structure 96 Create Base VM Image Using Packer 98 Running the Bash Script 99 Checking for Required Software 99 Using Device Authentication with Azure CLI 101 Specify a Unique Name for Storage Account, Resource Group, and Client 102 Application Secret 102 Choose a Location 103 Successful Script Output 106 Create Packer Image VM 107 The Packer Template for Azure 107 Configuring the Terraform Remote State 112 The Backend.tf File 113 Run Terraform Init 114 Create Core Azure Resources Using Terraform 116 About Terraform Modules 116 Defining the Core Networking Resources 117 Terraforming the SharePoint 2016 Servers in the Farm Topology 125 Preparing to Deploy Resources to Azure 125 Execute Terraform Plan 126 Executing Terraform Apply 131 Testing Terraform Infrastructure Deployments Using InSpec 139 About InSpec 2.0 139 Getting Started 139 Terraform Output 140 Testing Our Resource Group 140 Testing Virtual Machines 141 Executing InSpec Tests 143 Generating the Dynamic Ansible Inventory File for Azure Resources 144 Using azure_rm.py and azure_rm.ini Files Provided by Ansible 145 Installing Azure Python SDK 146 Configuring Ansible Credentials Using Environment Variables 147 Running the azure_rm.py Python Script 147 Use Ansible Playbooks to Install and Configure SharePoint 2016 Farm 148 Execute Ansible Ad Hoc Commands Against Azure Virtual Machine 149 Run Ansible’s Setup Module via Ad Hoc Command 149 Resolving Errors 154 Run Ansible Playbooks by Role 156 Perform Configuration Management for the Domain Controller 156 Perform Configuration Management for the Database Server 161 Perform Configuration Management for the App Server and WFE Roles 162 Summary 163 Chapter 5: Provisioning the SharePoint Farm to AWS Using Terraform and Ansible 164 About the Solution Architecture 164 Terraform Folder Structure 166 Set up AWS Credentials 167 Configuring the Terraform Back End 168 Create the S3 Bucket via AWS CLI 168 Run Terraform Init 171 Define the Terraform Back End 173 Create Core AWS Resources Using Terraform 174 About Terraform Modules 174 Defining the Core Networking Resources 176 Terraforming the SharePoint 2016 Servers in the Farm Topology 182 Preparing to Deploy Resources to AWS 184 Execute Terraform Plan 184 Executing Terraform Apply 186 Generating the Dynamic Ansible Inventory File for AWS Resources 188 Using EC2.py and EC2.ini Files Provided by Ansible 188 Installing Boto 189 Running the EC2 Script 189 Use Ansible Playbooks to Install and Configure SharePoint 2016 Farm 190 Execute Ansible Ad Hoc Commands Against AWS Instance 191 Obtaining EC2 Instances IP Addresses 191 Run Ansible’s Setup Module via Ad Hoc Command 194 Run Ansible Playbooks by Role 197 Perform Configuration Management for the Domain Controller 197 Perform Configuration Management for the Database Server 198 Perform Configuration Management for SharePoint Server 198 Perform Configuration Management for SharePoint Web Server 202 Validating the Installation 202 Resolving Errors 203 Summary 205 Chapter 6: Scaling the Farm Using Terraform and Ansible 206 Farm Topology 206 Architecture Changes 207 Building the Packer WFE Image 208 Getting Started 208 The WFE Packer Template 208 The Ansible WFE Playbook Changes 211 Scaling Farm Topology Using Terraform 215 Scaling Up 215 Scaling Out 217 About Virtual Machine Availability Sets 217 Terraforming Multiple WFE Virtual Machines 217 Changing the Network Interface Declaration 218 Declaring the Virtual Machine Availability Set 219 Modifying the WFE VM Terraform Declaration 219 Terraforming the Load Balancer and NAT Rules 221 Provisioning the New SharePoint 2016 Farm 226 Performing Configuration Management on the WFEs 229 Accessing the WFEs in the Availability Set 229 Joining WFEs to the SharePoint Farm 230 Summary 232 Chapter 7: Establishing an Enterprise Environment to Manage and Collaborate as a Team 233 Version Control 233 Orchestration 234 Security 238 Secrets Management 238 Access Control 239 Code 240 Automations 240 Environments 240 Resilience 241 One Equals Zero 241 The “Nines” of Availability 241 Shared Collaboration 242 Terraform Enterprise 242 Workspace Management and GUI 242 Version Control Connection 242 Secure Variable Management 243 Remote Plans, Applies, State Storage, Locking, State Rollback 243 Private Module Registry 243 SaaS Install 244 Summary 244 Index 245 Deploy a SharePoint farm in a repeatable, predictable, and reliable fashion using Infrastructure as Code (IaC) techniques to automate provisioning. Savvy IT pros will learn how to use DevOps practices and open source tools to greatly reduce costs, and streamline management operations for SharePoint farms deployed via Amazon Web Services (AWS), Azure, or on premise. DevOps for SharePoint will help you navigate the complex challenges of deploying and managing SharePoint Server farms. You will learn how to reduce time-consuming tasks and errors when generating development, testing, or production environments. And you will benefit from learning proven methods to apply Microsoft updates with minimal downtime and productivity loss. Whether you are a SharePoint architect, IT pro, or developer helping customers with the SharePoint platform, this book will teach you the most useful DevOps practices to tackle those issues and broaden your skill set. What You'll Learn: Understand the basics of the most popular open source tools--Vagrant, Packer, Terraform, and Ansible--and how to use them in the context of deploying and scaling a SharePoint farm Use Vagrant to build SharePoint development environments in less than an hour, and add automated testing Use Packer to create a "golden image" with preconfigured settings, and then use it as the base image in your Terraform configuration for both AWS and Azure farms Use Terraform to scale your SharePoint farm topology Use Red Hat's Ansible Playbooks to perform configuration management on your farm Use Terraform to deploy immutable infrastructure environments using IaC (Infrastructure as Code) Use InSpec 2.0 to stay in compliance by testing your cloud infrastructure Use Ansible to apply Microsoft updates and patches This book is for IT pros and developers who are looking to expand their knowledge and take a modern approach by using open source technologies to work with Microsoft products. Experience installing SharePoint, and a basic understanding of either Azure or AWS, is helpful. Oscar Medina, CEO and founder of SharePointAce LLC, is a multi-platform senior technologist, consultant, author, and speaker, with more than 16 years of experience. Prior to that, he worked at Microsoft, guiding and supporting Fortune 1000 companies on SharePoint. Oscar is passionate about Azure, Office 365, and SharePoint, particularly in how they are enabled with emerging and open source technologies. Ethan Schumann is a Solutions Architect for Onica, a leading AWS Cloud Consulting Partner. He specializes in automation and DevOps transformations with a focus on enterprise implementations using cloud native services and modern tooling. He has experience designing and implementing various technologies such as Kubernetes containerization, CI/CD constructs for various development workflows, and large-scale Microsoft products, including Sharepoint Front Matter ....Pages i-xiii Introduction (Oscar Medina, Ethan Schumann)....Pages 1-20 Getting up and Running: Set up Your Environment (Oscar Medina, Ethan Schumann)....Pages 21-30 Build a Dev SharePoint Farm with Vagrant and ServerSpec (Oscar Medina, Ethan Schumann)....Pages 31-83 Provisioning the SharePoint Farm to Azure Using Terraform (Oscar Medina, Ethan Schumann)....Pages 85-153 Provisioning the SharePoint Farm to AWS Using Terraform and Ansible (Oscar Medina, Ethan Schumann)....Pages 155-196 Scaling the Farm Using Terraform and Ansible (Oscar Medina, Ethan Schumann)....Pages 197-223 Establishing an Enterprise Environment to Manage and Collaborate as a Team (Oscar Medina, Ethan Schumann)....Pages 225-236 Back Matter ....Pages 237-240