BigQuery for Data Warehousing : Managed Data Analysis in the Google Cloud

Create a data warehouse, complete with reporting and dashboards using Google’s BigQuery technology. This book takes you from the basic concepts of data warehousing through the design, build, load, and maintenance phases. You will build capabilities to capture data from the operational environment, and then mine and analyze that data for insight into making your business more successful. You will gain practical knowledge about how to use BigQuery to solve data challenges in your organization.BigQuery is a managed cloud platform from Google that provides enterprise data warehousing and reporting capabilities. Part I of this book shows you how to design and provision a data warehouse in the BigQuery platform. Part II teaches you how to load and stream your operational data into the warehouse to make it ready for analysis and reporting. Parts III and IV cover querying and maintaining, helping you keep your information relevant with other Google Cloud Platform services and advanced BigQuery. Part V takes reporting to the next level by showing you how to create dashboards to provide at-a-glance visual representations of your business situation. Part VI provides an introduction to data science with BigQuery, covering machine learning and Jupyter notebooks.**What You Will Learn** * Design a data warehouse for your project or organization * Load data from a variety of external and internal sources * Integrate other Google Cloud Platform services for more complex workflows * Maintain and scale your data warehouse as your organization grows * Analyze, report, and create dashboards on the information in the warehouse * Become familiar with machine learning techniques using BigQuery ML **Who This Book Is For**Developers who want to provide business users with fast, reliable, and insightful analysis from operational data, and data analysts interested in a cloud-based solution that avoids the pain of provisioning their own servers. Create a data warehouse, complete with reporting and dashboards using Google’s BigQuery technology. This book takes you from the basic concepts of data warehousing through the design, build, load, and maintenance phases. You will build capabilities to capture data from the operational environment, and then mine and analyze that data for insight into making your business more successful. You will gain practical knowledge about how to use BigQuery to solve data challenges in your organization. BigQuery is a managed cloud platform from Google that provides enterprise data warehousing and reporting capabilities. Part I of this book shows you how to design and provision a data warehouse in the BigQuery platform. Part II teaches you how to load and stream your operational data into the warehouse to make it ready for analysis and reporting. Parts III and IV cover querying and maintaining, helping you keep your information relevant with other Google Cloud Platform services and advanced BigQuery. Part V takes reporting to the next level by showing you how to create dashboards to provide at-a-glance visual representations of your business situation. Part VI provides an introduction to data science with BigQuery, covering machine learning and Jupyter notebooks. What You Will Learn Design a data warehouse for your project or organization Load data from a variety of external and internal sources Integrate other Google Cloud Platform services for more complex workflows Maintain and scale your data warehouse as your organization grows Analyze, report, and create dashboards on the information in the warehouse Become familiar with machine learning techniques using BigQuery ML Who This Book Is For Developers who want to provide business users with fast, reliable, and insightful analysis from operational data, and data analysts interested in a cloud-based solution that avoids the pain of provisioning their own servers. Front Matter ....Pages i-xxxv Front Matter ....Pages 1-1 Settling into BigQuery (Mark Mucchetti)....Pages 3-21 Starting Your Warehouse Project (Mark Mucchetti)....Pages 23-43 All My Data (Mark Mucchetti)....Pages 45-60 Managing BigQuery Costs (Mark Mucchetti)....Pages 61-71 Front Matter ....Pages 73-73 Loading Data into the Warehouse (Mark Mucchetti)....Pages 75-103 Streaming Data into the Warehouse (Mark Mucchetti)....Pages 105-121 Dataflow (Mark Mucchetti)....Pages 123-152 Front Matter ....Pages 153-153 Care and Feeding of Your Warehouse (Mark Mucchetti)....Pages 155-172 Querying the Warehouse (Mark Mucchetti)....Pages 173-207 Scheduling Jobs (Mark Mucchetti)....Pages 209-230 Serverless Functions with GCP (Mark Mucchetti)....Pages 231-251 Cloud Logging (Mark Mucchetti)....Pages 253-269 Front Matter ....Pages 271-271 Advanced BigQuery (Mark Mucchetti)....Pages 273-303 Data Governance (Mark Mucchetti)....Pages 305-332 Adapting to Long-Term Change (Mark Mucchetti)....Pages 333-351 Front Matter ....Pages 353-353 Reporting (Mark Mucchetti)....Pages 355-378 Dashboards and Visualization (Mark Mucchetti)....Pages 379-400 Google Data Studio (Mark Mucchetti)....Pages 401-416 Front Matter ....Pages 417-417 BigQuery ML (Mark Mucchetti)....Pages 419-468 Jupyter Notebooks and Public Datasets (Mark Mucchetti)....Pages 469-496 Conclusion (Mark Mucchetti)....Pages 497-498 Cloud Shell and Cloud SDK (Mark Mucchetti)....Pages 499-508 Sample Project Charter (Mark Mucchetti)....Pages 509-513 Back Matter ....Pages 515-525