Let's Go Further: Advanced patterns for building APIs and web applications in Go

Let's Go Further guides you through the start-to-finish build of a modern JSON API in Go – from project setup to deployment in production. As well as covering fundamental topics like sending and receiving JSON data, the book goes in-depth and explores practical code patterns and best practices for advanced functionality like implementing graceful shutdowns, managing background tasks, reporting metrics, and much more. You’ll learn a lot about topics that are often important to your real-world work, but which are rarely discussed in beginner-level courses and aren't fully explained by the official Go documentation. Let's Go Further also goes beyond development. It outlines tools and techniques to help manage your project on an ongoing basis, and also gives you a step-by-step playbook for deploying your API to a live production server. By the end of the book you'll have all the knowledge you need to create robust and professional APIs which act as backends for SPAs and native mobile applications, or function as stand-alone services. If you read and enjoyed the first Let’s Go book, this course should be a great fit for you and an ideal next step in mastering Go. Contents Introduction Conventions About the author Copyright and disclaimer Prerequisites Background knowledge Go 1.17 Other software Getting Started Project Setup and Skeleton Structure Generating the skeleton directory structure Hello world! A Basic HTTP Server Creating the healthcheck handler Demonstration Additional Information API versioning API Endpoints and RESTful Routing Choosing a router Encapsulating the API routes Adding the new handler functions Creating a helper to read ID parameters Additional Information Conflicting routes Customizing httprouter behavior Sending JSON Responses Fixed-Format JSON Additional Information JSON charset JSON Encoding Creating a writeJSON helper method Additional Information How different Go types are encoded Using json.Encoder Performance of json.Encoder and json.Marshal Additional JSON encoding nuances Encoding Structs Changing keys in the JSON object Hiding struct fields in the JSON object Additional Information The string struct tag directive Formatting and Enveloping Responses Relative performance Enveloping responses Additional Information Response structure Advanced JSON Customization Customizing the Runtime field Additional Information Alternative #1 - Customizing the Movie struct Alternative #2 - Embedding an alias Sending Error Messages Routing errors Additional Information System-generated error responses Parsing JSON Requests JSON Decoding Zero values Additional Information Supported destination types Using the json.Unmarshal function Additional JSON decoding nuances Managing Bad Requests Triaging the Decode error Making a bad request helper Additional Information Panicking vs returning errors Restricting Inputs Custom JSON Decoding The json.Unmarshaler interface Validating JSON Input Creating a validator package Performing validation checks Making validation rules reusable Database Setup and Configuration Setting up PostgreSQL Installing PostgreSQL Connecting to the PostgreSQL interactive terminal Creating databases, users, and extensions Connecting as the new user Additional Information Optimizing PostgreSQL settings Connecting to PostgreSQL Establishing a connection pool Decoupling the DSN Additional Information Using the DSN with psql Configuring the Database Connection Pool Configuring the pool The SetMaxOpenConns method The SetMaxIdleConns method The SetConnMaxLifetime method The SetConnMaxIdleTime method Putting it into practice Configuring the connection pool SQL Migrations An Overview of SQL Migrations Installing the migrate tool Working with SQL Migrations Executing the migrations Additional Information Migrating to a specific version Executing down migrations Fixing errors in SQL migrations Remote migration files Running migrations on application startup CRUD Operations Setting up the Movie Model Additional Information Mocking models Creating a New Movie Executing the SQL query Hooking it up to our API handler Creating additional records Additional Information $N notation Executing multiple statements Fetching a Movie Updating the API handler Additional Information Why not use an unsigned integer for the movie ID? Updating a Movie Executing the SQL query Creating the API handler Using the new endpoint Deleting a Movie Adding the new endpoint Advanced CRUD Operations Handling Partial Updates Performing the partial update Demonstration Additional Information Null values in JSON Optimistic Concurrency Control Preventing the data race Implementing optimistic locking Additional Information Round-trip locking Locking on other fields or types Managing SQL Query Timeouts Mimicking a long-running query Adding a query timeout Timeouts outside of PostgreSQL Updating our database model Additional Information Using the request context Filtering, Sorting, and Pagination Parsing Query String Parameters Creating helper functions Adding the API handler and route Creating a Filters struct Validating Query String Parameters Listing Data Updating the application Filtering Lists Dynamic filtering in the SQL query Full-Text Search Adding indexes Additional Information Non-simple configuration and more information Using STRPOS and ILIKE Sorting Lists Implementing sorting Paginating Lists The LIMIT and OFFSET clauses Updating the database model Returning Pagination Metadata Calculating the total records Updating the code Structured Logging and Error Handling Structured JSON Log Entries Creating a custom logger Additional Information Integration with the http.Server error log Third-party logging packages Panic Recovery Additional Information Panic recovery in other goroutines Rate Limiting Global Rate Limiting Enforcing a global rate limit IP-based Rate Limiting Deleting old limiters Additional Information Distributed applications Configuring the Rate Limiters Graceful Shutdown Sending Shutdown Signals Intercepting Shutdown Signals Catching SIGINT and SIGTERM signals Executing the Shutdown User Model Setup and Registration Setting up the Users Database Table Setting up the Users Model Adding Validation Checks Creating the UserModel Registering a User Additional Information Email case-sensitivity User enumeration Sending Emails SMTP Server Setup Setting up Mailtrap Creating Email Templates Sending a Welcome Email Creating an email helper Using embedded file systems Using our mail helper Checking the email in Mailtrap Additional Information Retrying email send attempts Sending Background Emails Recovering panics Using a helper function Graceful Shutdown of Background Tasks An introduction to sync.WaitGroup Fixing our application User Activation Setting up the Tokens Database Table Creating Secure Activation Tokens Creating the TokenModel and Validation Checks Additional Information The math/rand package Sending Activation Tokens Additional Information A standalone endpoint for generating tokens Activating a User Creating the activateUserHandler The UserModel.GetForToken method Additional Information Web application workflow SQL query timing attack Authentication Authentication Options HTTP basic authentication Token authentication Stateful token authentication Stateless token authentication API-key authentication OAuth 2.0 / OpenID Connect What authentication approach should I use? Generating Authentication Tokens Building the endpoint Additional Information The Authorization header Authenticating Requests Creating the anonymous user Reading and writing to the request context Creating the authentication middleware Demonstration Permission-based Authorization Requiring User Activation Demonstration Splitting up the middleware Additional Information In-handler checks Setting up the Permissions Database Table Relationship between permissions and users Creating the SQL migrations Setting up the Permissions Model Checking Permissions Demonstration Granting Permissions Updating the permissions model Updating the registration handler Cross Origin Requests An Overview of CORS Demonstrating the Same-Origin Policy Demonstration Simple CORS Requests Restricting Origins Supporting multiple dynamic origins Additional Information Partial origin matches The null origin Authentication and CORS Preflight CORS Requests Demonstrating a preflight request Responding to preflight requests Updating our middleware Additional Information Caching preflight responses Preflight wildcards Metrics Exposing Metrics with Expvar Creating Custom Metrics Dynamic metrics Additional Information Protecting the metrics endpoint Removing default metrics Request-level Metrics Additional Information Calculating additional metrics Recording HTTP Status Codes Additional Information Visualizing and analyzing metrics Building, Versioning and Quality Control Creating and Using Makefiles A simple makefile Environment variables Passing arguments Namespacing targets Prerequisite targets and asking for confirmation Displaying help information Phony targets Managing Environment Variables Using a .envrc file Quality Controlling Code Additional Information Testing Module Proxies and Vendoring Module proxies Vendoring Vendoring new dependencies Additional Information The ./... pattern Building Binaries Reducing binary size Cross-compilation Additional Information Build caching Managing and Automating Version Numbers Displaying the version number Including the build time Automated version numbering with Git Using Git tags Deployment and Hosting Creating a Digital Ocean Droplet Creating a SSH key Adding the SSH key to Digital Ocean Creating a droplet Server Configuration and Installing Software Connecting as the greenlight user Connecting to the droplet Additional Information Future changes to the droplet configuration Deployment and Executing Migrations Running the API Running the API as a Background Service Restarting after reboot Disable port 4000 Additional Information Listening on a restricted port Viewing logs Configuring the SMTP provider Additional unit file options Using Caddy as a Reverse Proxy Blocking access to application metrics Using a domain name Enabling HTTPS Additional Information Scaling the infrastructure Appendices Managing Password Resets Additional Information Web application workflow Creating Additional Activation Tokens Authentication with JSON Web Tokens JSON Encoding Nuances Nil and empty slices are encoded differently Using omitempty on a zero-valued struct doesn’t work Using omitempty on a zero-value time.Time doesn’t work Non-ASCII punctuation characters aren’t supported in struct tags Integer, time.Time and net.IP values can be used as map keys Angle brackets and ampersands in strings are escaped Trailing zeroes are removed from floats Working with pre-computed JSON The MarshalText fallback The receiver matters when using MarshalJSON JSON Decoding Nuances Decoding into Go arrays Partial JSON decoding Decoding into interface{} types Decoding a JSON number to an interface{} Struct tag directives Request Context Timeouts Feedback