Cloud Native Observability

With this insightful guide, authors Kenichi Shibata, Rob Skillington, and Martin Mao take you through the differences between traditional and cloud native system observability. SREs, cloud native engineers, CIOs, and CTOs will learn that while many principles of cloud native and traditional systems are similar, highly scalable and dynamic cloud native systems present unique challenges to overcome. 1. The Cloud Native Impact on Observability Challenges of Cloud Native Observability Deep Dive into Observability Data Observability Data Is Growing in Scale Understanding Cardinality and Dimensionality Cloud Native Systems Are Flexible and Ephemeral The Goldilocks Zone of Cloud Native Observability Cloud Native Environments Emit Exponentially More Data Than Traditional Environments Delivering Reduced Business Outcomes Observability Practitioners Lose Focus Increasing Cost of Observability Data The Cloud Native Impact Slower Troubleshooting Tools Become Unreliable Use Context to Troubleshoot Faster The Three Phases of Observability: An Outcome-Focused Approach Remediating at Any Phase, with Any Signal Conclusion 2. Cloud Native Challenges in the Real World Impact of Uncontrolled Data Growth on System Performance Controlling Cost Case Study 1: Improving Performance While Gaining Huge Cost Savings The Challenge Approach Impact of Uncontrolled Data Growth on Observability Reliability Poor Developer Experience Caused by Poor Observability Data Case Study 2: Increased Observability Reliability and Improved Developer Experience The Challenge Approach Making Way for Fast-Paced Innovation Regulatory Requirements Case Study 3: Navigating Observability Challenges in Balancing Rapid Fintech Growth and SLA Compliance The Challenge Approach Conclusion 3. Strategies for Controlling Observability Data Growth and Complexity Emerging Solution Using a Repeatable Framework Using FinOps as an Inspiration Observability Data Optimization Cycle Step 0: Centralized Governance Autonomy and Allocations to Increase Responsibility and Improve Responsiveness Usable Capacity by Allocation to Optimize Use Cases Using Observability Team as Consultants Instead of as Bottlenecks Framework Components Step 1: Analyze Traffic Analysis Usage Analysis Combining Traffic and Usage Analysis to Make Decisions Output of Analyze Step Step 2: Refine Dropping Retention Resolution Downsampling Aggregation Output of Refine Step Step 3: Operate Expanding Visibility and Coverage Freeing Up More of the Observability Team’s Time to Tackle Strategic Projects Conclusion 4. Open Source Telemetry Standards: Prometheus, OpenTelemetry, and Beyond Instrumentation Before Prometheus and OTel Data Collection Is Controlled by Users Prometheus Interoperability Between Different Observability Tools Standardization to Prometheus Prometheus Reliability Prometheus: The Good Prometheus: The Not-So-Good OpenTelemetry What Is OTel? The OTel Specification OTel SDK OpenTelemetry Collector OTel: The Promise OTel: The Reality Limitations of maturity Backend support Where to Start with OTel Implications of OTel’s Approach Fluent Bit Conclusion