Point Cloud Compression: Technologies and Standardization

3D point clouds have broad applications across various industries and have contributed to advancements in fields such as autonomous driving, immersive media, metaverse, and cultural heritage protection. With the fast growth of 3D point cloud data and its applications, the need for efficient compression technologies has become paramount. This book delves into the forefront of point cloud compression, exploring key technologies, standardization efforts, and future prospects. This comprehensive book uncovers the foundational concepts, data acquisition methods, and datasets associated with point cloud compression. By examining the fundamental compression technologies, readers can obtain a clear understanding of prediction coding, transform coding, quantization techniques, and entropy coding. Through vivid illustrations and examples, the book elucidates how these techniques have evolved over the years and their potentials for the future. To provide a complete picture, the book presents cutting-edge research methods in point cloud compression and facilitates comparisons among them. Readers can be equipped with an in-depth understanding of the latest advancements, and can gain insights into the various approaches employed in this dynamic field. Another distinguishing aspect of this book is its exploration of standardization works for point cloud compression. Notable standards, such as MPEG G-PCC, AVS PCC, and MPEG V-PCC, are thoroughly illustrated. By delving into the methods used in geometry-based, video-based, and deep learning-based compression, readers become familiar with the latest breakthroughs in the standard communities. Preface Acknowledgments Contents Acronyms 1 Introduction 1.1 Basic Concepts of Point Clouds 1.1.1 Definition of Point Clouds 1.1.2 Characteristics of Point Clouds 1.1.3 Different Types of Point Clouds 1.2 Point Cloud Data Acquisition 1.2.1 Laser Scanner 1.2.2 Depth Camera Structured Light Depth Camera TOF Depth Camera Binocular Stereo Depth Camera 1.2.3 Other Acquisition Methods 1.3 Representative Point Cloud Compression Datasets 1.3.1 MPEG PCC Datasets Category 1 (Static Objects and Scenes) Category 2 (Dynamic Objects) Category 3 (Dynamic Acquisitions) 1.3.2 AVS PCC Datasets 1.3.3 Other Datasets Microsoft Voxelized Upper Bodies KITTI Synthetic Datasets 1.4 Typical Application Fields of Point Cloud Technologies 1.4.1 Autonomous Driving 1.4.2 Digital Twin Cities 1.4.3 Cultural Relics Protection 1.4.4 Topographic Mapping 1.4.5 Gaming and Entertainment 1.5 Developments and Challenges of Point Cloud Technologies 1.5.1 Large-scale Data Acquisition 1.5.2 Storage, Transmission, and Visualization 1.6 Summary Exercises References 2 Background Knowledge 2.1 Fundamentals of Information Theory 2.1.1 Basic Concepts Entropy Joint Entropy Conditional Entropy Mutual Information 2.1.2 Shannon's Coding Theorems Source Coding Theorem Channel Coding Theorem Rate-Distortion Theorem 2.2 Point Cloud Compression Technologies 2.2.1 Data Structure of Point Cloud 1D Structure 2D Structure 3D Structure 4D Structure 2.2.2 Data Redundancy of Point Cloud 2.2.3 Predictive Coding 2.2.4 Transform Coding 2.2.5 Quantization 2.2.6 Entropy Coding 2.3 Point Cloud Quality Assessment 2.3.1 Subjective Quality Assessment 2.3.2 Objective Quality Assessment 2.4 Summary Exercises References 3 Predictive Coding 3.1 Introduction 3.2 Intra Prediction 3.2.1 Geometry Intra Prediction Octree Intra Prediction Predictive Tree Intra Prediction 3.2.2 Attribute Intra Prediction Generating Prediction Order Determining Neighbor Points Calculating Prediction Values 3.3 Inter Prediction 3.3.1 Geometry Inter Prediction Motion Estimation and Motion Compensation Octree Inter Prediction Predictive Tree Inter Prediction 3.3.2 Attribute Inter Prediction 3.4 Summary Exercises References 4 Transform Coding 4.1 Introduction 4.2 Discrete Cosine Transform 4.2.1 Introduction of DCT 4.2.2 Properties of DCT 4.2.3 Application in Point Cloud Compression 4.3 Wavelet Transform 4.3.1 Fourier Transform 4.3.2 Wavelet Transform 4.3.3 Lifting Transform Split Prediction Update 4.3.4 Application in Point Cloud Compression 4.4 Graph Fourier Transform 4.4.1 Introduction to GFT Relation with KLT Relation with DCT 4.4.2 Basics of Graph Spectral Compression The Property of Transform The Scheme of Graph Spectral Compression 4.4.3 Application in Point Cloud Compression Geometry-Driven Graph Feature-Driven Graph Rate Cost-Driven Graph 4.5 Summary Exercises References 5 Quantization Techniques 5.1 Introduction 5.1.1 Scalar and Vector Quantization 5.1.2 Uniform and Non-uniform Quantization 5.1.3 Rate-Distortion Optimization Quantization 5.2 Application in Point Cloud Compression 5.2.1 Geometry Quantization 5.2.2 Attribute Quantization Quantization Table Slice-Based Attribute Quantization Control Layer-Based Attribute Quantization Control Region-Based Attribute Quantization Control Adaptive Quantization for RAHT 5.3 Quantization for Rate Control 5.4 Summary Exercises References 6 Entropy Coding 6.1 Introduction 6.2 Fundamental Principles 6.2.1 Entropy and Optimal Coding 6.2.2 Variable-Length Coding Huffman Coding Golomb Coding 6.2.3 Arithmetic Coding 6.3 Application in Point Cloud Compression 6.3.1 Run-Length Coding 6.3.2 Context-Adaptive Binary Arithmetic Coding 6.3.3 Asymmetric Numeral Systems Coding 6.4 Summary Exercises References 7 MPEG Geometry-Based Point Cloud Compression (G-PCC) Standard 7.1 Introduction 7.2 Tile and Slice Partition 7.2.1 Uniform Geometry Partition 7.2.2 Uniform Square Partition 7.2.3 Uniform Octree Partition 7.3 Prediction on Geometry 7.3.1 OT and Implicit QTBT 3D Space Representation with OT, QT, and BT C7:qtbt10 Context-Based Occupancy Coding 7.3.2 Prediction Tree Coding Framework High Latency Strategy Slow Mode High Latency Strategy Fast Mode Low Latency Strategy Angular Mode C7:anPrediction 7.3.3 Trisoup 7.4 Prediction and Transform on Attribute 7.4.1 Prediction Transform 7.4.2 Lifting Transform 7.4.3 RAHT 7.5 Quantization 7.5.1 Geometry Quantization Slice-Based Geometry Quantization Control C7:sliceQuan1,C7:sliceQuan2 Fine Granularity Quantization Step Size Control C7:fineQuan Octree-Based Geometry Quantization Control C7:octreeQuan Combination of Slice-Based and Octree-Based Geometry Quantization Control C7:sliceQuan1 Harmonization with Planar Coding Mode C7:harQuan Harmonization with Implicit QTBT C7:harQuan Quantization of IDCM Nodes 7.5.2 Attribute Quantization Quantization Table C7:qt1,C7:qt2 Adaptive Quantization Scheme for Lifting Transform Adaptive Quantization Scheme for RAHT Quantization Control 7.6 Entropy Coding 7.6.1 Geometry Entropy Coding Bitwise-Based Binary Coding of Occupancy Code C7:geoBitEntropy Bytewise-Based Binary Coding for Occupancy Code C7:geoByteEntropy 7.6.2 Attribute Entropy Coding 7.7 Emerging Standard 7.7.1 Inter-geometry Coding Inter-prediction for Octree Geometry Coding Inter-prediction for Predictive Geometry Coding 7.7.2 Inter-attribute Coding 7.8 Summary Exercises References 8 AVS Point Cloud Compression Standard 8.1 Introduction 8.2 Coding Scheme 8.3 Geometry Coding 8.3.1 Octree Context Model I Context Model II Planar Mode Isolated Point Mode 8.3.2 Predictive Tree 8.4 Attribute Coding 8.4.1 Attribute Prediction Prediction Points Searching Reference Point Set Updating Prediction Values Calculation Cross Attributes Prediction 8.4.2 Attribute Transform Single-Layer Transform Multi-Layer Transform 8.5 Entropy Coding 8.5.1 Residual Run-Length Coding 8.5.2 Non-zero Residual Coding 8.6 Summary Exercises References 9 MPEG Video-Based Point Cloud Compression (V-PCC)Standard 9.1 Introduction 9.1.1 Developing History of V-PCC 9.1.2 Encoding and Decoding Frameworks of V-PCC Encoding Framework Decoding Framework 9.2 Point Cloud Projection 9.2.1 Patch Segmentation 9.2.2 Patch Packing Patch Packing Strategy Patch Packing Optimization 9.2.3 Image Generation Occupancy Image Generation Geometry Image Generation Attribute Image Generation Padding 9.3 Encoding of Projected Videos 9.3.1 Geometry and Attribute Sequences Compression 9.3.2 Occupancy Sequence Compression 9.3.3 Auxiliary Information 9.4 Summary Exercises References 10 MPEG AI-Based 3D Graphics Coding Standard 10.1 Introduction 10.2 AI-Based Point Cloud Compression Technologies 10.2.1 Point-Based Point Cloud Compression 10.2.2 Octree-Based Point Cloud Compression 10.2.3 Voxel-Based Point Cloud Compression 10.3 MPEG AI-3DGC Standard 10.3.1 Static Point Cloud Compression GRASP-Net DDA-Net OctSqueeze VoxelContext-Net SparsePCGC CJDU 10.3.2 Dynamic Point Cloud Compression D-DPCC S-DPCC 10.4 Summary Exercises References 11 Future Work 11.1 Future Work on Main Point Cloud Coding Tools 11.1.1 Prediction 11.1.2 Transform 11.1.3 Quantization 11.1.4 Entropy Coding 11.2 Future Work on MPEG Geometry-Based Point Cloud Compression 11.2.1 Dynamic Point Cloud Coding 11.2.2 Low Latency and Low Complexity Point Cloud Compression 11.2.3 HVS-Inspired Quality Metrics for Point Cloud Compression 11.3 Future Work on AVS Point Cloud Compression 11.3.1 Developing Inter-frame Framework 11.3.2 Realizing Video-Based Compression 11.3.3 Exploring Deep Learning Methods 11.3.4 Extension to Mesh Coding 11.4 Future Work on MPEG Video-Based Point CloudCompression 11.5 Future Work on Deep Learning-Based Point CloudCompression 11.6 Future Work on Point Cloud Feature Compression References Index