Image-Based Modeling

“This book guides you in the journey of 3D modeling from the theory with elegant mathematics to applications with beautiful 3D model pictures. Written in a simple, straightforward, and concise manner, readers will learn the state of the art of 3D reconstruction and modeling.” —Professor Takeo Kanade, Carnegie Mellon University About this book: * The computer vision and graphics communities use different terminologies for the same ideas * This books provides a translation, enabling graphics researchers to apply vision concepts, and vice-versa * Independence of chapters allows readers to directly jump into a specific chapter of interest * Compared to other texts, gives more succinct treatment overall, and focuses primarily on vision geometry __Image-Based Modeling__ is for graduate students, researchers, and engineers working in the areas of computer vision, computer graphics, image processing, robotics, virtual reality, and photogrammetry. About the Author: Long Quan is a Professor of the Department of Computer Science and Engineering at the Hong Kong University of Science and Technology. He received a Ph.D. degree in Computer Science from INRIA, and has been a CNRS researcher at INRIA. Professor Quan is a Fellow of the IEEE Computer Society. Foreword 6 Preface 7 Acknowledgements 9 Notation 10 Contents 11 Introduction 15 Part I 19 Geometry prerequisite 20 2.1 Introduction 21 2.2 Projective geometry 21 2.2.1 The basic concepts 21 2.2.2 Projective spaces and transformations 23 2.2.3 Affine and Euclidean specialization 29 2.3 Algebraic geometry 34 2.3.1 The simple methods 34 2.3.2 Ideals, varieties, and Gr ̈obner bases 36 2.3.3 Solving polynomial equations with Gr ̈obner bases 37 Multi-view geometry 41 3.1 Introduction 42 3.2 The single-view geometry 42 3.2.1 What is a camera? 42 3.2.2 Where is the camera? 47 3.2.3 The DLT calibration 49 3.2.4 The three-point pose algorithm 51 3.3 The uncalibrated two-view geometry 54 3.3.1 The fundamental matrix 55 3.3.2 The seven-point algorithm 57 3.3.3 The eight-point linear algorithm 58 3.4 The calibrated two-view geometry 59 3.4.1 The essential matrix 59 3.4.2 The five-point algorithm 61 3.5 The three-view geometry 65 3.5.1 The trifocal tensor 66 3.5.2 The six-point algorithm 70 3.5.3 The calibrated three views 75 3.6 The N-view geometry 78 3.6.1 The multi-linearities 78 3.6.2 Auto-calibration 80 3.7 Discussions 84 3.8 Bibliographic notes 84 Part II 86 Feature point 87 4.1 Introduction 88 4.2 Points of interest 88 4.2.1 Tracking features 88 4.2.2 Matching corners 90 4.2.3 Discussions 91 4.3 Scale invariance 92 4.3.1 Invariance and stability 92 4.3.2 Scale, blob and Laplacian 92 4.3.3 Recognizing SIFT 93 4.4 Bibliographic notes 94 Structure from Motion 95 5.1 Introduction 96 5.1.1 Least squares and bundle adjustment 96 5.1.2 Robust statistics and RANSAC 98 5.2 The standard sparse approach 100 5.2.1 A sequence of images 102 5.2.2 A collection of images 103 5.3 The match propagation 104 5.3.1 The best-first match propagation 104 5.3.2 The properties of match propagation 107 5.3.3 Discussions 111 5.4 The quasi-dense approach 113 5.4.1 The quasi-dense resampling 113 5.4.2 The quasi-dense SFM 114 5.4.3 Results and discussions 121 5.5 Bibliographic notes 127 Part III 129 Surface modeling 130 6.1 Introduction 131 6.2 Minimal surface functionals 132 6.3 A unified functional 133 6.4 Level-set method 133 6.5 A bounded regularization method 134 6.6 Implementation 136 6.7 Results and discussions 138 6.8 Bibliographic notes 145 Hair modeling 146 7.1 Introduction 147 7.2 Hair volume determination 148 7.3 Hair fiber recovery 149 7.3.1 Visibility determination 149 7.3.2 Orientation consistency 150 7.3.3 Orientation triangulation 150 7.4 Implementation 151 7.5 Results and discussions 153 7.6 Bibliographic notes 157 Tree modeling 158 8.1 Introduction 159 8.2 Branche recovery 162 8.2.1 Reconstruction of visible branches 162 8.2.2 Synthesis of occluded branches 164 8.2.3 Interactive editing 166 8.3 Leaf extraction and reconstruction 168 8.3.1 Leaf texture segmentation 168 8.3.2 Graph-based leaf extraction 171 8.3.3 Model-based leaf reconstruction 174 8.4 Results and discussions 176 8.5 Bibliographic notes 183 Fac ̧ade modeling 185 9.1 Introduction 186 9.2 Fac ̧ade initialization 188 9.2.1 Initial flat rectangle 189 9.2.2 Texture composition 189 9.2.3 Interactive refinement 191 9.3 Fac ̧ade decomposition 192 9.3.1 Hidden structure discovery 192 9.3.2 Recursive subdivision 193 9.3.3 Repetitive pattern representation 194 9.3.4 Interactive subdivision refinement 195 9.4 Fac ̧ade augmentation 196 9.4.1 Depth optimization 196 9.4.2 Cost definition 198 9.4.3 Interactive depth assignment 198 9.5 Fac ̧ade completion 200 9.6 Results and discussions 200 9.7 Bibliographic notes 205 Building modeling 207 10.1 Introduction 208 10.2 Pre-processing 209 10.3 Building segmentation 211 10.3.1 Supervised class recognition 211 10.3.2 Multi-view semantic segmentation 213 10.4 Building partition 215 10.4.1 Global vertical alignment 216 10.4.2 Block separator 216 10.4.3 Local horizontal alignment 217 10.5 Fac ̧ade modeling 218 10.5.1 Inverse orthographic composition 219 10.5.2 Structure analysis and regularization 221 10.5.3 Repetitive pattern rediscovery 224 10.5.4 Boundary regularization 225 10.6 Post-processing 226 10.7 Results and discussions 227 10.8 Bibliographic notes 232 List of Algorithms 234 List of Figures 235 References 243 Index 255 "This book guides you in the journey of 3D modeling from the theory with elegant mathematics to applications with beautiful 3D model pictures. Written in a simple, straightforward, and concise manner, readers will learn the state of the art of 3D reconstruction and modeling."--Professor Takeo Kanade, Carnegie Mellon University About this book: The computer vision and graphics communities use different terminologies for the same ideas This books provides a translation, enabling graphics researchers to apply vision concepts, and vice-versa Independence of chapters allows readers to directly jump into a specific chapter of interest Compared to other texts, gives more succinct treatment overall, and focuses primarily on vision geometry Image-Based Modeling is for graduate students, researchers, and engineers working in the areas of computer vision, computer graphics, image processing, robotics, virtual reality, and photogrammetry. About the Author: Long Quan is a Professor of the Department of Computer Science and Engineering at the Hong Kong University of Science and Technology. He received a Ph. D. degree in Computer Science from INRIA, and has been a CNRS researcher at INRIA. Professor Quan is a Fellow of the IEEE Computer Society The computer vision and graphics communities use different techniques for the same purposes. Image-Based Modeling bridges the gap, enabling graphics researcher to apply vision concepts, and vice-versa. Independence of chapters allows readers to directly jump into a specific chapter of interest. Compared to other texts, it gives a more succinct treatment overall, and focuses primarily on vision geometry and 3D object modeling Image-Based Modeling is for graduate students, researchers, and engineers working in the areas of computer vision, computer graphics, image processing, robotics, virtual reality, and photogrammetry. --Book Jacket