Statistical Image Processing and Multidimensional Modeling (Information Science and Statistics)

Images are all around us! The proliferation of low-cost, high-quality imaging devices has led to an explosion in acquired images. When these images are acquired from a microscope, telescope, satellite, or medical imaging device, there is a statistical image processing task: the inference of something--an artery, a road, a DNA marker, an oil spill--from imagery, possibly noisy, blurry, or incomplete. A great many textbooks have been written on image processing. However this book does not so much focus on images, per se, but rather on spatial data sets, with one or more measurements taken over a two or higher dimensional space, and to which standard image-processing algorithms may not apply. There are many important data analysis methods developed in this text for such statistical image problems. Examples abound throughout remote sensing (satellite data mapping, data assimilation, climate-change studies, land use), medical imaging (organ segmentation, anomaly detection), computer vision (image classification, segmentation), and other 2D/3D problems (biological imaging, porous media). The goal, then, of this text is to address methods for solving multidimensional statistical problems. The text strikes a balance between mathematics and theory on the one hand, versus applications and algorithms on the other, by deliberately developing the basic theory (Part I), the mathematical modeling (Part II), and the algorithmic and numerical methods (Part III) of solving a given problem. The particular emphases of the book include inverse problems, multidimensional modeling, random fields, and hierarchical methods. Paul Fieguth is a professor in Systems Design Engineering at the University of Waterloo in Ontario, Canada. He has longstanding research interests in statistical signal and image processing, hierarchical algorithms, data fusion, and the interdisciplinary applications of such methods, particularly to problems in medical imaging, remote sensing, and scientific imaging Cover 1 Information Science and Statistics 3 Statistical Image Processing and Multidimensional Modeling 4 ISBN 9781441972934 5 Preface 6 Table of Contents 8 List of Examples 14 List of Code Samples 16 Nomenclature 18 1 Introduction 25 Part I Inverse Problems and Estimation 36 2 Inverse Problems 37 Data Fusion 40 Posedness 43 Conditioning 47 Regularization and Prior Models 53 Deterministic Regularization 58 Bayesian Regularization 61 Statistical Operations 64 Canonical Problems 64 Prior Sampling 66 Estimation 68 Posterior Sampling 73 Parameter Estimation 74 Application 2: Ocean Acoustic Tomography 74 Summary 76 For Further Study 77 Sample Problems 77 3 Static Estimation and Sampling 81 Non-Bayesian Estimation 82 Bayesian Estimation 88 Bayesian Static Problem 91 Bayesian Estimation and Prior Means 92 Approximate Bayesian Estimators 94 Bayesian / NonBayesian Duality 97 Static Sampling 98 Data Fusion 100 Application 3: Atmospheric Temperature Inversion [282] 102 Summary 104 For Further Study 105 Sample Problems 105 4 Dynamic Estimation and Sampling 109 The Dynamic Problem 110 First-Order Gauss–Markov Processes 112 Static — Dynamic Duality 113 Kalman Filter Derivation 117 Kalman Filter Variations 124 Kalman Filter Algorithms 126 Steady-State Kalman Filtering 131 Kalman Filter Smoother 133 Nonlinear Kalman Filtering 138 Dynamic Sampling 142 Dynamic Estimation for Discrete-State Systems 143 Markov Chains 143 The Viterbi Algorithm 144 Comparison to Kalman Filter 146 Application 4: Temporal Interpolation of Ocean Temperature [191] 149 Summary 149 For Further Study 151 Sample Problems 151 Part II Modelling of Random Fields 155 5 Multidimensional Modelling 157 Challenges 158 Coupling and Dimensionality Reduction 159 Sparse Storage and Computation 163 Sparse Matrices 163 Matrix Kernels 165 Computation 167 Modelling 172 Deterministic Models 173 Boundary Effects 177 Discontinuity Features 179 Prior-Mean Constraints 180 Statistical Models 182 Analytical Forms 184 Analytical Forms and Nonstationary Fields 188 Recursive / Dynamic Models 190 Banded Inverse-Covariances 191 Model Determination 193 Choice of Representation 196 Application 5: Synthetic Aperture Radar Interferometry [53] 197 For Further Study 199 Sample Problems 199 6 Markov Random Fields 203 One-Dimensional Markovianity 204 Markov Chains 205 Gauss--Markov Processes 205 Multidimensional Markovianity 206 Gauss–Markov Random Fields 209 Causal Gauss–Markov Random Fields 213 Gibbs Random Fields 216 Model Determination 223 Autoregressive Model Learning 223 Noncausal Markov Model Learning 225 Choices of Representation 231 Application 6: Texture Classification 232 Summary 235 For Further Study 236 Sample Problems 236 7 Hidden Markov Models 239 Hidden Markov Models 240 Image Denoising 240 Image Segmentation 243 Texture Segmentation 244 Edge Detection 245 Classes of Joint Markov Models 246 Conditional Random Fields 249 Discrete-State Models 251 Local Gibbs Models 252 Nonlocal Statistical-Target Models 253 Local Joint Models 254 Model Determination 255 Application 7: Image Segmentation 257 For Further Study 261 Sample Problems 261 8 Changes of Basis 265 Change of Basis 267 Reduction of Basis 271 Principal Components 272 Multidimensional Basis Reduction 277 Local Processing 283 FFT Methods 286 FFT Diagonalization 287 FFT and Spatial Models 289 FFT Sampling and Estimation 290 Hierarchical Bases and Preconditioners 293 Interpolated Hierarchical Bases 296 Wavelet Hierarchical Bases 297 Wavelets and Statistics 299 Basis Changes and Markov Random Fields 302 Basis Changes and Discrete-State Fields 305 Application 8: Global Data Assimilation [111] 309 Summary 311 For Further Study 312 Sample Problems 312 Part III Methods and Algorithms 316 9 Linear Systems Estimation 317 Direct Solution 319 Gaussian Elimination 319 Cholesky Decomposition 319 Nested Dissection 320 Iterative Solution 322 Gauss–Jacobi / Gauss–Seidel 323 Successive Overrelaxation (SOR) 327 Conjugate Gradient and Krylov Methods 330 Iterative Preconditioning 334 Multigrid 337 Application 9: Surface Reconstruction 344 For Further Study 345 Sample Problems 346 10 Kalman Filtering and Domain Decomposition 349 Marching Methods 351 Efficient, Large-State Kalman Filters 354 Large-State Kalman Smoother 355 Steady-State KF 358 Strip KF 358 Reduced-Update KF 360 Sparse KF 361 Reduced-Order KF 362 Multiscale 363 Application 10: Video Denoising [178] 371 Summary 374 For Further Study 375 Sample Problems 375 11 Sampling and Monte Carlo Methods 379 Dynamic Sampling 381 Static Sampling 382 FFT 385 Marching 386 Multiscale Sampling 387 MCMC 389 Stochastic Sampling 390 Continuous-State Sampling 393 Large-Scale Discrete-State Sampling 394 Nonparametric Sampling 398 Application 11: Multi-Instrument Fusion of Porous Media [236] 401 For Further Study 403 Sample Problems 403 Part IV Appendices 406 A Algebra 407 Linear Algebra 407 Matrix Operations 409 Matrix Positivity 412 Matrix Positivity of Covariances 413 Matrix Types 415 Matrix / Vector Derivatives 415 Matrix Transformations 419 Eigendecompositions 420 Singular Value Decomposition 424 Cholesky, Gauss, LU, Gram--Schmidt, QR, Schur 425 Matrix Square Roots 431 Pseudoinverses 433 B Statistics 435 Random Variables, Random Vectors, and Random Fields 435 Random Variables 435 Joint Statistics 436 Random Vectors 438 Random Fields 439 Transformation of Random Vectors 440 Multivariate Gaussian Distribution 441 Covariance Matrices 443 C Image Processing 447 Convolution 448 Image Transforms 452 Image Operations 454 Reference Summary 457 References 461 Index 475 1441972935,9781441972934 Springer Front Matter....Pages i-xxii Introduction....Pages 1-10 Front Matter....Pages 11-11 Inverse Problems....Pages 13-55 Static Estimation and Sampling....Pages 57-84 Dynamic Estimation and Sampling....Pages 85-129 Front Matter....Pages 131-131 Multidimensional Modelling....Pages 133-177 Markov Random Fields....Pages 179-214 Hidden Markov Models....Pages 215-239 Changes of Basis....Pages 241-290 Front Matter....Pages 291-291 Linear Systems Estimation....Pages 293-324 Kalman Filtering and Domain Decomposition....Pages 325-353 Sampling and Monte Carlo Methods....Pages 355-380 Front Matter....Pages 381-381 Algebra....Pages 383-409 Statistics....Pages 411-421 Image Processing....Pages 423-432 Back Matter....Pages 433-454