3D Structure from Multiple Images of Large-Scale Environments: European Workshop, SMILE'98, Freiburg, Germany, June 6-7, 1998, Proceedings (Lecture Notes in Computer Science, 1506)

The contributions in this volume give an overview of state-of-the-art results presented at the Workshop on 3D Structure from Multiple Images of Lar- scale Environments (SMILE). This workshop was held in conjunction with the Fifth European Conference on Computer Vision 1998 in Freiburg, Germany. SMILE was a joint e ort of the European ACTS projects VANGUARD and PANORAMA and the Esprit projectCUMULI, all of which are involved in the analysis and reconstruction of 3D scenes from image sequences. The potential for 3D reconstructions of scenes and objects is tremendous. Much of the work reported here is to be seen especially against the background of a convergence between computer vision and computer graphics, and of a shift from signal-based to content-based image analysis in telecommunications. - cordingly, the requirementsfor 3D modelsand acquisitionsystemsarealso shi- ing. Visualization rather than mensuration is the primary issue. The perceptual qualityofthe models, the?exibility ofthe acquisition, andthe costofthe system are three driving forces in the search for new methods. The last few years have seen important steps toward genuine ?exibility. A case in point is the use of multiple images to generate 3D models, without an explicit knowledgeof the relativeposition of the camerasor the camera settings. The same developments also hold good promise to make 3D acquisition cheaper and more widely available. front-matter 3D Structure from Multiple Images of Large-Scale Environments Preface Table of Contents fulltext The {sc Cumuli} Project Multi-camera Geometry, Discrete Images 3D Perception from Continuous Image Sequences Algebraic Geometric Reasoning from Image Data The {sc Panorama} Project Goals of {sc Panorama} 3-D Reconstruction Approach in {sc Panorama} Telepresence Visualization The {sc Vanguard} Project Approach Applications References fulltext2 Introduction Carlsson Duality Reconstruction Ambiguity Dual Algorithms Justification of the Algorithm. Refinements to the Dual Algorithm Experimental Performance Conclusion References fulltext3 Introduction Review of the Affine Camera Model Unifying 2-View and 3-View Geometry of Points Two-View Constraints Three-View Constraints Euclidean Motion/Structure from 3 Calibrated Affine Views Comparison with Related Work Unifying 3-View Geometry of Points and Lines Trifocal Tensor of Lines Trifocal Tensor of Points Some Minimal Configurations Experimental Results Conclusion Acknowledgements References fulltext4 Introduction Notations Tensor Embedding of the Fundamental Matrix Bifocal Tensor Contractions The Primitive Homography Matrices Applications Using Primitive Homography Matrices Quasi-Metric Reference Plane Triangulation from 3 Views Other Applications of the Bifocal Tensor Representation View-Synthesis Ego-motion Recovery Summary Acknowledgments References Appendix Trilinearities and the Trifocal Tensor Contraction Properties and Tensor Slices fulltext5 Introduction and Motivation Optimal Geometric Fitting Direct Approach Reduced Approach Parametrizing the Inter-image Geometry Redundant Parametrizations with Gauge Freedom Constrained Parametrizations Local Coordinate Patches / Minimal Parametrizations Library Architecture and Numerical Methods Numerical Methods for Gauge Freedom Numerical Methods for Constrained Optimization Search Control Robustification Measuring Performance Summary References fulltext6 Introduction Recapitulation of the Two View Case Failure of Small Baseline Algorithm Viewpoint Invariant Affinity Measure Wide Baseline Stereo Matching Algorithm Extension to Three Views: Trifocal Tensor Estimation Results Direct Estimation of View Relations from Homographies Using Planes to Compute the Fundamental Matrix Using Planes to Compute the Trifocal Tensor Results Matching and Reconstruction over Multiple Views Discussion References fulltext7 Introduction Block-Based Disparity Estimation Definition of Image Block Transformation Maximum-Likelihood (ML) Disparity Estimator Local Disparity Estimation Error Variance Distribution Disparity Estimation with Eccentric Image Blocks Experimental Results Conclusions References fulltext8 Introduction and Motivation Organization of the Paper Preliminaries Depth Estimation from Motion Fields Estimation of Viewing Geometry and Segmentation Algorithm Description Patch Segmentation Algorithm Analysis The Epipolar Constraint Self-Calibration Experimental Results Conclusions References fulltext9 1. Introduction 2. Calibration Calibration Strategy Adaptive Calibration 3. Partial Reconstruction Edge-Based Approach Area-Based Approach 4. Motion Estimation through Line Matching Egomotion from Straight Lines Egomotion Refinement from Curved Contours Examples of Application 5. Conclusions References fulltext10 Introduction Overview of the Method Notations Projective Reconstruction Relating the Images Initial Reconstruction Adding a View Self-Calibration Initial Calibration Refined Calibration Dense Depth Estimation Rectification Dense Stereo Matching Multiview Matching Building the Model Conclusion References fulltext11 Introduction The Projective Geometry of Single Axis Motion 3D Geometry Image Fixed Entities Camera Matrices Reconstruction Ambiguity Two-View Geometry Three View Geometry Removing the Reconstruction Ambiguity Estimation of Camera Matrices Algorithm Summary Point Tracking F Estimation T Estimation Bundle Adjustment Space Carving and Surface Rendering Conclusions References fulltext12 Introduction Previous Work General Problem Formulation Structure from Motion with Planes Fundamental Matrices from Point Correspondences Hallucinating Additional Correspondences Reprojecting Points Based on Homographies Fundamental Matrices from Homographies Plane plus Parallax Global Optimization (Bundle Adjustment) Constraints on Planes Experiments Discussion and Conclusions Future Work References fulltext13 Introduction Single View Geometry by Affine Shape and Depth Multiple Views Main Theorem The Chasles Matrix Relation to Fundamental Matrices and Multilinear Forms Proximity Measures Algorithms for Projective Reconstruction Complete Data, No Occlusions Missing Data, Occlusions Using Subsidiary Information Coplanarities Parallelity Known Focii Locations Appendix Affine Shape and Depth Subspace Formulation Matrix Formulation Relation to Grassman Manifolds Projective Transformations and Spaces Background Shape, Depth, and Projective Transformations Location of Focal Point Depth, Shape, and Camera Matrices fulltext14 Introduction Geometry on the Reference Plane Reference Plane Coordinate System Projection on the Reference Plane Epipolar Geometry on the Reference Plane Dual Geometry on Reference Plane Using Affine Normalization Stratified Reconstruction Experiments Medium Depth Sequence Large Depth Sequence Lab Sequence Dense Height Maps Discussion Summary References fulltext15 Introduction Geometry and Symbolic Algebra Computation An Introduction to the Ritt-Wu's Method Constraints and Parametrization of a Geometric Model Algorithm Experiment Conclusion Appendix References fulltext16 Introduction 3D Modeling from Panoramas Interactive Modeling System Modeling Steps Recovering Camera Pose Estimating Plane Normals Estimating the 3D Model 3D Modeling Using Layered Stereo Overview of Layered Stereo Approach Estimation of Plane Equations Estimation of Layer Sprites Estimation of Residual Depth Pixel Assignment to Layers Layer Refinement by Re-synthesis Experiments Discussion and Conclusions fulltext17 Introduction Presentation of the Context Definition of Inconsistency Consistency Processing Overview of the Algorithm Inconsistency Evaluation and Outlier Identification Multiview Fusion Geometry and Texture Representation Results and Conclusion fulltext18 Introduction Overview Definition of the Implicit Surface Distance Function Distance Transformation Generating the Template Mesh Improving the Mesh Topology Deletion of Short Edges Deletion of Redundant Points Subdivision Edge Swapping Improving Vertex Positions A Fast Approach: Smooth and Reproject GDM Results fulltext19 Introduction Reality Models for Exterior Construction Applications AR Applications in Exterior Construction Projects Reality Models Use of Existing Models Manual Approach Interactive Approach Towards Automatically Generated Models Range Data Models Use of Reality Models for Camera Calibration Augmenting Reality Occlusion Handling Using Geometric Reality Models or Depth Maps Simulation of Illumination Effects and Physical Constraints Future Work Leaving Reality Behind Diminishing Reality Conclusions Acknowledgments Sources of Graphical Material References fulltext20 Introduction System Requirements Analysis Display Technology Tracking Networking Collaborative Design Application Overview of the Augmented Reality System User Interface Calibration/Tracking Adding of Virtual Objects to the Real World Handling of Real - Virtual Object Interactions Distributing Augmented Reality Conclusion and Future Work Acknowledgement References fulltext21 Introduction The Perspective Camera Model Relation Between Two Views: The Epipolar Constraint Relations Between Three Views: The Trifocal Constraints The Fundamental Trifocal Constraint The Trifocal Constraint for Corresponding Lines The Trifocal Constraints for Corresponding Points The Trifocal Constraints as Incidence Relations and as Transfer Principle The Connection Between Epipolar and Trifocal Constraints Relations Between Four Views: The Quadrifocal Constraints Relations Between n Views: The General Picture Relations Between n geq 5 Views Have All Relations Between n leq 4 Views Been Found? back-matter Author Index The Contributions In This Volume Give An Overview Of State-of-the-art Results Presented At The Workshop On 3d Structure From Multiple Images Of Lar- Scale Environments (smile). This Workshop Was Held In Conjunction With The Fifth European Conference On Computer Vision 1998 In Freiburg, Germany. Smile Was A Joint E Ort Of The European Acts Projects Vanguard And Panorama And The Esprit Projectcumuli, All Of Which Are Involved In The Analysis And Reconstruction Of 3d Scenes From Image Sequences. The Potential For 3d Reconstructions Of Scenes And Objects Is Tremendous. Much Of The Work Reported Here Is To Be Seen Especially Against The Background Of A Convergence Between Computer Vision And Computer Graphics, And Of A Shift From Signal-based To Content-based Image Analysis In Telecommunications. - Cordingly,the Requirementsfor 3d Modelsand Acquisitionsystemsarealso Shi- Ing. Visualization Rather Than Mensuration Is The Primary Issue. The Perceptual Qualityofthe Models,the?exibility Ofthe Acquisition,andthe Costofthe System Are Three Driving Forces In The Search For New Methods. The Last Few Years Have Seen Important Steps Toward Genuine ?exibility. A Case In Point Is The Use Of Multiple Images To Generate 3d Models, Without An Explicit Knowledgeof The Relativeposition Of The Camerasor The Camera Settings. The Same Developments Also Hold Good Promise To Make 3d Acquisition Cheaper And More Widely Available. Cumuli, Panorama, And Vanguard Project Overview / Ralf Buschmann [and Others] -- Dualizing Scene Reconstruction Algorithms / Richard Hartley And Gilles Debunne -- Geometry Of Multiple Affine Views / Long Quan, Yuichi Ohta, And Roger Mohr -- Tensor Embedding Of The Fundamental Matrix / Shai Avidan And Amnon Shashua -- Optimal Estimation Of Matching Constraints / Bill Triggs -- Matching And Reconstruction From Widely Separated Views / Philip Pritchett And Andrew Zisserman -- Improving Block-based Disparity Estimation By Considering The Non-uniform Distribution Of The Estimation Error / Lutz Falkenhagen And Thomas Wedi -- Beyond The Epipolar Constraint : Integrating 3d Motion And Structure Estimation / Tomáš Brodský, Cornelia Fermüller, And Yiannis Aloimonos -- Multi-camera Acquisitions For High-accuracy 3d Reconstruction / Federico Pedersini, Augusto Sarti, And Stefano Tubaro -- Metric 3d Surface Reconstruction From Uncalibrated Image Sequences / Marc Pollefeys [and Others] --^ Automatic 3d Model Construction For Turn-table Sequences / Andrew W. Fitzgibbon, Geoff Cross, And Andrew Zisserman -- Geometrically Constrained Structure From Motion : Points On Planes / Richard Szeliski And P.h.s. Torr -- Euclidean And Affine Structure/motion For Uncalibrated Cameras From Affine Shape And Subsidiary Information / Gunnar Sparr -- From Ordinal To Euclidean Reconstruction With Partial Scene Calibration / Daphna Weinshall, P. Anandan, And Michal Irani -- Imposing Euclidean Constraints During Self-calibration Processes / Didier Bondyfalat And Sylvain Bougnoux -- Interactive 3d Modeling From Multiple Images Using Scene Regularities / Heung-yeung Shum [and Others] -- Integration Of Multiple Range Maps Through Consistency Processing / Philippe Robert And Damien Minaud -- Fitting Geometrical Deformable Models To Registered Range Images / Stefan Grosskopf And Peter J. Neugebauer --^ The Use Of Reality Models In Augmented Reality Applications / Gudrun Klinker, Didier Stricker, And Dirk Reiners -- Applying Augmented Reality Techniques In The Field Of Interactive Collaborative Design / Hagen Schumann, Silviu Burtescu, And Frank Siering -- A Guided Tour Through Multiview Relations / Theo Moons. Reinhard Koch, Luc Van Gool (eds.). Held In Conjunction With The Fifth European Conference On Computer Vision, 1998, In Freiburg, Germany. Includes Bibliographical References And Index. This book constitutes the strictly refereed post-workshop proceedings of the European Workshop on 3D Structure from Multiple Images of Large-Scale Environments, SMILE'98, held in conjunction with ECCV'98 in Freiburg, Germany, in June 1998. The 21 revised full papers presented went through two cycles of reviewing and were carefully selected for inclusion in the book. The papers are organized in sections on multiview relations and correspondence search, 3D structure from multiple images, callibration and reconstruction using scene constraints, range integration and augmented reality application