### Persönlicher Status und Werkzeuge

Home Research Image-based 3D Reconstruction Multi-View 3D Reconstruction

# Multi-View 3D Reconstruction

For a human, it is usually an easy task to get an idea of the 3D structure shown in an image. Due to the loss of one dimension in the projection process, the estimation of the true 3D geometry is difficult and a so called ill-posed problem, because usually infinitely many different 3D surfaces may produce the same set of images.

3D reconstruction of barley from 25 input images.

## 3D Reconstruction from multiple views

The goal of multiview 3D reconstruction is to infer geometrical structure of a scene captured by a collection of images. Usually the camera position and internal parameters are assumed to be known or they can be estimated from the set of images. By using multiple images, 3D information can be (partially) recovered by solving a pixel-wise correspondence problem. Since automatic correspondence estimation is usually ambiguous and incomplete further knowledge (prior knowledge) about the object is necessary. A typical prior is assume that the object surface is smooth.

Our research is focused on convex variational methods. The 3D reconstruction problem is formulated as an energy minimization problem. Due to the convexity of this energy, any (local) minimizer corresponds to the global minimum of this energy.

Minimizers of this energy are found with iterative numerical optimization methods which evolve the surface gradually from the initial surface to best one with respect to energy functional.
As a further consequence of the convexity, these methods are independent of the initialization. The initial surface can be of any shape, for example a simple box.

Two example reconstructions are shown below along with some of their corresponding input images.

## Spatio-Temporal 3D Reconstruction from multiple videos

Considering a dynamic scene that changes over time, 3D reconstruction can be applied to every time step independently. However, one can achieve temporally more consistent results by using the information from several time frames together, thus computing a spatio-temporal hyper-surface in 4D space.

The following video shows results of our latest publication on 4D reconstruction.

# Related publications

Sort Order:  by type by year
Book Chapters 2011 Book Chapters Convex Relaxation Techniques for Segmentation, Stereo and Multiview Reconstruction (D. Cremers, T. Pock, K. Kolev, A. Chambolle), Chapter in Markov Random Fields for Vision and Image Processing, MIT Press, 2011. [bib] [pdf] Book Chapters Fast Joint Estimation of Silhouettes and Dense 3D Geometry from Multiple Images (K. Kolev, T. Brox, D. Cremers), In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 34, 2012. [bib] [pdf] A Variational Approach to Vesicle Membrane Reconstruction from Fluorescence Imaging (K. Kolev, N. Kirchgessner, S. Houben, A. Csiszar, W. Rubner, C. Palm, B. Eiben, R. Merkel, D. Cremers), In Pattern Recognition, volume 44, 2011. [bib] [pdf] Multiview Stereo and Silhouette Consistency via Convex Functionals over Convex Domains (D. Cremers, K. Kolev), In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 33, 2011. [bib] [pdf] Continuous Global Optimization in Multiview 3D Reconstruction (K. Kolev, M. Klodt, T. Brox, D. Cremers), In International Journal of Computer Vision, volume 84, 2009. [bib] [pdf] Weighted Minimal Hypersurface Reconstruction (B. Goldluecke, I. Ihrke, C. Linz, M. Magnor), In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 29, 2007. [bib] [pdf] Book Chapters De-noising, Stabilizing and Completing 3D Reconstructions On-the-go using Plane Priors (M. Dzitsiuk, J. Sturm, R. Maier, L. Ma, D. Cremers), In International Conference on Robotics and Automation (ICRA), 2017. ([video]) [bib] [pdf] Generalized Connectivity Constraints for Spatio-temporal 3D Reconstruction (M. R. Oswald, J. Stühmer, D. Cremers), In European Conference on Computer Vision (ECCV), 2014. [bib] [pdf] [video] Surface Normal Integration for Convex Space-time Multi-view Reconstruction (M. R. Oswald, D. Cremers), In British Machine Vision Conference (BMVC), 2014. [bib] [pdf] [video] Spatial and Temporal Interpolation of Multi-View Image Sequences (T. Gurdan, M. R. Oswald, D. Gurdan, D. Cremers), In German Conference on Pattern Recognition (GCPR), volume 36, 2014. [bib] [pdf] [video] Volumetric 3D Mapping in Real-Time on a CPU (F. Steinbruecker, J. Sturm, D. Cremers), In Int. Conf. on Robotics and Automation, 2014. [bib] [pdf] Large-Scale Multi-Resolution Surface Reconstruction from RGB-D Sequences (F. Steinbruecker, C. Kerl, J. Sturm, D. Cremers), In IEEE International Conference on Computer Vision (ICCV), 2013. [bib] [pdf] A Convex Relaxation Approach to Space Time Multi-view 3D Reconstruction (M. R. Oswald, D. Cremers), In ICCV Workshop on Dynamic Shape Capture and Analysis (4DMOD), 2013. [bib] [pdf] Fast and Accurate Large-scale Stereo Reconstruction using Variational Methods (G. Kuschk, D. Cremers), In ICCV Workshop on Big Data in 3D Computer Vision, 2013. [bib] [pdf] Decoupling Photometry and Geometry in Dense Variational Camera Calibration (M. Aubry, K. Kolev, B. Goldluecke, D. Cremers), In IEEE International Conference on Computer Vision (ICCV), 2011. [bib] [pdf] Parallel Generalized Thresholding Scheme for Live Dense Geometry from a Handheld Camera (J. Stühmer, S. Gumhold, D. Cremers), In ECCV Workshop on Computer Vision on GPUs (CVGPU), 2010. [bib] [pdf] Real-Time Dense Geometry from a Handheld Camera (J. Stühmer, S. Gumhold, D. Cremers), In Pattern Recognition (Proc. DAGM), 2010. [bib] [pdf] Anisotropic Minimal Surfaces Integrating Photoconsistency and Normal Information for Multiview Stereo (K. Kolev, T. Pock, D. Cremers), In European Conference on Computer Vision (ECCV), 2010. [bib] [pdf] A Superresolution Framework for High-Accuracy Multiview Reconstruction (B. Goldluecke, D. Cremers), In Pattern Recognition (Proc. DAGM), 2009. [bib] [pdf]Received DAGM Best Paper Award Continuous Ratio Optimization via Convex Relaxation with Applications to Multiview 3D Reconstruction (K. Kolev, D. Cremers), In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009. [bib] [pdf] An Experimental Comparison of Discrete and Continuous Shape Optimization Methods (M. Klodt, T. Schoenemann, K. Kolev, M. Schikora, D. Cremers), In European Conference on Computer Vision (ECCV), 2008. [bib] [pdf] Integration of Multiview Stereo and Silhouettes via Convex Functionals on Convex Domains (K. Kolev, D. Cremers), In European Conference on Computer Vision (ECCV), 2008. [bib] [pdf] Continuous Global Optimization in Multiview 3D Reconstruction (K. Kolev, M. Klodt, T. Brox, S. Esedoglu, D. Cremers), In Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR), Springer, volume 4679, 2007. [bib] [pdf] Propagated Photoconsistency and Convexity in Variational Multiview 3D Reconstruction (K. Kolev, M. Klodt, T. Brox, D. Cremers), In Workshop on Photometric Analysis for Computer Vision, 2007. [bib] [pdf] Robust variational segmentation of 3D objects from multiple views (K. Kolev, T. Brox, D. Cremers), In Pattern Recognition (Proc. DAGM) (K. Franke et al., ed.), Springer, volume 4174, 2006. [bib] [pdf] Reconstructing the Geometry of Flowing Water (I. Ihrke, B. Goldluecke, M. Magnor), In IEEE International Conference on Computer Vision (ICCV), IEEE, 2005. [bib] [pdf] Spacetime-Continous Geometry Meshes from Multi-View Video Sequences (B. Goldluecke, M. Magnor), In Proceedings of the IEEE International Conference on Image Processing, IEEE Computer Society, 2005. [bib] [pdf] Space-Time Isosurface Evolution for Temporally Coherent 3D Reconstruction (B. Goldluecke, M. Magnor), In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, volume I, 2004. [bib] [pdf] Weighted Minimal Hypersurfaces and Their Applications in Computer Vision (B. Goldluecke, M. Magnor), In European Conference on Computer Vision (ECCV), Springer, volume 3022, 2004. [bib] [pdf] Spacetime-coherent Geometry Reconstruction from Multiple Video Streams (M. Magnor, B. Goldluecke), In 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT) (Y. Aloimonos, ed.), IEEE, 2004. [bib] [pdf] Joint 3D Reconstruction and Background Separation in Multiple Views using Graph Cuts (B. Goldluecke, M. Magnor), In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, volume I, 2003. [bib] [pdf]