Persönlicher Status und Werkzeuge

Home Research Image-based 3D Reconstruction Multi-View 3D Reconstruction

Multi-View 3D Reconstruction

Contact: Martin Oswald, Maria Klodt, Jörg Stückler, Prof. Dr. Daniel Cremers


For a human, it is usually an easy task to get an idea of the 3D structure shown in an image. Due to the loss of one dimension in the projection process, the estimation of the true 3D geometry is difficult and a so called ill-posed problem, because usually infinitely many different 3D surfaces may produce the same set of images.


3D reconstruction of barley from 25 input images.

3D Reconstruction from multiple views

The goal of multiview 3D reconstruction is to infer geometrical structure of a scene captured by a collection of images. Usually the camera position and internal parameters are assumed to be known or they can be estimated from the set of images. By using multiple images, 3D information can be (partially) recovered by solving a pixel-wise correspondence problem. Since automatic correspondence estimation is usually ambiguous and incomplete further knowledge (prior knowledge) about the object is necessary. A typical prior is assume that the object surface is smooth.



Our research is focused on convex variational methods. The 3D reconstruction problem is formulated as an energy minimization problem. Due to the convexity of this energy, any (local) minimizer corresponds to the global minimum of this energy.

Minimizers of this energy are found with iterative numerical optimization methods which evolve the surface gradually from the initial surface to best one with respect to energy functional.
As a further consequence of the convexity, these methods are independent of the initialization. The initial surface can be of any shape, for example a simple box.






Two example reconstructions are shown below along with some of their corresponding input images.




Spatio-Temporal 3D Reconstruction from multiple videos

Considering a dynamic scene that changes over time, 3D reconstruction can be applied to every time step independently. However, one can achieve temporally more consistent results by using the information from several time frames together, thus computing a spatio-temporal hyper-surface in 4D space.

The following video shows results of our latest publication on 4D reconstruction.




Related publications

Sort Order:  by type by year
Book Chapters Journal Articles Conference and Workshop Papers 
Book Chapters
2011
Convex Relaxation Techniques for Segmentation, Stereo and Multiview Reconstruction (D. Cremers, T. Pock, K. Kolev, A. Chambolle), Chapter in Markov Random Fields for Vision and Image Processing, MIT Press, 2011. [bib] [pdf]
Book Chapters Journal Articles Conference and Workshop Papers 
Journal Articles
2012
Fast Joint Estimation of Silhouettes and Dense 3D Geometry from Multiple Images (K. Kolev, T. Brox, D. Cremers), In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 34, 2012. [bib] [pdf]
2011
A Variational Approach to Vesicle Membrane Reconstruction from Fluorescence Imaging (K. Kolev, N. Kirchgessner, S. Houben, A. Csiszar, W. Rubner, C. Palm, B. Eiben, R. Merkel, D. Cremers), In Pattern Recognition, volume 44, 2011. [bib] [pdf]
Multiview Stereo and Silhouette Consistency via Convex Functionals over Convex Domains (D. Cremers, K. Kolev), In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 33, 2011. [bib] [pdf]
2009
Continuous Global Optimization in Multiview 3D Reconstruction (K. Kolev, M. Klodt, T. Brox, D. Cremers), In International Journal of Computer Vision, volume 84, 2009. [bib] [pdf]
2007
Weighted Minimal Hypersurface Reconstruction (B. Goldluecke, I. Ihrke, C. Linz, M. Magnor), In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 29, 2007. [bib] [pdf]
Book Chapters Journal Articles Conference and Workshop Papers 
Conference and Workshop Papers
2016
De-noising, Stabilizing and Completing 3D Reconstructions On-the-go using Plane Priors (M. Dzitsiuk, J. Sturm, R. Maier, L. Ma, D. Cremers), In arXiv:1609.08267, 2016. ([video]) [bib] [pdf]
2014
Generalized Connectivity Constraints for Spatio-temporal 3D Reconstruction (M. R. Oswald, J. Stühmer, D. Cremers), In European Conference on Computer Vision (ECCV), 2014. [bib] [pdf] [video]
Surface Normal Integration for Convex Space-time Multi-view Reconstruction (M. R. Oswald, D. Cremers), In British Machine Vision Conference (BMVC), 2014. [bib] [pdf] [video]
Spatial and Temporal Interpolation of Multi-View Image Sequences (T. Gurdan, M. R. Oswald, D. Gurdan, D. Cremers), In German Conference on Pattern Recognition (GCPR), volume 36, 2014. [bib] [pdf] [video]
Volumetric 3D Mapping in Real-Time on a CPU (F. Steinbruecker, J. Sturm, D. Cremers), In Int. Conf. on Robotics and Automation, 2014. [bib] [pdf]
2013
Large-Scale Multi-Resolution Surface Reconstruction from RGB-D Sequences (F. Steinbruecker, C. Kerl, J. Sturm, D. Cremers), In IEEE International Conference on Computer Vision (ICCV), 2013. [bib] [pdf]
A Convex Relaxation Approach to Space Time Multi-view 3D Reconstruction (M. R. Oswald, D. Cremers), In ICCV Workshop on Dynamic Shape Capture and Analysis (4DMOD), 2013. [bib] [pdf]
Fast and Accurate Large-scale Stereo Reconstruction using Variational Methods (G. Kuschk, D. Cremers), In ICCV Workshop on Big Data in 3D Computer Vision, 2013. [bib] [pdf]
2011
Decoupling Photometry and Geometry in Dense Variational Camera Calibration (M. Aubry, K. Kolev, B. Goldluecke, D. Cremers), In IEEE International Conference on Computer Vision (ICCV), 2011. [bib] [pdf]
2010
Parallel Generalized Thresholding Scheme for Live Dense Geometry from a Handheld Camera (J. Stühmer, S. Gumhold, D. Cremers), In ECCV Workshop on Computer Vision on GPUs (CVGPU), 2010. [bib] [pdf]
Real-Time Dense Geometry from a Handheld Camera (J. Stühmer, S. Gumhold, D. Cremers), In Pattern Recognition (Proc. DAGM), 2010. [bib] [pdf]
Anisotropic Minimal Surfaces Integrating Photoconsistency and Normal Information for Multiview Stereo (K. Kolev, T. Pock, D. Cremers), In European Conference on Computer Vision (ECCV), 2010. [bib] [pdf]
2009
A Superresolution Framework for High-Accuracy Multiview Reconstruction (B. Goldluecke, D. Cremers), In Pattern Recognition (Proc. DAGM), 2009. [bib] [pdf]Received DAGM Best Paper Award
Continuous Ratio Optimization via Convex Relaxation with Applications to Multiview 3D Reconstruction (K. Kolev, D. Cremers), In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009. [bib] [pdf]
2008
An Experimental Comparison of Discrete and Continuous Shape Optimization Methods (M. Klodt, T. Schoenemann, K. Kolev, M. Schikora, D. Cremers), In European Conference on Computer Vision (ECCV), 2008. [bib] [pdf]
Integration of Multiview Stereo and Silhouettes via Convex Functionals on Convex Domains (K. Kolev, D. Cremers), In European Conference on Computer Vision (ECCV), 2008. [bib] [pdf]
2007
Continuous Global Optimization in Multiview 3D Reconstruction (K. Kolev, M. Klodt, T. Brox, S. Esedoglu, D. Cremers), In Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR), Springer, volume 4679, 2007. [bib] [pdf]
Propagated Photoconsistency and Convexity in Variational Multiview 3D Reconstruction (K. Kolev, M. Klodt, T. Brox, D. Cremers), In Workshop on Photometric Analysis for Computer Vision, 2007. [bib] [pdf]
2006
Robust variational segmentation of 3D objects from multiple views (K. Kolev, T. Brox, D. Cremers), In Pattern Recognition (Proc. DAGM) (K. Franke et al., ed.), Springer, volume 4174, 2006. [bib] [pdf]
2005
Reconstructing the Geometry of Flowing Water (I. Ihrke, B. Goldluecke, M. Magnor), In IEEE International Conference on Computer Vision (ICCV), IEEE, 2005. [bib] [pdf]
Spacetime-Continous Geometry Meshes from Multi-View Video Sequences (B. Goldluecke, M. Magnor), In Proceedings of the IEEE International Conference on Image Processing, IEEE Computer Society, 2005. [bib] [pdf]
2004
Space-Time Isosurface Evolution for Temporally Coherent 3D Reconstruction (B. Goldluecke, M. Magnor), In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, volume I, 2004. [bib] [pdf]
Weighted Minimal Hypersurfaces and Their Applications in Computer Vision (B. Goldluecke, M. Magnor), In European Conference on Computer Vision (ECCV), Springer, volume 3022, 2004. [bib] [pdf]
Spacetime-coherent Geometry Reconstruction from Multiple Video Streams (M. Magnor, B. Goldluecke), In 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT) (Y. Aloimonos, ed.), IEEE, 2004. [bib] [pdf]
2003
Joint 3D Reconstruction and Background Separation in Multiple Views using Graph Cuts (B. Goldluecke, M. Magnor), In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, volume I, 2003. [bib] [pdf]
Powered by bibtexbrowser
Export as PDF or BIB

Last edited 20.05.2015 00:43 by Jörg Stückler