Direkt zum Inhalt springen
Computer Vision Group
TUM Department of Informatics
Technical University of Munich

Technical University of Munich

Menu

Links


Photometric Depth Super-Resolution Dataset

Photometric Depth Super-Resolution

Bjoern Haefner1 Songyou Peng2 Alok Verma1 Yvain Quéau3 Daniel Cremers1
1Technical University of Munich 2University of Illinois at Urbana-Champaign 3GREYC, UMR CNRS 6072

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) Special Issue on RGB-D Vision: Methods and Applications

This study explores the use of photometric techniques (shape-from-shading and uncalibrated photometric stereo) for upsampling the low-resolution depth map from an RGB-D sensor to the higher resolution of the companion RGB image. A single-shot variational approach is first put forward, which is effective as long as the target's reflectance is piecewise-constant. It is then shown that this dependency upon a specific reflectance model can be relaxed by focusing on a specific class of objects (e.g., faces), and delegate reflectance estimation to a deep neural network. A multi-shot strategy based on randomly varying lighting conditions is eventually discussed. It requires no training or prior on the reflectance, yet this comes at the price of a dedicated acquisition setup. Both quantitative and qualitative evaluations illustrate the effectiveness of the proposed methods on synthetic and real-world scenarios.

Code

The code that generated the data shown here is available on github:
https://github.com/BjoernHaefner/DepthSRfromShading
https://github.com/pengsongyou/SRmeetsPS.

Dataset

The following dataset contains RGB-D sequences and reconstructed 3D models of multiple different scenes. We captured the RGB-D data under different scaling factors using an Asus Xtion Pro and an Intel RealSense D415 RGB-D sensor. Please refer to the respective publication when using this data.

Format

For each scene of the Photometric Depth Super-Resolution dataset, we provide the respective RGB-D sequence as well as the refined 3D models. Each RGB-D sequence contains:

  • Color frames
  • Depth frames
  • Masks
  • Intrinsic parameters (default factory calibration).

Rucksack

Android

Download (mat-files):

Download (obj-files):

Basecap

Download (mat-files):

Download (obj-files):

Minion

Download (mat-files):

Download (obj-files):

Blanket

Clothes

Monkey

Wool

Face 1

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Face 2

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Face 3

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Face 4

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Face 5

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Face 6

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Tabletcase

Shirt

Backpack

Download (mat-files):

Download (obj-files):

Ovenmitt

Hat

Download (mat-files):

Download (obj-files):

Vase

Download (mat-files):

Download (obj-files):

License

Unless stated otherwise, all data in the Photometric Depth Super-Resolution Dataset is licensed under a Creative Commons 4.0 Attribution License (CC BY-NC-SA 4.0).


Export as PDF, XML, TEX or BIB

Journal Articles
2020
[]Photometric Depth Super-Resolution (B. Haefner, S. Peng, A. Verma, Y. Quéau and D. Cremers), In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 42, 2020. ([supp] [project page] [ieeexplore]) [bibtex] [arXiv:1809.10097] [pdf]
Conference and Workshop Papers
2018
[]Fight ill-posedness with ill-posedness: Single-shot variational depth super-resolution from shading (B. Haefner, Y. Quéau, T. Möllenhoff and D. Cremers), In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. ([supp] [poster] [code] [cvf] [video]) [bibtex] [pdf] [video]Spotlight Presentation
2017
[]Depth Super-Resolution Meets Uncalibrated Photometric Stereo (S. Peng, B. Haefner, Y. Quéau and D. Cremers), In International Conference on Computer Vision Workshops (ICCVW), 2017. ([code] [cvf]) [bibtex] [arXiv:1708.00411] [pdf]Oral Presentation at ICCV Workshop on Color and Photometry in Computer Vision
Powered by bibtexbrowser
Export as PDF, XML, TEX or BIB

Rechte Seite

Informatik IX
Chair of Computer Vision & Artificial Intelligence

Boltzmannstrasse 3
85748 Garching info@vision.in.tum.de

Follow us on:
CVG Group DVL Group

News

07.04.2021

Thomas Pock (TU Graz) will give a talk in the TUM AI lecture series on April 15th, 3pm! Livestream

26.03.2021

Max Welling (University of Amsterdam) will give a talk in the TUM AI lecture series on April 1st, 3pm! Livestream

02.03.2021

We have seven papers (4 orals, 3 posters) accepted to CVPR 2021!

02.03.2021

We have two papers accepted to ICRA 2021!

01.03.2021

Maks Ovsjanikov (Ecole Polytechnique) will give a talk in the TUM AI lecture series on March 11th, 3pm! Livestream

More