Direkt zum Inhalt springen
Computer Vision Group
TUM Department of Informatics
Technical University of Munich

Technical University of Munich

Menu
Home Data Datasets Photometric Depth Super-Resolution Dataset

Photometric Depth Super-Resolution Dataset

Photometric Depth Super-Resolution

Bjoern Haefner1    Songyou Peng2    Alok Verma1    Yvain Quéau3    Daniel Cremers1
1Technical University of Munich    2University of Illinois at Urbana-Champaign    3GREYC, UMR CNRS 6072

Submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) Special Issue on RGB-D Vision: Methods and Applications

This study explores the use of photometric techniques (shape-from-shading and uncalibrated photometric stereo) for upsampling the low-resolution depth map from an RGB-D sensor to the higher resolution of the companion RGB image. A single-shot variational approach is first put forward, which is effective as long as the target's reflectance is piecewise-constant. It is then shown that this dependency upon a specific reflectance model can be relaxed by focusing on a specific class of objects (e.g., faces), and delegate reflectance estimation to a deep neural network. A multi-shot strategy based on randomly varying lighting conditions is eventually discussed. It requires no training or prior on the reflectance, yet this comes at the price of a dedicated acquisition setup. Both quantitative and qualitative evaluations illustrate the effectiveness of the proposed methods on synthetic and real-world scenarios.

Code

The code that generated the data shown here is available on github:
https://github.com/BjoernHaefner/DepthSRfromShading
https://github.com/pengsongyou/SRmeetsPS.

Dataset

The following dataset contains RGB-D sequences and reconstructed 3D models of multiple different scenes. We captured the RGB-D data under different scaling factors using an Asus Xtion Pro and an Intel RealSense D415 RGB-D sensor. Please refer to the respective publication when using this data.

Format

For each scene of the Photometric Depth Super-Resolution dataset, we provide the respective RGB-D sequence as well as the refined 3D models. Each RGB-D sequence contains:

  • Color frames
  • Depth frames
  • Masks
  • Intrinsic parameters (default factory calibration).

Rucksack

Android

Download (mat-files):

Download (obj-files):

Basecap

Download (mat-files):

Download (obj-files):

Minion

Download (mat-files):

Download (obj-files):

Blanket

Clothes

Monkey

Wool

Face 1

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Face 2

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Face 3

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Face 4

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Face 5

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Face 6

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Tabletcase

Shirt

Backpack

Download (mat-files):

Download (obj-files):

Ovenmitt

Hat

Download (mat-files):

Download (obj-files):

Vase

Download (mat-files):

Download (obj-files):

License

Unless stated otherwise, all data in the Photometric Depth Super-Resolution Dataset is licensed under a Creative Commons 4.0 Attribution License (CC BY-NC-SA 4.0).


Export as PDF, TEX or BIB

Journal Articles
2019
[]Photometric Depth Super-Resolution (Haefner, B., Peng, S., Verma, A., Quéau, Y. and Cremers, D.), In IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2019. ([supp] [project page] [ieeexplore]) [bibtex] [arXiv:1809.10097] [pdf]
Conference and Workshop Papers
2018
[]Fight ill-posedness with ill-posedness: Single-shot variational depth super-resolution from shading (Haefner, B., Quéau, Y., Möllenhoff, T. and Cremers, D.), In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. ([supp] [poster] [code] [cvf] [video]) [bibtex] [pdf] [video]Spotlight Presentation
2017
[]Depth Super-Resolution Meets Uncalibrated Photometric Stereo (Peng, S., Haefner, B., Quéau, Y. and Cremers, D.), In International Conference on Computer Vision Workshops (ICCVW), 2017. ([code] [cvf]) [bibtex] [arXiv:1708.00411] [pdf]Oral Presentation at ICCV Workshop on Color and Photometry in Computer Vision
Powered by bibtexbrowser
Export as PDF, TEX or BIB

Rechte Seite

Informatik IX
Chair of Computer Vision & Artificial Intelligence

Boltzmannstrasse 3
85748 Garching

info@vision.in.tum.de