Direkt zum Inhalt springen
Computer Vision Group
TUM Department of Informatics
Technical University of Munich

Technical University of Munich

Menu

Links


Photometric Depth Super-Resolution Dataset

Photometric Depth Super-Resolution

Bjoern Haefner1 Songyou Peng2 Alok Verma1 Yvain Quéau3 Daniel Cremers1
1Technical University of Munich 2University of Illinois at Urbana-Champaign 3GREYC, UMR CNRS 6072

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) Special Issue on RGB-D Vision: Methods and Applications

This study explores the use of photometric techniques (shape-from-shading and uncalibrated photometric stereo) for upsampling the low-resolution depth map from an RGB-D sensor to the higher resolution of the companion RGB image. A single-shot variational approach is first put forward, which is effective as long as the target's reflectance is piecewise-constant. It is then shown that this dependency upon a specific reflectance model can be relaxed by focusing on a specific class of objects (e.g., faces), and delegate reflectance estimation to a deep neural network. A multi-shot strategy based on randomly varying lighting conditions is eventually discussed. It requires no training or prior on the reflectance, yet this comes at the price of a dedicated acquisition setup. Both quantitative and qualitative evaluations illustrate the effectiveness of the proposed methods on synthetic and real-world scenarios.

Code

The code that generated the data shown here is available on github:
https://github.com/BjoernHaefner/DepthSRfromShading
https://github.com/pengsongyou/SRmeetsPS.

Dataset

The following dataset contains RGB-D sequences and reconstructed 3D models of multiple different scenes. We captured the RGB-D data under different scaling factors using an Asus Xtion Pro and an Intel RealSense D415 RGB-D sensor. Please refer to the respective publication when using this data.

Format

For each scene of the Photometric Depth Super-Resolution dataset, we provide the respective RGB-D sequence as well as the refined 3D models. Each RGB-D sequence contains:

  • Color frames
  • Depth frames
  • Masks
  • Intrinsic parameters (default factory calibration).

Rucksack

Android

Download (mat-files):

Download (obj-files):

Basecap

Download (mat-files):

Download (obj-files):

Minion

Download (mat-files):

Download (obj-files):

Blanket

Clothes

Monkey

Wool

Face 1

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Face 2

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Face 3

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Face 4

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Face 5

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Face 6

Download (mat-files):

Download (png-file of deep net albedo estimate):

Download (obj-files):

Tabletcase

Shirt

Backpack

Download (mat-files):

Download (obj-files):

Ovenmitt

Hat

Download (mat-files):

Download (obj-files):

Vase

Download (mat-files):

Download (obj-files):

License

Unless stated otherwise, all data in the Photometric Depth Super-Resolution Dataset is licensed under a Creative Commons 4.0 Attribution License (CC BY-NC-SA 4.0).


Export as PDF, XML, TEX or BIB

Journal Articles
2020
[]Photometric Depth Super-Resolution (B. Haefner, S. Peng, A. Verma, Y. Quéau and D. Cremers), In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 42, 2020. ([supp] [project page] [ieeexplore]) [bibtex] [arXiv:1809.10097] [pdf]
Conference and Workshop Papers
2018
[]Fight ill-posedness with ill-posedness: Single-shot variational depth super-resolution from shading (B. Haefner, Y. Quéau, T. Möllenhoff and D. Cremers), In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. ([supp] [poster] [code] [cvf] [video]) [bibtex] [pdf] [video]Spotlight Presentation
2017
[]Depth Super-Resolution Meets Uncalibrated Photometric Stereo (S. Peng, B. Haefner, Y. Quéau and D. Cremers), In International Conference on Computer Vision Workshops (ICCVW), 2017. ([code] [cvf]) [bibtex] [arXiv:1708.00411] [pdf]Oral Presentation at ICCV Workshop on Color and Photometry in Computer Vision
Powered by bibtexbrowser
Export as PDF, XML, TEX or BIB

Rechte Seite

Informatik IX
Chair of Computer Vision & Artificial Intelligence

Boltzmannstrasse 3
85748 Garching info@vision.in.tum.de

Follow us on:
CVG Group DVL Group

News

04.06.2021

Bernt Schiele (Max Planck Institute for Informatics) will give a talk in the TUM AI lecture series on June 10th, 3pm! Livestream

05.05.2021
French-German Machine Learning Symposium

French-German Machine Learning Symposium

The French-German Machine Learning Symposium aims to strengthen interactions and inspire collaborations between both countries. We invited some of the leading ML researchers from France and Germany to this two-day symposium to give a glimpse into their research, and engage in discussions on the future of machine learning and how to strengthen research collaborations in ML between France and Germany.

The list of speakers includes Yann LeCun, Cordelia Schmid, Jean-Bernard Lasserre, Bernhard Schölkopf, and many more! For the full program please visit the webpage.

03.05.2021

Ron Kimmel (Technion - Israel Institute of Technology) will give a talk in the TUM AI lecture series on May 6th, 3pm! Livestream

23.04.2021

4Seasons Dataset: We have released a novel dataset for benchmarking multi-weather SLAM in autonomous driving.

19.04.2021

Hao Li (Pinscreen) will give a talk in the TUM AI lecture series on April 22nd, 8pm! Livestream

More