Direkt zum Inhalt springen
Computer Vision Group
TUM Department of Informatics
Technical University of Munich

Technical University of Munich

Menu

Links


TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo

Authors: Lukas Koestler*, Nan Yang*, Niclas Zeller, Daniel Cremers

For up-to-date information, code, and data please visit our GitHub repository.

Abstract

In this paper, we present TANDEM a real-time monocular tracking and dense mapping framework. For pose estimation, TANDEM performs photometric bundle adjustment based on a sliding window of keyframes. To increase the robustness, we propose a novel tracking front-end that performs dense direct image alignment using depth maps rendered from a global model that is built incrementally from dense depth predictions. To predict the dense depth maps, we propose Cascade View-Aggregation MVSNet (CVA-MVSNet) that utilizes the entire active keyframe window by hierarchically constructing 3D cost volumes with adaptive view aggregation to balance the different stereo baselines between the keyframes. Finally, the predicted depth maps are fused into a consistent global map represented as a truncated signed distance function (TSDF) voxel grid. Our experimental results show that TANDEM outperforms other state-of-the-art traditional and learning-based monocular visual odometry (VO) methods in terms of camera tracking. Moreover, TANDEM shows state-of-the-art real-time 3D reconstruction performance.

Video

Poster

Publications


Export as PDF, XML, TEX or BIB

Conference and Workshop Papers
2021
[]TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo (L Koestler, N Yang, N Zeller and D Cremers), In Conference on Robot Learning (CoRL), 2021. ([GitHub][video][project page]) [bibtex] [pdf]
Powered by bibtexbrowser
Export as PDF, XML, TEX or BIB

Rechte Seite

Informatik IX
Chair of Computer Vision & Artificial Intelligence

Boltzmannstrasse 3
85748 Garching info@vision.in.tum.de

Follow us on:
CVG Group DVL Group

News

04.06.2021

Bernt Schiele (Max Planck Institute for Informatics) will give a talk in the TUM AI lecture series on June 10th, 3pm! Livestream

05.05.2021
French-German Machine Learning Symposium

French-German Machine Learning Symposium

The French-German Machine Learning Symposium aims to strengthen interactions and inspire collaborations between both countries. We invited some of the leading ML researchers from France and Germany to this two-day symposium to give a glimpse into their research, and engage in discussions on the future of machine learning and how to strengthen research collaborations in ML between France and Germany.

The list of speakers includes Yann LeCun, Cordelia Schmid, Jean-Bernard Lasserre, Bernhard Schölkopf, and many more! For the full program please visit the webpage.

03.05.2021

Ron Kimmel (Technion - Israel Institute of Technology) will give a talk in the TUM AI lecture series on May 6th, 3pm! Livestream

23.04.2021

4Seasons Dataset: We have released a novel dataset for benchmarking multi-weather SLAM in autonomous driving.

19.04.2021

Hao Li (Pinscreen) will give a talk in the TUM AI lecture series on April 22nd, 8pm! Livestream

More