Direkt zum Inhalt springen
Computer Vision Group
TUM Department of Informatics
Technical University of Munich

Technical University of Munich

Menu

Links


Visual-Inertial Event Dataset

Contact : Simon Klenk, Jason Chui, Nikolaus Demmel.

TUM-VIE: The TUM Stereo Visual-Inertial Event Data Set

Event cameras are bio-inspired vision sensors which measure per pixel brightness changes. They offer numerous benefits over traditional, frame based cameras, including low latency, high dynamic range, high temporal resolution and low power consumption. Thus, these sensors are suited for robotics and virtual reality applications. To foster the development of 3D perception and navigation algorithms with event cameras, we present the TUM-VIE dataset. It consists of a large variety of handheld and head-mounted sequences in indoor and outdoor environments, including rapid motion during sports and high dynamic range scenarios. The dataset contains stereo event data, stereo grayscale frames at 20Hz as well as IMU data at 200Hz. Timestamps between all sensors are synchronized in hardware. The event cameras contain a large sensor of 1280x720 pixels, which is significantly larger than the sensors used in existing stereo event datasets (at least by a factor of ten). We provide ground truth poses from a motion capture system at 120Hz during the beginning and end of each sequence, which can be used for trajectory evaluation. TUM- VIE includes challenging sequences where state-of-the art visual SLAM algorithms either fail or result in large drift. Hence, our dataset can help to push the boundary of future research on event-based visual-inertial perception algorithms.

Dataset Sequences

All sequences have been processed to have consistent timestamps for visual cameras, IMU, event cameras.

Sequence name Zipped Bag File Event Raw Data
mocap-1d-trans zipped-bag (5.5GB) left-event-camera (1.8GB) right-event-camera (1.7GB)
mocap-3d-trans mocap-3d-trans (6.6GB) left-event-camera (2.6GB) right-event-camera (2.5GB)
mocap-6dof mocap-6dof (4.3GB) left-event-camera (1.7GB) right-event-camera (1.6GB)
mocap-desk mocap-desk (8.3GB) left-event-camera (3.3GB) right-event-camera (3.4GB)
mocap-desk2 mocap-6dof (4.6GB) left-event-camera (1.9GB) right-event-camera (1.8GB)
mocap-shake mocap-shake (5.1GB) left-event-camera (2.1GB) right-event-camera (2.1GB)
mocap-shake2 mocap-shake2 (4.5GB) left-event-camera (1.8GB) right-event-camera (1.7GB)
office-maze office-maze (33.7GB) left-event-camera (13.3GB) right-event-camera (13.3GB)
running-easy running-easy (15.1GB) left-event-camera (6.0GB) right-event-camera (6.0GB)
running-hard running-hard (14.6GB) left-event-camera (5.8GB) right-event-camera (5.8GB)
skate-easy skate-easy (16.1GB) left-event-camera (6.3GB) right-event-camera (6.3GB)
skate-hard skate-hard (17.1GB) left-event-camera (6.7GB) right-event-camera (6.7GB)
loop-floor0 loop-floor0 (62.4GB) left-event-camera (24.4GB) right-event-camera (24.4GB)
loop-floor1 loop-floor1 (56.4GB) left-event-camera (22.3GB) right-event-camera (22.3GB)
loop-floor2 loop-floor2 (51.1GB) left-event-camera (20.1GB) right-event-camera (19.9GB)
loop-floor3 loop-floor3 (56.7GB) left-event-camera (22.3GB) right-event-camera (22.4GB)
floor2-dark floor2-dark left-event-camera (8.5GB) right-event-camera (8.5GB)
slide slide (41.7GB) left-event-camera (16.3GB) right-event-camera (16.2GB)
bike-easy bike-easy (58.7GB) left-event-camera (1.8GB) right-event-camera (1.7GB)
bike-hard bike-hard (58.7GB) left-event-camera (22.7GB) right-event-camera (22.4GB)
bike-dark bike-night (50.8GB) left-event-camera (19.7GB) right-event-camera (19.7GB)

Calibration results

Sequence name Zipped Bag File (VIE data)
calibration1 26th February calib-26-02 ()
calibration2 26th February calib2-26-02 ()
calibration1 27th February calib-27-02 ()
calibration2 27th February calib2-27-02 ()

MoCap Time offset

The estimated time offsets in nanoseconds between IMU and MoCap clocks are summarized in the following table. These can be used when working with the ground truth data sequences. For trajectories which exit and re-enter the Motion capture room, we provide three such offsets in the following order: <OffsetBeforeExitingMocap, OffsetAfterReneteringMocap, OffsetIgnoringClockDrift>. If very accurate time alignment is required, it can be beneficial to work with the two separate offsets OffsetBeforeExitingMocap and OffsetAfterReneteringMocap, especially for the longer sequences. Usually, it should be enough to simply use the tird value, which is obtained from alignemnt of IMU and MotionCapture measurements over the whole trajectory, effectively ignoring the clock drift between MoCap clock and our main sensor clock (left event camera):

mocap-1d-trans: -751015909094
mocap-3d-trans: -1493639818778
mocap-6dof: -23105236939027
mocap-desk: -1379932254349
mocap-desk2: -1497671780771
mocap-shake: -25967023717479
mocap-shake2: -26201740113146


office-maze:  -19465849562412, -19465853571556, -19465851769812
running-easy: -16736726887757, -16736727587368,  -16736727600659
running-hard: -16827571466934, -16827572469728, -16827572418176
skate-easy:   -13031586379644, -13031588568030, -13031588062872
skate-hard:   -1652428013680,  -1652430957009,  -1652429545627
loop-floor0:  -5285259175038,  -5285268784356,  -5285264987194
loop-floor1:  -5000341897302,  -5000351815211,  -5000347602591
loop-floor2:  -374918028807,   -374927954540,   -374923120368
loop-floor3:  -4638756461222,  -4638767499736,  -4638762771268
floor2-dark:  -28509321395462, -28509326232315, -28509323872804
slide:        -7872192374019,  -7872199572677,  -7872196568503
bike-easy:    -11434398806710, -11434409729896, -11434403676226
bike-hard:    -13780153106785, -13780162900458, -13780157880112
bike-dark:    -185530460448,   -185539615280,   -185534506945

Calibration Sequences

Sample Videos Event Stream

We provide a few selected sequences as avi video, using an accumulation time of 10 milliseconds. White pixels represent positive events, black pixels negative events and gray pixels no change.

Sequence name AVI video (left events) AVI video (right events)
running-easy running-easy-left (1.2GB) running-easy-right (1.2GB)
slide slide-left (3.2GB) slide-right (3.2GB)
bike-night-left bike-night-left (4.8GB) running-easy-right (4.8GB)

License

All data in the TUM-VIE Dataset is licensed under a Creative Commons 4.0 Attribution License (CC BY 4.0) and the accompanying source code is licensed under a BSD-2-Clause License.

Rechte Seite

Informatik IX
Chair of Computer Vision & Artificial Intelligence

Boltzmannstrasse 3
85748 Garching info@vision.in.tum.de

Follow us on:
CVG Group DVL Group

News

07.04.2021

Thomas Pock (TU Graz) will give a talk in the TUM AI lecture series on April 15th, 3pm! Livestream

26.03.2021

Max Welling (University of Amsterdam) will give a talk in the TUM AI lecture series on April 1st, 3pm! Livestream

02.03.2021

We have seven papers (4 orals, 3 posters) accepted to CVPR 2021!

02.03.2021

We have two papers accepted to ICRA 2021!

01.03.2021

Maks Ovsjanikov (Ecole Polytechnique) will give a talk in the TUM AI lecture series on March 11th, 3pm! Livestream

More