Persönlicher Status und Werkzeuge

Home Teaching Summer Semester 2013 Visual Navigation for Flying Robots

Visual Navigation for Flying Robots

In recent years, flying robots such as autonomous quadrocopters have gained increased interest in robotics and computer vision research. For navigating safely, these robots need the ability to localize themselves autonomously using their onboard sensors. Potential applications of such systems include the automatic 3D reconstruction of buildings, inspection and simple maintenance tasks, surveillance of public places as well as in search and rescue systems.

ECTS credits: 6

Content

In recent years, flying robots such as autonomous quadrocopters have gained increased interest in robotics and computer vision research. For navigating safely, these robots need the ability to localize themselves autonomously using their onboard sensors. Potential applications of such systems include the automatic 3D reconstruction of buildings, inspection and simple maintenance tasks, surveillance of public places as well as in search and rescue systems.

In this course, we will provide an overview of current techniques for 3D localization, mapping and navigation that are suitable for quadrocopters. This course will cover the following topics:

- necessary background on robot hardware, sensors, 3D transformations

- motion estimation from images (including interest point detection, feature descriptors, robust estimation, visual odometry, iteratively closest point)

- filtering techniques and data fusion

- non-linear minimization, bundle adjustment, place recognition, 3D reconstruction

- autonomous navigation, path planning, exploration of unknown environments

The lecture will be accompanied by a lab course where the students will implement their own visual navigation system. This course is an excellent preparation for a master thesis project in this area.

Lab Course

Exercise sheets will be passed out every other week, containing both theoretical problems and programming exercises (typically in C++). In an exercise group every other week, we will discuss the solutions to the theory problems and the programming problems. Active participation in the exercises is a requirement for participation in the final exam. This will be written or oral, depending on the number of attendees. The questions will cover all material presented in class. During the semester, we offer a weekly lab course where students can get programming support for the quadrocopters for the exercises.

The practical exercises will be implemented directly on a Parrot Ardrone quadrocopter, so we expect a lot of fun (and broken propellers).

Organization

The lecture will be given by Jürgen Sturm.

Lecture: Tuesday, 10:15-11:45, room 02.09.23 (FMI, Boltzmannstrasse 3)

Teaching assistants: Jakob Engel, Christian Kerl

Contact: visnav2013@vision.in.tum.de use this mail address for all questions related to the lecture and the lab course

Lab course/practice: Thursday: 14:00-15:30, room 02.05.014 (FMI, Boltzmannstrasse 3)

Registration: via TUM campus

Note that the number of participants is limited to 30. Registration in TUMonline for this course starts on 28.03.13,12:00.

Schedule

Lectures

The lectures will take place every Tuesday at 10:15 (CT), in the CVPR seminar room (02.09.023). A printer-friendly version of the lecture slides is available here.

Date Slides
16.04.2013 Introduction pdf video
23.04.2013 Linear algebra, geometry, sensors pdf video
30.04.2013 State estimation pdf video
07.05.2013 Robot control pdf video
28.05.2013 Visual motion estimation pdf video
04.06.2013 Structure from Motion pdf video
11.06.2013 RANSAC, ICP, and SLAM pdf video
18.06.2013 Dense reconstruction pdf video
25.06.2013 Guest talks (Jakob Engel, Christian Kerl, Frank Steinbrücker/TUM) pdf 1 pdf 2pdf 3 video
02.07.2013 Path planning and navigation pdf video
09.07.2013 Guest talks (Friedrich Fraundorfer/ETH and Korbinian Schmid/DLR) pdf 1video 1 pdf 2video 2
16.07.2013 Exploration, coverage and benchmarking pdf video

Lecture Videos

Lab Course

The lab courses (mandatory!) will take place every Thursday at 14:00 (sharp), in 02.05.014

Date Topic Material
18.04.2013 Introduction to ROS Slides Exercise Sheet 0 (ROS Intro)
25.04.2013 Exercise 1: Robot Odometry Exercise Sheet 1 flight_square.bag flight_z.bag flight_manual.bag
02.05.2013 Lab 1 Solution Sheet 1
09.05.2013 no Lab (holiday)
16.05.2013 Exercise 2: Robot Localization Exercise Sheet 2 (updated 13.5.) Bagfile
23.05.2013 Lab 2 Solution Sheet 2
06.06.2013 Exercise 3 Exercise Sheet 3 control_flight.bag
13.06.2013 Lab 3 Solution Sheet 3
20.06.2013 Project: Proposal Exercise Sheet 4 (deadline updated 13.06., 2:50 p.m. )
27.06.2013 Project Lab 1
04.07.2013 Project: Mid-term Exercise Sheet 5
11.07.2013 Project Lab 2
18.07.2013 Project: Final presentation Exercise Sheet 6

Projects

Team Title Proposal Midterm Final
1 Waving recognition
2 Quadcopter Navigation through Obstacles using Potential Field
3 POI Localization and Mapping
4 Avoiding obstacles while following a line video
5 Where am I? Appearance Based Mapping
6 Simple Hand-Gesture Based Control of Quadcopters
7 Free navigation of the quadrocopter via visual waypoints
8 gofi - the golfball finder
9 Group Photo

Additional Material

  • ICRA proceedings (username: visnav, password: ask by email)
    • Real-Time Motion Tracking on a Cellphone Using Inertial Sensing and a Rolling-Shutter Camera pdf video
    • First Flight Tests for a Quadrotor UAV with Tilting Propellers pdf video
    • Infrastructure-Free Shipdeck Tracking for Autonomous Landing pdf video
    • An Open Source and Open Hardware Embedded Metric Optical Flow CMOS Camera for Indoor and Outdoor Applications pdf video
    • First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehiclespdf video
    • Vision-Based State Estimation for Autonomous Rotorcraft MAVs in Complex Environments pdf video
    • Learning Monocular Reactive UAV Control in Cluttered Natural Environments pdf video
    • Fast Visual Odometry and Mapping from RGB-D Data pdf video
    • Stereo Vision and IMU Based Real-Time Ego-Motion and Depth Image Computation on a Handheld Device pdf
    • Parallel, real-time monocular visual odometry pdf
Last edited 11.09.2013 10:37 by Juergen Sturm