Lecture: Visual Navigation for Flying Robots
In recent years, flying robots such as quadcopters have gained increased interest in robotics and computer vision research. For navigating safely, these robots need the ability to localize themselves autonomously using their onboard sensors. Potential applications of such systems include the autonomous 3D reconstruction of buildings, inspection and simple maintenance tasks, surveillance of public places as well as in search and rescue systems.
(6 ECTS, changed on 12.7.2012)
December 07, 2012: This course has been distinguished with the Teach Inf Award 2011/12 at our faculty for the best lecture in computer science in the summer term 2012. Thank you very much for your support!
In this course, we will provide an overview of current techniques for 3D localization, mapping and navigation that are suitable for quadcopters. This course will cover the following topics:
- necessary background on robot hardware, sensors, 3D transformations
- motion estimation from images (including interest point detection, feature descriptors, robust estimation, visual odometry, iteratively closest point)
- filtering techniques and data fusion
- non-linear minimization, bundle adjustment, place recognition, 3D reconstruction
- autonomous navigation and exploration of unknown environments
The lecture will be accompanied by a lab course where the students will implement their own visual navigation system. This course is an excellent preparation for a master thesis project in this area.
Exercise sheets will be passed out every other week, containing both theoretical problems and programming exercises (in C++). In an exercise group every other week, we will discuss the solutions to the theory problems and the programming problems. Active participation in the exercises is the requirement for participation in the final exam. This will be written or oral, depending on the number of attendees. The questions will cover all material presented in class.
The practical exercises will be implemented directly on a Parrot Ardrone quadrocopter, so we expect a lot of fun (and broken propellors).
The lecture will be given by Jürgen Sturm.
Lecture: Tuesday, 10:15-11:45, room 02.09.23 (FMI, Boltzmannstrasse 3)
Teaching assistant: Nikolas Engelhard
Lab course/practice: Thursday: 14:15-15:45, room 02.09.23 or 02.09.38 (lab) (FMI, Boltzmannstrasse 3)
Registration: via TUM campus
The oral exam takes place in room 02.09.59. Sign up for a time slot on the list in front of the secretary (room 02.09.52). On the examination day: Please take a seat in the sofa corner in front of room 02.09.52 until we pick you up.
A printer ready version (two sides, six pages per sheet) of the lecture notes can be found here: pdf
|17.04.2012||Introduction pdf avi mp4 (sorry, poor audio quality!)|
|24.04.2012||Linear algebra, geometry, sensors pdf mp4 (bad synchronization + missing end –> use pdf)|
|08.05.2012||State estimation pdf mp4 (bad synchronization –> use pdf)|
|22.05.2012||Robot control pdf mp4|
|05.06.2012||Visual motion estimation pdf mp4|
|12.06.2012||Simultaneous localization and mapping pdf mp4|
|19.06.2012||Bundle adjustment and stereo correspondence pdf mp4|
|26.06.2012||Place recognition, ICP, and dense reconstruction pdf mp4|
|03.07.2012||Global navigation and path planning pdf mp4|
|10.07.2012||Planning under uncertainty, exploration and coordination pdf mp4|
|17.07.2012||Evaluation and benchmarking, time for questions pdf mp4|
|19.04.2012||Robot lab pdftgz tgz bag1bag2bag3|
|03.05.2012||Exercise: Robot Odometry pdf|
|10.05.2012||Robot lab pdf (Last update: May 14, 12:00)|
|24.05.2012||Exercise: Robot Localization pdf (Last update: May 24, 21:30)|
|31.05.2012||Robot lab pdf (Last update: June 4, 12:00)|
|14.06.2012||Exercise: Position Control pdf|
|21.06.2012||Exercise: Project proposal pdf pdf|
|05.07.2012||Exercise: Project mid-term pdf|
|19.07.2012||Exercise: Project presentation pdf pdf|
The participation in the exercises is obligatory. Participation in the robot lab is recommended, but not mandatory.
|Project title||Team name||Proposal||Midterm||Final|
|Trajectory Generation and Following with Position Correction||Crash Pilots|
|Localization with a particle filter||Viking|
|Autonomous Landing on a Moving Platform||Beer|
|Circling around a person||Dragon Sheep|
|Autonomous flying drone for building surveillance||Red One|
|Using a Saliency Map to turn the Quadcopter towards interesting points||Brezel|
|Gesture Based Control||Weissbier|
|Fast landing on a moving vehicle||Roter Baron|
|Autonomous Landing on a Moving Platform||Weisswurst|
The video-taped talks from the final presentations are now available online (password required, same as for ICRA proceedings).
- Probabilistic Robotics. Sebastian Thrun, Wolfram Burgard and Dieter Fox. MIT Press, 2005.
- Computer Vision: Algorithms and Applications. Richard Szeliski. Springer, 2010.
- Lehrevaluation (in German)
|Lecture Notes: Visual Navigation for Flying Robots , Technische Universität München, Germany, 2013. [bib] [pdf]Distinguished with the TUM TeachInf Award for the best lecture in summer term 2013|
|Lecture Notes: Visual Navigation for Flying Robots , Technische Universität München, Germany, 2012. [bib] [pdf]Distinguished with the TUM TeachInf Award for the best lecture in summer term 2012|