Optical Flow and SLAM with Event Cameras
Event cameras are novel camera systems that sense intensity change independently per pixel and report these events of change — brighter or darker by a specific amount — with a very accurate timestamp. As such, they are inspired from biology (retina) and offer the potential to overcome difficulties with motion blur or dynamic range that standard frame-based cameras face.
We have been looking at two different challenges: first, we tried to simply reconstruct both video and optical flow from the events: the approach should be able to deal with any scene content. Second, we tackled reconstruction of semi-dense depth and intensity keyframes along with general camera motion, where the scene is assumed to be static — effectively SLAM with an event camera.
Current collaborators:
- Patrick Bardow (Dyson Robotics Lab)
- Prof. Andrew Davison
Former collaborators
- Hanme Kim (previously Robot Vision Group)
Keyframe-Based Visual-Inertial Odometry Using Nonlinear Optimisation
Here, we fuse inertial measurements with visual measurements: due to the complementary characteristics of these sensing modalities, they have become a popular choice for accurate SLAM in mobile robotics. While historically the problem has been addressed with filtering, advancements in visual estimation suggest that non-linear optimization offers superior accuracy, while still tractable in complexity thanks to the sparsity of the underlying problem. Taking inspiration from these findings, we formulate a rigorously probabilistic cost function that combines reprojection error of landmarks and inertial terms. We ensure real-time operation by limiting the optimization to a bounded window of keyframes by applying marginalization. Keyframes may be spaced in time by arbitrary intervals, while still related by linearized inertial terms.
Collaborators:
- Simon Lynen (Google)
- Dr Mike Bosse
- Dr Vincent Rabaud
- Dr Kurt Konolige (Google)
- Andreas Forster (Oculus)
- Dr Margarita Chli (ETH Zurich)
- Prof. Roland Siegwart (ETH Zurich)
- Dr Paul Furgale
BRISK: Keypoint Associations
Effective and efficient generation of keypoints from an image is a well-studied problem in the literature and forms the basis of numerous Computer Vision applications, including data association for vision-supported odometry or Simultaneous Localization and Mapping. Established leaders in the field are the SIFT and SURF algorithms which exhibit great performance under a variety of image transformations, with SURF in particular considered as the most computationally efficient amongst the high-performance methods to date. We propose BRISK: Binary Robust Invariant Scalable Keypoints, a novel method for keypoint detection, description and matching. A comprehensive evaluation on benchmark datasets reveals BRISK’s adaptive, high descriptive power as in state-of-the-art algorithms, albeit at a dramatically lower computational cost. The key to speed lies in the application of a novel scale-space corner detector in combination with the assembly of a bit-string descriptor from intensity comparisons retrieved by dedicated sampling of each keypoint neighbourhood.
Collaborators from ETH Zurich:
Multi-Sensor Fusion without Cameras
A reliable and precise state estimation is crucial for any kind of unmanned airplanes. The Extended Kalman Filter (EKF) widely deployed in aerospace state estimation has been the workhorse fusing inertial sensor feeds with navigation aids for decades. With the advent of increasingly precise, high bandwidth and low noise MEMS inertial sensors and three-axis electronic compasses during the last few years, such state estimation filters may be run on-board low-cost electronics. Thus also the autonomous robotics community showed a rising interest in such systems. We present a full EKF INS filter that is robust with respect to GPS outage as well as dynamic maneuvers, including long-term circling. We use updates obtained by magnetometers, static pressure and position in normal operation, and airspeed updates during GPS loss. Therefore, the covariances are staying small compared to nonlinearities, thus both operation modes form scenarios where the EKF has a long history of success.
Collaborators
- Amir Melzer (ETH Zurich)
- Prof Kostas Alexis (previously ETH Zurich, now Reno University)
- Prof. Roland Siegwart (ETH Zurich)
