Voting strategy for visual ego-motion from stereo

• US 8,744,169 B2
• Filed: 05/31/2011
• Issued: 06/03/2014
• Est. Priority Date: 05/31/2011
• Status: Active Grant

• Current Assignee: Czech Technical University In Prague, Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Czech Technical University In Prague, Toyota Motor Europe SA (Toyota Motor Corporation)
• Inventors: Othmezouri, Gabriel, Sakata, Ichiro, Matas, Jiri, Obdrlek, tpn
• Primary Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US13/118,959
• Publication Number:
  US 20120308114A1
• Time in Patent Office: 1,099 Days
• Field of Search: 382/154, 382/103, 382/285, 382/312, 382/107, 382/100, 348/43, 348/135, 348/208.1, 348/42, 348/208.2, 701/1, 701/301
• US Class Current: 382/154
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs

Abstract

Methods and systems for egomotion estimation (e.g. of a vehicle) from visual inputs of a stereo pair of video cameras are described. 3D egomotion estimation is a six degrees of freedom problem in general. In embodiments of the present invention, this is simplified to four dimensions and further decomposed to two two-dimensional sub-solutions. The decomposition allows use of a voting strategy that identifies the most probable solution. An input is a set of image correspondences between two temporally consecutive stereo pairs, i.e. feature points do not need to be tracked over time. The experiments show that even if a trajectory is put together as a simple concatenation of frame-to-frame increments, the results are reliable and precise.

Claims

1. A method of estimating egomotion of a vehicle from a sequence of stereo images taken with a stereo camera, the method comprising:

computing, from a first pair of stereo images and a second pair of stereo images preceding the first pair of stereo images in the sequence, increments between the first and second pairs of stereo images by determining an image correspondence set, correspondences of the image correspondence set being computed for all stereo image pairs in the sequence, and 3D triangulation of the correspondences to provide 3D points in a camera-centric co-ordinate system;

estimating rotational egomotion by voting on image motion vectors, the image motion vectors being weighted by the 3D distance from the correspondences to the stereo camera; and

estimating translational egomotion by voting on 3D motion vectors obtained by triangulating stereo correspondences in the first and second image pairs and by connecting the 3D points from correspondences in images from one camera of the stereo camera.

2. The method of claim 1, wherein the 3D motion vectors are weighted proportionally to the 3D distances.

3. The method of claim 1 wherein the estimation of egomotion is done in four dimensions.

4. The method of claim 1, wherein the first and second images are consecutive images.

5. The method of claim 1 wherein images that are obscured by other vehicles are ignored.

6. An image processor for estimation of egomotion of a vehicle from a sequence of stereo images taken with a stereo camera, the processor comprising:

a first computational part for computing, from a first pair of stereo images and a second previous pair of stereo images in the sequence, increments between the first and second pairs of stereo images by determining an image correspondence set, wherein correspondences of the image correspondence set being computed for all stereo image pairs in the sequence;

a second computational part for 3D triangulation of the correspondences to provide 3D points in a camera-centric co-ordinate system;

a third computational part for estimating rotational egomotion by voting on image motion vectors, the image motion vectors being weighted by the 3D distance from the correspondences to the stereo camera; and

a fourth computational part for estimating translational egomotion by voting on 3D motion vectors obtained by triangulating stereo correspondences in the first and second image pairs and by connecting the 3D points from correspondences in images from one camera of the stereo camera.

7. The image processor of claim 6, wherein the 3D motion vectors are weighted proportionally to the 3D distances.

8. The image processor of claim 6 wherein the estimation of egomotion is done in four dimensions.

9. The image processor of claim 6, wherein the first and second images are consecutive images.

10. The image processor of claim 6 wherein images are ignored that are obscured by other vehicles.

11. A computer program product stored on a machine readable non-transient storage medium for implementing the method of claim 6.

12. A method for computation of egomotion of a moving stereo camera from a stereo image sequence, comprising computing pitch and yaw of camera rotation independently of translation by voting, wherein a six-dimensional egomotion estimation is decomposed into first and second two-dimensional subproblems, wherein the first two-dimensional subproblem is solved for camera rotation and the second two-dimensional subproblem being solved for the camera translation.

13. A computer based system for computation of egomotion of a moving stereo camera from a stereo image sequence, wherein pitch and yaw of camera rotation are determined independently of translation by voting, wherein the system is adapted for a six-dimensional egomotion estimation by decomposing into first and second two-dimensional subproblems, wherein the first two-dimensional subproblem is solved for camera rotation and the second two-dimensional subproblem is solved the camera translation, and wherein the votes of the voting correspond directly to motion vectors by way of image correspondence in temporally consecutive images observed in one of the cameras, and the weight of the votes is a function of 3D distance of the observed objects.