VOTING STRATEGY FOR VISUAL EGO-MOTION FROM STEREO

• US 20120308114A1
• Filed: 05/31/2011
• Published: 12/06/2012
• Est. Priority Date: 05/31/2011
• Status: Active Grant

• Current Assignee: Czech Technical University In Prague, Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Czech Technical University In Prague, Toyota Motor Europe SA (Toyota Motor Corporation)
• Inventors: Othmezouri, Gabriel, Matas, Jiri, Sakata, Ichiro, Obdrzálek, Stépàn
• Granted Patent: US 8,744,169 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs

Abstract

Methods and systems for egomotion estimation (e.g. of a vehicle) from visual inputs of a stereo pair of video cameras are described. 3D egomotion estimation is a six degrees of freedom problem in general. In embodiments of the present invention, this is simplified to four dimensions and further decomposed to two two-dimensional sub-solutions. The decomposition allows use of a voting strategy that identifies the most probable solution. An input is a set of image correspondences between two temporally consecutive stereo pairs, i.e. feature points do not need to be tracked over time. The experiments show that even if a trajectory is put together as a simple concatenation of frame-to-frame increments, the results are reliable and precise.

Claims

1. A method of estimation of egomotion of a vehicle from a sequence of stereo images taken with a stereo camera, the method comprising:

computing from a first pair of stereo images and a second previous pair of stereo images in the sequence increments between the first and second pairs of stereo images by determining an image correspondence set, correspondences of the image correspondence set being computed for all stereo image pairs in the sequence, 3D triangulation of the correspondences to provide 3D points in a camera-centric co-ordinate system;

estimating rotational egomotion by voting on image motion vectors, the image motion vectors being weighted by the 3D distance from the correspondences to the stereo camera; and

estimating translational egomotion from 3D motion vectors obtained by triangulating stereo correspondences in the first and second image pairs and by connecting the 3D points from correspondences in images from one camera of the stereo camera.

2. The method of claim 1, wherein the 3D motion vectors are weighted proportionally to the 3D distances.

3. The method of claim 1 wherein the estimation of egomotion is done in four dimensions.

4. The method of claim 1, wherein the first and second images are consecutive images.

5. The method of claim 1 wherein images are ignored that are obscured by other vehicles.

6. An image processor for estimation of egomotion of a vehicle from a sequence of stereo images taken with a stereo camera, the processor comprising:

a first computational part for computing from a first pair of stereo images and a second previous pair of stereo images in the sequence increments between the first and second pairs of stereo images by determining an image correspondence set, correspondences of the image correspondence set being computed for all stereo image pairs in the sequence,

a second computational part for 3D triangulation of the correspondences to provide 3D points in a camera-centric co-ordinate system;

a third computational part for estimating rotational egomotion by voting on image motion vectors, the image motion vectors being weighted by the 3D distance from the correspondences to the stereo camera; and

a fourth computational part for estimating translational egomotion from 3D motion vectors obtained by triangulating stereo correspondences in the first and second image pairs and by connecting the 3D points from correspondences in images from one camera of the stereo camera.

7. The image processor of claim 6, wherein the 3D motion vectors are weighted proportionally to the 3D distances.

8. The image processor of claim 6 wherein the estimation of egomotion is done in four dimensions.

9. The image processor of claim 6, wherein the first and second images are consecutive images.

10. The image processor of claim 6 wherein images are ignored that are obscured by other vehicles.

17. A computer program product stored on a machine readable non-transient storage medium for implementing the method of claim 7.

11. A method for computation of egomotion of a moving stereo camera from a stereo image sequence, wherein a six-dimensional egomotion estimation is decomposed into first and second two-dimensional subproblems, the first two-dimensional subproblem being solved for camera rotation and the second subproblem being solved for the camera translation.

12. The method of claim 11, further comprising computing pitch and yaw of camera rotation independently of translation by voting.

13. The method of claim 12 wherein the votes of the voting are directly motion vectors, that is image correspondence in temporally consecutive images, observed in one of the cameras and the weight of the votes is a function of 3D distance of the observed objects.

14. A computer based system for computation of egomotion of a moving stereo camera from a stereo image sequence, wherein the system is adapted for a six-dimensional egomotion estimation by decomposing into first and second two-dimensional subproblems, the first two-dimensional subproblem is solved for camera rotation and the second subproblem is solved the camera translation.

15. The computer based system of claim 14, wherein pitch and yaw of camera rotation are determined independently of translation by voting.

16. The computer based system of claim 15 wherein votes of the voting are directly the motion vectors, that is image correspondence in temporally consecutive images, observed in one of the cameras and the weight of the votes is a function of 3D distance of the observed objects