REAL-TIME MONOCULAR VISUAL ODOMETRY

• US 20140078258A1
• Filed: 04/06/2013
• Published: 03/20/2014
• Est. Priority Date: 09/17/2012
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Laboratories America Inc (NEC Corporation)
• Inventors: Chandraker, Manmohan, Song, Shiyu
• Granted Patent: US 9,148,650 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04N 13/20 Image signal generators
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00

Abstract

Systems and methods are disclosed for multithreaded visual odometry by acquired with a single camera on-board a vehicle; using 2D-3D correspondences for continuous pose estimation; and combining the pose estimation with 2D-2D epipolar search to replenish 3D points.

Claims

1. A method for multithreaded visual odometry, comprising

acquired with a single camera on-board a vehicle;

using 2D-3D correspondences for continuous pose estimation;

combining the pose estimation with 2D-2D epipolar search to replenish 3D points; and

generating structure-from-motion (SFM).

2. The method of claim 1, comprising using the visual odometry for autonomous driving applications.

3. The method of claim 1, comprising pose-guided matching to provide fast 3D-2D correspondences.

4. The method of claim 1, comprising performing epipolar constrained search to produce per-frame 2D-2D correspondences.

5. The method of claim 4, comprising constructing long tracks, and inserting new 3D points guaranteed to be accurate.

6. The method of claim 1, comprising validating each 3D point against all frames in real-time.

7. The method of claim 1, comprising performing local bundle adjustment to refine cameras and 3D points.

8. The method of claim 7, comprising collecting and refining 3D points from an epipolar thread.

9. The method of claim 1, comprising inserting new 3D points in a main thread.

10. The method of claim 1, comprising collecting and refinding mechanism allows bundle adjustment using long tracks.

11. The method of claim 1, comprising performing real-time global bundle adjustment in a thread-safe architecture with real-time pose estimation.

12. The method of claim 1, comprising handling failures due to tracking or scale drift.

13. The method of claim 1, comprising using a 1-point RANSAC between 3D points reconstructed before and after the tracking failure.

14. The method of claim 1, comprising scale correcting by combining scale estimates from 3D points and planar homography mappings.

15. The method of claim 1, comprising planar homography between a set of road points in two frames, using a 4-point RANSAC where hypotheses are generated by linear estimation of algebraic error and followed by Levenberg-Marquardt based nonlinear optimization of the optimal symmetric reprojection error over the largest consensus set:

16. A system for multithreaded visual odometry, comprising

a single camera on-board a vehicle;

and

a multi-threaded processor coupled to the camera, the multi-threaded processor using 2D-3D correspondences for continuous pose estimation and combining the pose estimation with 2D-2D epipolar search to replenish 3D points

17. The system of claim 16, comprising code for autonomous driving applications using the visual odometry.

18. The system of claim 16, comprising code for pose-guided matching to provide fast 3D-2D correspondences.

19. The system of claim 16, comprising code to perform epipolar constrained search to produce per-frame 2D-2D correspondences.

20. The system of claim 19, comprising code for constructing long tracks, and inserting new 3D points guaranteed to be accurate.