Method and apparatus for visual odometry

• US 20070288141A1
• Filed: 06/22/2005
• Published: 12/13/2007
• Est. Priority Date: 06/22/2004
• Status: Abandoned Application

• Current Assignee: Sarnoff Corporation (SRI International, Inc.)
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Naroditsky, Oleg, Nister, David, Bergen, James
• Application Number:
  US11/159,968
• Publication Number:
  US 20070288141A1
• Time in Patent Office: Days
• US Class Current: 701/38
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A method and apparatus for visual odometry (e.g., for navigating a surrounding environment) is disclosed. In one embodiment a sequence of scene imagery is received (e.g., from a video camera or a stereo head) that represents at least a portion of the surrounding environment. The sequence of scene imagery is processed (e.g., in accordance with video processing techniques) to derive an estimate of a pose relative to the surrounding environment. This estimate may be further supplemented with data from other sensors, such as a global positioning system or inertial or mechanical sensors.

Claims

1. A method for navigating a surrounding environment, comprising:

receiving a sequence of scene imagery representing at least a portion of said surrounding environment; and

processing said sequence of scene imagery to derive an estimate of a pose relative to said surrounding environment.

2. The method of claim 1, further comprising:

receiving supplemental position-related data from at least one sensor; and

supplementing said estimate with said supplemental position-related data.

3. The method of claim 2, wherein said at least one sensor is at least one of:

a global positioning system, an inertial sensor and a mechanical sensor.

4. The method of claim 1, wherein said pose comprises:

a three-dimensional location; and

an angular orientation.

5. The method of claim 1, wherein said processing comprises:

locating a plurality of point features in a first frame of said sequence of scene imagery;

tracking said plurality of point features over a plurality of subsequent frames of said sequence of scene imagery to generate a first plurality of associated point feature trajectories; and

estimating said pose in accordance with said first plurality of point feature trajectories.

6. The method of claim 5, wherein said locating comprises:

computing a corner response for each pixel in said first frame of said sequence of scene imagery; and

declaring a point feature at each pixel for which said corner response is stronger than at all other pixels within a defined radius.

7. The method of claim 5, wherein said tracking comprises:

comparing each of said plurality of point features in said first frame of said sequence of scene imagery against a second plurality of point features in at least one subsequent frame of said sequence of scene imagery to generate a plurality of potential matches; and

selecting one of said potential matches in accordance with mutual consistency.

8. The method of claim 7, wherein said second plurality of point features comprises every point feature in said subsequent frame that is within a predefined distance from a point feature that corresponds to a given point feature in said first frame.

9. The method of claim 5, wherein said estimating comprises:

generating a plurality of frame-to-frame incremental pose estimates based on designated subsets of said first plurality of point feature trajectories; and

selecting one of said plurality of frame-to-frame incremental pose estimates as being most likely to be indicative of said pose.

10. The method of claim 9, wherein said generating comprises:

receiving two simultaneous stereo views of said first plurality of point feature trajectories;

triangulating a plurality of three-dimensional points in accordance with said two simultaneous stereo views;

receiving subsequent point feature trajectory data associated with said plurality of three-dimensional points; and

generating at least one incremental pose estimate in accordance with said plurality of three-dimensional points and said first plurality of point feature trajectories.

11. The method of claim 5, further comprising:

incrementally triangulating a second plurality of point feature trajectories to a plurality of three-dimensional points;

receiving subsequent point feature trajectory data associated with said second plurality of point feature trajectories; and

generating at least one incremental pose estimate in accordance with said plurality of three-dimensional points and said second plurality of point feature trajectories.

12. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps of a method for navigating a surrounding environment, comprising:

receiving a sequence of scene imagery representing at least a portion of said surrounding environment; and

processing said sequence of scene imagery to derive an estimate of a pose relative to said surrounding environment.

13. The computer-readable medium of claim 12, further comprising:

receiving supplemental position-related data from at least one sensor; and

supplementing said estimate with said supplemental position-related data.

14. The computer-readable medium of claim 13, wherein said at least one sensor is at least one of:

a global positioning system, an inertial sensor and a mechanical sensor.

15. The computer-readable medium of claim 12, wherein said pose comprises:

a three-dimensional location; and

an angular orientation.

16. The computer-readable medium of claim 12, wherein said processing comprises:

locating a plurality of point features in a first frame of said sequence of scene imagery;

tracking said plurality of point features over a plurality of subsequent frames of said sequence of scene imagery to generate a first plurality of associated point feature trajectories; and

estimating said pose in accordance with said first plurality of point feature trajectories.

17. The computer-readable medium of claim 16, wherein said locating comprises:

computing a corner response for each pixel in said first frame of said sequence of scene imagery; and

declaring a point feature at each pixel for which said corner response is stronger than at all other pixels within a defined radius.

18. The computer-readable medium of claim 16, wherein said tracking comprises:

comparing each of said plurality of point features in said first frame of said sequence of scene imagery against a second plurality of point features in at least one subsequent frame of said sequence of scene imagery to generate a plurality of potential matches; and

selecting one of said potential matches in accordance with mutual consistency.

19. The computer-readable medium of claim 18, wherein said second plurality of point features comprises every point feature in said subsequent frame that is within a predefined distance from a point feature that corresponds to a given point feature in said first frame.

20. The computer-readable medium of claim 16, wherein said estimating comprises:

generating a plurality of frame-to-frame incremental pose estimates based on designated subsets of said first plurality of point feature trajectories; and

selecting one of said plurality of frame-to-frame incremental pose estimates as being most likely to be indicative of said pose.

21. The computer-readable medium of claim 20, wherein said generating comprises:

receiving two simultaneous stereo views of said first plurality of point feature trajectories;

triangulating a plurality of three-dimensional points in accordance with said two simultaneous stereo views;

receiving subsequent point feature trajectory data associated with said plurality of three-dimensional points; and

generating at least one incremental pose estimate in accordance with said plurality of three-dimensional points and said first plurality of point feature trajectories.

22. The computer-readable medium of claim 16, further comprising:

incrementally triangulating a second plurality of point feature trajectories to a plurality of three-dimensional points;

receiving subsequent point feature trajectory data associated with said second plurality of point feature trajectories; and

generating at least one incremental pose estimate in accordance with said plurality of three-dimensional points and said second plurality of point feature trajectories.

23. An apparatus for navigating a surrounding environment, comprising:

means for receiving a sequence of scene imagery representing at least a portion of said surrounding environment; and

means for processing said sequence of scene imagery to derive an estimate of a pose relative to said surrounding environment.