Methods, systems, and computer readable media for visual odometry using rigid structures identified by antipodal transform

• US 9,280,832 B2
• Filed: 05/08/2015
• Issued: 03/08/2016
• Est. Priority Date: 05/08/2014
• Status: Active Grant

• Current Assignee: Trustees Of The University Of Pennsylvania (University of Pennsylvania)
• Original Assignee: Trustees Of The University Of Pennsylvania (University of Pennsylvania)
• Inventors: Cleveland, Jonas, Daniilidis, Kostas
• Primary Examiner(s): Abdi, Amara
• Application Number:
  US14/707,632
• Publication Number:
  US 20150325003A1
• Time in Patent Office: 305 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments

Abstract

The subject matter described herein includes methods for visual odometry using rigid structures identified by an antipodal transform. One exemplary method includes receiving a sequence of images captured by a camera. The method further includes identifying rigid structures in the images using an antipodal transform. The method further includes identifying correspondence between rigid structures in different image frames. The method further includes estimating motion of the camera based on motion of corresponding rigid structures among the different image frames.

Claims

1. A method for visual odometry using rigid structures identified by an antipodal transform, the method comprising:

receiving a sequence of images captured by a camera;

identifying rigid structures in the images using an antipodal transform;

identifying correspondence between rigid structures in different image frames; and

estimating a path of motion of the camera based on motion of corresponding rigid structures among the different image frames;

wherein identifying rigid structures using the antipodal transform includes;

generating a binary occupancy matrix for pixels in an image frame, wherein each element in the occupancy matrix indicates whether the pixel includes point on a structure or not;

linearizing the matrix;

marking occupied elements in the matrix as having a score that is different from unoccupied elements; and

for each unoccupied element in the matrix, determining a distance in each of a plurality of directions to the nearest occupied element, computing a score for each distance, and summing the scores for each distance, and identifying the rigid structures using the scores.

2. The method of claim 1 wherein identifying rigid structures using the scores includes identifying unoccupied elements with the highest scores as keypoints that comprise center points of rigid structures.

3. The method of claim 2 wherein identifying correspondence between rigid structures includes generating an image descriptor for each keypoint, the image descriptor including an encoded gradient magnitude, wherein the gradient magnitude represents a change in contrast between image pixels and neighboring image pixels.

4. The method of claim 3 wherein identifying correspondence between rigid structures includes comparing image descriptors in different image frames.

5. The method of claim 4 wherein comparing image descriptors includes utilizing a similarity metric to characterize differences between the image descriptors.

6. The method of claim 5 wherein the similarity metric comprises a Hamming distance.

7. The method of claim 6 wherein comparing the image descriptors includes maintaining sets of closest matching image descriptors among frame triplets after outliner elimination until a predetermined number of matching descriptors is located.

8. The method of claim 1 wherein estimating motion of the camera includes computing absolute rotation of the camera based on identified principal directions in the scene.

9. The method of claim 8 wherein the principal directions are identified using vanishing points computed as intersections of orthogonal groups of parallel lines in the scene.

10. The method of claim 9 comprising computing an absolute position of the camera in each frame.

11. The method of claim 10 wherein estimating motion of the camera includes estimating the motion based on the change in the absolute position of the camera between frames.

12. A system for visual odometry using rigid structures identified by an antipodal transform, the system comprising:

a line and point feature extractor for receiving a sequence of images captured by a camera, identifying rigid structures in the images using an antipodal transform, and identifying correspondence between rigid structures in different image frames; and

a camera motion estimator estimating a path of motion of the camera based on motion of corresponding rigid structures among the different image frames;

wherein the line and point feature extractor is configured to compute the antipodal transform by;

generating a binary occupancy matrix for pixels in an image frame, wherein each element in the occupancy matrix indicates whether the pixel includes point on a structure or not;

linearizing the matrix;

marking occupied elements in the matrix as having a score that is different from unoccupied elements;

for each unoccupied element in the matrix, determining a distance in each of a plurality of directions to the nearest occupied element, computing a score for each distance, and summing the scores for each distance, and identifying the rigid structures using the scores.

13. The system of claim 12 wherein the line and point feature extractor is configured to identify unoccupied elements with the highest scores as keypoints that comprise center points of rigid structures.

14. The system of claim 13 wherein the line and point feature extractor is configured to generate an image descriptor for each keypoint, the image descriptor including an encoded gradient magnitude, wherein the gradient magnitude represents a change in contrast between image pixels and neighboring image pixels.

15. The system of claim 14 wherein the line and point feature extractor is configured to compare image descriptors in different image frames.

16. The system of claim 15 wherein the line and point feature extractor is configured to compare image descriptors utilizing a similarity metric to characterize differences between the image descriptors.

17. The system of claim 16 wherein the similarity metric comprises a Hamming distance.

18. The system of claim 17 wherein the line and point feature extractor is configured to compare descriptors by maintaining sets of closest matching descriptors among frame triplets after outliner elimination until a predetermined number of matching image descriptors is located.

19. The system of claim 12 wherein the motion estimator is configured to estimate motion of the camera includes by computing absolute rotation of the camera based on identified principal directions in the scene.

20. The system of claim 19 wherein the principal directions are identified using vanishing points computed as intersections of orthogonal groups of parallel lines in the scene.

21. The system of claim 20 wherein the motion estimator is configured to compute an absolute position of the camera in each frame.

22. The system of claim 21 wherein the motion estimator is configured to estimate motion of the camera based on the change in the absolute position of the camera between frames.

23. A non-transitory computer readable medium having stored thereon executable instructions that when executed by the processor of a computer control the computer to perform steps comprising:

receiving a sequence of images captured by a camera;

identifying rigid structures in the images using an antipodal transform;

identifying correspondence between rigid structures in different image frames; and

estimating a path of motion of the camera based on motion of corresponding rigid structures among the different image frames;

wherein identifying rigid structures using the antipodal transform includes;

generating a binary occupancy matrix for pixels in an image frame, wherein each element in the occupancy matrix indicates whether the pixel includes point on a structure or not;

linearizing the matrix;

marking occupied elements in the matrix as having a score that is different from unoccupied elements; and

for each unoccupied element in the matrix, determining a distance in each of a plurality of directions to the nearest occupied element, computing a score for each distance, and summing the scores for each distance, and identifying the rigid structures using the scores.