Wide baseline feature matching using collobrative navigation and digital terrain elevation data constraints

• US 8,660,338 B2
• Filed: 03/22/2011
• Issued: 02/25/2014
• Est. Priority Date: 03/22/2011
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Ma, Yunqian, Mohr, Benjamin
• Primary Examiner(s): Repko, Jason M
• Assistant Examiner(s): ROWLAND, JHERE M
• Application Number:
  US13/053,777
• Publication Number:
  US 20120243775A1
• Time in Patent Office: 1,071 Days
• Field of Search: None
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30181 Earth observation
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

A method for wide baseline feature matching comprises capturing one or more images from an image sensor on each of two or more platforms when the image sensors have overlapping fields of view, performing a 2-D feature extraction on each of the captured images in each platform using local 2-D image feature descriptors, and calculating 3-D feature locations on the ellipsoid of the Earth surface from the extracted features using a position and attitude of the platform and a model of the image sensor. The 3-D feature locations are updated using digital terrain elevation data (DTED) as a constraint, and the extracted features are matched using the updated 3-D feature locations to create a common feature zone. A subset of features from the common feature zone is selected, and the subset of features is inputted into a collaborative filter in each platform. A convergence test is then performed on other subsets in the common feature zone, and falsely matched features are pruned from the common feature zone.

Claims

1. A method for wide baseline feature matching, the method comprising:

capturing one or more images from an image sensor on each of two or more platforms when the image sensors have overlapping fields of view;

performing a two-dimensional (2-D) feature extraction on each of the captured images in each platform using local 2-D image feature descriptors;

calculating three-dimensional (3-D) feature locations on the ellipsoid of the Earth surface from the extracted features in each platform using a position and attitude of the platform and a model of the image sensor;

updating the 3-D feature locations in each platform using digital terrain elevation data (DTED) as a constraint;

matching the extracted features in each of the platforms using the updated 3-D feature locations to create a common feature zone;

selecting a subset of features from the common feature zone;

inputting the subset of features into a collaborative filter in each platform;

performing a convergence test on other subsets in the common feature zone; and

pruning falsely matched features from the common feature zone.

2. The method of claim 1, wherein the image sensor in each of the platforms comprises an electro-optical sensor.

3. The method of claim 1, wherein the platforms comprise two or more vehicles in motion and in operative communication with each other for collaborative navigation.

4. The method of claim 3, wherein the vehicles comprise a first aircraft at a higher altitude than a second aircraft.

5. The method of claim 1, wherein the 2-D feature extraction includes selecting one or more interest points, and representing a neighborhood of each interest point by a feature vector.

6. The method of claim 1, wherein the local image feature descriptors comprise speed-up robust features (SURF) descriptors or scaled invariant features transform (SIFT) descriptors.

7. The method of claim 1, wherein the matching of the extracted features uses a combination of spatial neighborhood and similarity of feature appearance descriptors.

8. The method of claim 7, wherein the 3-D feature locations calculated in each platform provide a neighborhood searching area for feature matching, and the 2-D image feature descriptors are matched in the neighborhood.

9. The method of claim 1, further comprising projecting one or more 3-D feature locations on the ellipsoid of the Earth surface onto a DTED surface to generate one or more 3-D point features on the DTED surface.

10. A computer program product, comprising:

a non-transitory computer readable medium having instructions stored thereon executable by a processor to perform a method for wide baseline feature matching, the method comprising;

capturing one or more images from an electro-optical sensor on each of two or more vehicles in operative communication with each other when the electro-optical sensors have overlapping fields of view;

performing a two-dimensional (2-D) feature extraction on each of the captured images in each vehicle using local 2-D image feature descriptors;

calculating three-dimensional (3-D) feature locations on the ellipsoid of the Earth surface from the extracted features in each vehicle using a position and attitude of the vehicle and a model of the electro-optical sensor;

updating the 3-D feature locations in each vehicle using digital terrain elevation data (DT ED) as a constraint;

matching the extracted features in each vehicle using the updated 3-D feature locations to create a common feature zone;

selecting a subset of features from the common feature zone;

inputting the subset of features into a collaborative filter in each vehicle;

performing a convergence test on other subsets in the common feature zone; and

pruning falsely matched features from the common feature zone.

11. The computer program product of claim 10, wherein the 2-D feature extraction includes selecting one or more interest points, and representing a neighborhood of each interest point by a feature vector.

12. The computer program product of claim 10, wherein the local image feature descriptors comprise speed-up robust features (SURF) descriptors or scaled invariant features transform (SIFT) descriptors.

13. The computer program product of claim 10, wherein the matching of the extracted features uses a combination of spatial neighborhood and similarity of feature appearance descriptors.

14. The computer program product of claim 10, wherein the 3-D feature locations calculated in each vehicle provide a neighborhood searching area for feature matching, and the 2-D image feature descriptors are matched in the neighborhood.

15. The computer program product of claim 10, wherein the method further comprises projecting one or more 3-D feature locations on the ellipsoid of the Earth surface onto a DTED surface to generate one or more 3-D point features on the DTED surface.

16. A system for wide baseline feature matching, the system comprising:

a first platform; and

a second platform in operative communication with the first platform;

wherein the first platform and the second platform each comprise;

at least one processing unit mounted on the platform;

at least one electro-optical sensor mounted on the platform and operatively coupled to the processing unit;

at least one inertial measurement unit mounted on the platform and operatively coupled to the processing unit;

a least one navigation filter operatively coupled to the processing unit; and

a memory unit operatively coupled to the processing unit, the memory unit containing digital terrain elevation data (DTED) and a model of the electro-optical sensor, the memory unit including machine readable instructions executable by the processing unit to;

capture one or more images from the electro-optical sensor;

perform a two-dimensional (2-D) feature extraction on each of the captured images using local 2-D image feature descriptors;

calculate three-dimensional (3-D) feature locations on the ellipsoid of the Earth surface from the extracted features using a position and attitude of the platform and a model of the electro-optical sensor;

update the 3-D feature locations using digital terrain elevation data (DTED) as a constraint;

match the extracted features with extracted features in the other platform using the updated 3-D feature locations to create a common feature zone;

select a subset of features from the common feature zone;

input the subset of features into each navigation filter;

perform a convergence test on other subsets in the common feature zone; and

prune falsely matched features from the common feature zone.

17. The system of claim 16, wherein the first and second platforms comprise two mobile vehicles in motion and in operative communication with each other for collaborative navigation.

18. The system of claim 17, wherein the mobile vehicles comprise a first aircraft at a first altitude and a second aircraft at a second altitude that is different from the first altitude.

19. The system of claim 16, wherein the first and second platforms further comprise one or more aiding sensors in operative communication with the processing unit, the aiding sensors comprising a magnetic sensor, an altimeter, or a global position system receiver.

20. The system of claim 16, wherein the memory unit in each of the first and second platforms is configured to store 3-D feature locations and 2-D feature descriptors.