METHOD OF USING IMAGE WARPING FOR GEO-REGISTRATION FEATURE MATCHING IN VISION-AIDED POSITIONING

• US 20150199556A1
• Filed: 01/13/2014
• Published: 07/16/2015
• Est. Priority Date: 01/13/2014
• Status: Abandoned Application

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Jeerage, Mahesh K., Ma, Yunqian, Qian, Gang
• Application Number:
  US14/153,495
• Publication Number:
  US 20150199556A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/247 by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

Systems and methods for using image warping to improve geo-registration feature matching in vision-aided positioning is provided. In at least one embodiment, the method comprises capturing an oblique optical image of an area of interest using an image capturing device. Furthermore, digital elevation data and at least one geo-referenced orthoimage of an area that includes the area of interest are provided. The area of interest in the oblique optical image is then correlated with the digital elevation data to create an image warping matrix. The at least one geo-referenced orthoimage is then warped to the perspective of the oblique optical image using the image warping matrix. And, features in the oblique optical image are matched with features in the at least one warped geo-referenced orthoimage.

Claims

1. A method comprising:

capturing an oblique optical image of an area of interest using an image capturing device;

providing digital elevation data of an area that includes the area of interest and at least one geo-referenced orthoimage of an area that includes the area of interest;

correlating the area of interest in the oblique optical image with the digital elevation data to create an image warping matrix;

warping the at least one geo-referenced orthoimage to the perspective of the oblique optical image using the image warping matrix; and

matching features in the oblique optical image with features in the at least one warped geo-referenced orthoimage.

2. The method of claim 1, further comprising estimating a first position and attitude of the image capturing device at the time the oblique optical image was captured.

3. The method of claim 2, further comprising using the matched features to estimate a second estimated position and attitude of the image capturing device at the time the oblique optical image was captured.

4. The method of claim 3, further comprising determining a difference between the first estimated position and attitude and the second estimated position and attitude, wherein if the difference between the first estimated position and attitude and the second estimated position and attitude is above a threshold, then iteratively performing the following steps until the difference between the first estimated position and attitude and the second estimated position and attitude is below the threshold:

substituting the second estimated position and attitude into the first position and attitude;

capturing an oblique optical image of an area of interest using an image capturing device;

providing digital elevation data of an area that includes the area of interest and at least one geo-referenced orthoimage of an area that includes the area of interest;

correlating the area of interest in the oblique optical image with the digital elevation data to create an image warping matrix;

warping the at least one geo-referenced orthoimage to the perspective of the oblique optical image using the image warping matrix;

matching features in the at least one oblique optical image with features in the warped geo-referenced orthoimage;

using the matched features to calculate a second estimated position and attitude of the image capturing device at the time the oblique optical image was captured; and

determining the difference between the first estimated position and attitude and the second estimated position and attitude.

5. The method of claim 1, wherein the image capturing device is attached to an aircraft.

6. The method of claim 1, wherein only a portion of the oblique optical image is used in correlating the oblique optical image with the digital elevation data.

7. The method of claim 1, wherein the features are matched using the scale-invariant feature transform.

8. The method of claim 1, further comprising removing matching feature errors.

9. A system comprising:

at least one sensor image capturing device configured to capture at least one optical image;

one or more processing devices;

one or more memory devices configured to store digital elevation data and geo-referenced orthoimagery, wherein the one or more memory devices are coupled to the one or more processing devices and including instructions which, when executed by the one or more processing devices, cause the one or more processing devices to;

capture an oblique optical image of an area of interest using the image capturing device;

retrieve digital elevation data of an area that includes the area of interest and at least one geo-referenced orthoimage of an area that includes the area of interest;

correlate the area of interest in the oblique optical image with the digital elevation data to create an image warping matrix;

warp the at least one geo-referenced orthoimage to the perspective of the oblique optical image using the image warping matrix; and

match features in the at least one oblique optical image with features in the warped geo-referenced orthoimage.

10. The system of claim 9, wherein the one or more processing devices are further configured to estimate a first position and attitude of the image capturing device at the time the oblique optical image was captured.

11. The system of claim 10, wherein the one or more processing devices are further configured to use the matched features to estimate a second estimated position and attitude of the image capturing device at the time the oblique optical image was captured.

12. The system of claim 11, wherein the one or more processing devices are further configured to determine a difference between the first estimated position and attitude and the second estimated position and attitude, wherein if the difference between the first estimated position and attitude and the second estimated position and attitude is above a threshold, then iteratively performing the following steps until the difference between the first estimated position and attitude and the second estimated position and attitude is below the threshold:

substitute the second estimated position and attitude into the first position and attitude;

capture an oblique optical image of an area of interest using an image capturing device;

provide digital elevation data of an area that includes the area of interest and at least one geo-referenced orthoimage of an area that includes the area of interest;

correlate the area of interest in the oblique optical image with the digital elevation data to create an image warping matrix;

warp the at least one geo-referenced orthoimage to the perspective of the oblique optical image using the image warping matrix;

match features in the at least one oblique optical image with features in the warped geo-referenced orthoimage;

use the matched features to calculate a second estimated position and attitude of the image capturing device at the time the oblique optical image was captured; and

determine the difference between the first estimated position and attitude and the second estimated position and attitude.

13. The system of claim 9, wherein the system is attached to an aircraft.

14. The system of claim 9, wherein only a portion of the oblique optical image is used in correlating the oblique optical image with the digital elevation data.

15. The system of claim 9, wherein the features are matched using the scale-invariant feature transform.

16. The system of claim 9, wherein the one or more processing devices are further configured to remove the matching feature errors.

17. A program product comprising a computer readable medium on which program instructions are embodied, wherein the program instructions are configured, when executed by at least one programmable processor, to cause the at least one programmable processor:

to capture an oblique optical image of an area of interest using a image capturing device,

to estimate a first position and attitude of the image capturing device at the time the oblique optical image was captured;

to provide digital elevation data of an area that includes the area of interest and at least one geo-referenced orthoimage of an area that includes the area of interest;

to correlate the area of interest in the oblique optical image with the digital elevation data to create an image warping matrix;

to warp the at least one geo-referenced orthoimage to the perspective of the oblique optical image using the image warping matrix;

to match features in the at least one oblique optical image with features in the warped geo-referenced orthoimage; and

to use the matched features to estimate a second estimated position and attitude of the image capturing device at the time the oblique optical image was captured.

18. The computer program product of claim 17, wherein only a portion of the oblique optical image is used to correlate the oblique optical image with the digital elevation data.

19. The computer program product of claim 17, wherein the processing device is further configured to determine whether the difference between the first estimated position and attitude and the second estimated position and attitude is above a threshold.

20. The computer program product of claim 18, wherein if the difference between the first estimated position and attitude and the second estimated position and attitude is above a threshold, then the program instructions are further configured to substitute the second estimated position and attitude into the first estimated position and attitude and repeat the program instructions:

to capture an oblique optical image of an area of interest using a image capturing device,

to provide digital elevation data of an area that includes the area of interest and at least one geo-referenced orthoimage of an area that includes the area of interest;

to correlate the oblique optical image with the digital elevation data to create an image warping matrix;

to warp the at least one geo-referenced orthoimage to the perspective of the oblique optical image using the image warping matrix;

to match features in the at least one oblique optical image with features in the warped geo-referenced orthoimage; and

to use the matched features to estimate a second position and attitude of the image capturing device at the time the oblique optical image was captured.