Method for vision-aided navigation for unmanned vehicles

• US 10,274,958 B2
• Filed: 11/30/2016
• Issued: 04/30/2019
• Est. Priority Date: 01/22/2015
• Status: Active Grant

• Current Assignee: BAE Systems Information and Electronic Systems Integration Incorporated (BAE Systems Plc)
• Original Assignee: BAE Systems Information and Electronic Systems Integration Incorporated (BAE Systems Plc)
• Inventors: DelMarco, Stephen P., Tom, Victor T.
• Primary Examiner(s): Aghevli, Reza
• Application Number:
  US15/365,210
• Publication Number:
  US 20170329335A1
• Time in Patent Office: 881 Days
• Field of Search: None
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G05D 1/02 Control of position or course in two dimensions
• CPC Class Codes: G05D 1/10 Simultaneous control of position or course in three dimensions G05D1/12 takes precedence
• CPC Class Codes: G05D 1/101 specially adapted for aircraft
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06T 2200/28 involving image processing hardware
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

A method for vision-aided navigation which utilizes an image registration system to update a navigation control system. One exemplary feature of the method includes the use of Enhanced Phase Correlation techniques to create a reference imagery database and register images against the created database. The method may be operated on a non-transitory computer-readable medium storing a plurality of instructions which when executed by one or more processors causes the one or more processors to perform the method for vision-aided navigation which utilizes an image registration system to update a navigation control system.

Claims

1. A method for vision-aided navigation comprising:

identifying fiducials that persist across multi-temporal imagery;

creating a reference imagery database based, at least partially, on the identified fiducials;

obtaining test imagery from a vehicle;

registering the test imagery against reference imagery from the reference imagery database;

obtaining a correct registration solution, whereby the correct registration solution indicates that the acquired test imagery matches the reference imagery;

determining coordinates of a location at which the test imagery was acquired;

estimating a position error of the vehicle;

wherein the estimated position error is based, at least partially, on coordinates of a location of the reference imagery;

correcting the position error of the vehicle; and

guiding the vehicle based, at least partially, on the corrected error position.

2. The method for vision-aided navigation of claim 1, further comprising:

generating a pre-planned route of travel for the vehicle, wherein the vehicle includes a guidance system.

3. The method for vision-aided navigation of claim 2, further comprising:

utilizing Enhanced Phase Correlation (EPC) techniques to identify the fiducials that persist across multi-temporal imagery.

4. The method for vision-aided navigation of claim 3, wherein the identified fiducials are image patches of the reference imagery located along the pre-planned route of travel.

5. The method for vision-aided navigation of claim 4, further comprising:

generating at least two waypoints along the pre-planned route of travel; and

establishing an initial position of the vehicle.

6. The method for vision-aided navigation of claim 5, wherein obtaining test imagery from a vehicle is accomplished by acquiring the test imagery with at least one sensor as the vehicle travels along the pre-planned route of travel.

7. The method for vision-aided navigation of claim 1, further comprising:

selecting reference imagery from the reference imagery database based, at least partially, on the acquired test imagery'"'"'s metadata.

8. The method for vision-aided navigation of claim 7, wherein registering the test imagery against reference imagery from the reference imagery database is accomplished by utilizing EPC techniques.

9. The method for vision-aided navigation of claim 1, further comprising:

updating a guidance system of the vehicle.

10. The method for vision-aided navigation of claim 9, further comprising:

determining a correct match between the test image and the reference image with an Enhanced Phase Correlation (EPC) Verification Metric (VM); and

determining an estimate of the accuracy of the registration with a registration error covariance estimation.

11. The method for vision-aided navigation of claim 10, wherein the method for vision-aided navigation further comprises:

searching for a local azimuth angle relative to a nominal reference image; and

registering reference imagery offset in an azimuth angle from the nominal reference image against the acquired test imagery, wherein registering reference imagery offset in an azimuth angle from the nominal reference image against the acquired test imagery is accomplished by utilizing EPC techniques.

12. The method for vision-aided navigation of claim 1, further comprising:

solving misregistration errors by utilizing a sensor parameter search paradigm.

13. A method for vision-aided navigation comprising the steps of:

acquiring an image frame from a sensor on a vehicle including a guidance system;

selecting an ortho-rectified reference image from a reference imagery database;

ortho-rectifying the acquired image frame;

geo-registering the ortho-rectified acquired image frame against the ortho-rectified reference image;

obtaining a correct registration solution, whereby the correct registration solution indicates that the ortho-rectified acquired image frame matches the ortho-rectified reference image;

determining coordinates of a location at which the ortho-rectified acquired image frame was acquired;

determining the vehicle'"'"'s positional alignment error;

wherein the vehicle'"'"'s positional alignment error is based, at least partially, on the coordinates of a location of the ortho-rectified reference image;

updating the vehicle'"'"'s guidance system; and

correcting the vehicle'"'"'s motion.

14. The method for vision-aided navigation of claim 13, further comprising:

utilizing Enhanced Phase Correlation (EPC) techniques to identify fiducials that persist across multi-temporal imagery; and

creating the reference imagery database based, at least partially, on the identified fiducials.

15. The method for vision-aided navigation of claim 13, wherein selecting an ortho-rectified reference image from the reference imagery database is based, at least partially, on sensor metadata collected from the sensor on the vehicle.

16. The method for vision-aided navigation of claim 15, wherein geo-registering the ortho-rectified acquired image frame against the ortho-rectified reference image is accomplished by utilizing EPC techniques.

17. A non-transitory computer-readable medium storing a plurality of instructions which when executed by one or more processors causes the one or more processors to perform a method for vision-aided navigation, the plurality of instructions for the method for vision-aided navigation comprising:

instructions for identifying fiducials that persist across multi-temporal imagery;

instructions for creating a reference imagery database based, at least partially, on the identified fiducials;

instructions for obtaining a test image from a vehicle;

instructions for registering the test image against a reference image from the reference imagery database;

instructions for obtaining a correct registration solution, whereby the correct registration solution indicates that the acquired test image matches the reference image;

instructions for determining coordinates of a location at which the test image was acquired;

instructions for estimating a position error of the vehicle;

wherein the estimated position error is based, at least partially, on coordinates of a location of the reference image;

instructions for correcting the position error of the vehicle; and

instructions for guiding the vehicle based, at least partially, on the corrected error position.

18. The non-transitory computer-readable medium of claim 17, further comprising:

instructions for generating a pre-planned route of travel for the vehicle, wherein the vehicle includes a guidance system; and

wherein the identified fiducials are image patches of the reference imagery located along the pre-planned route of travel.

19. The non-transitory computer-readable medium of claim 17, further comprising:

instructions for utilizing Enhanced Phase Correlation (EPC) techniques to identify the fiducials that persist across multi-temporal imagery;

wherein registering the test image against the reference image is accomplished by utilizing EPC techniques; and

wherein estimating a position error of the vehicle is accomplished by utilizing EPC techniques.

20. The non-transitory computer-readable medium of claim 17, further comprising:

Instructions for selecting reference imagery from the reference imagery database based, at least partially, on the acquired test imagery'"'"'s metadata.