VISION-AIDED AERIAL NAVIGATION

• US 20170328716A1
• Filed: 05/16/2016
• Published: 11/16/2017
• Est. Priority Date: 05/16/2016
• Status: Active Grant

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Inventors: MA, YUNQIAN
• Granted Patent: US 10,054,445 B2
• Time in Patent Office: Days
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G06F 18/2136 based on sparsity criteria, e.g. with an overcomplete basis
• CPC Class Codes: G06F 18/241 relating to the classification model, e.g. parametric or non-parametric approaches
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/17 taken from planes or by drones

Abstract

An aerial vehicle is navigated using vision-aided navigation that classifies regions of acquired still image frames as featureless or feature-rich, and thereby avoids expending time and computational resources attempting to extract and match false features from the featureless regions. The classification may be performed by computing a texture metric as by testing widths of peaks of the autocorrelation function of a region against a threshold, which may be an adaptive threshold, or by using a model that has been trained using a machine learning method applied to a training dataset comprising training images of featureless regions and feature-rich regions. Such machine learning method can use a support vector machine. The resultant matched feature observations can be data-fused with other sensor data to correct a navigation solution based on GPS and/or IMU data.

Claims

1. A method of navigation of an aerial vehicle, the method comprising:

acquiring an aerial image frame from a camera on the aerial vehicle;

partitioning the acquired aerial image frame into a plurality of regions;

classifying one or more of the plurality of regions as featureless or feature-rich by computing an autocorrelation function to determine at least one peak width, and testing the at least one peak width against a threshold;

extracting features for those regions classified as feature-rich and not for those regions classified as featureless; and

navigating the aerial vehicle based at least in part on the extracted features.

2. The method of claim 1, wherein the classifying of the regions is conducted in parallel, with each region classified in an independent software thread.

3. The method of claim 1, further comprising matching features following the extracting features and prior to the navigating the aerial vehicle, wherein matched feature observations are generated by matching features extracted from the regions in the feature extraction to corresponding features in a map feature database.

4. The method of claim 1, further comprising performing perspective warping following the classifying and prior to the extracting features, wherein the perspective of a region is ortho-rectified.

5. The method of claim 1, wherein the partitioning the acquired aerial image frame comprises partitioning the acquired aerial image frame into regions using a watershed segmentation method.

6. The method of claim 1, wherein the partitioning the acquired aerial image frame comprises partitioning the acquired aerial image frame into regions using a grid partition method.

7. The method of claim 3, further comprising performing data fusion following the matching features and prior to the navigating the aerial vehicle, wherein the matched feature observations are combined with inertial measurement unit (IMU) and global positioning system (GPS) sensor readings using a Kalman filter to arrive at a corrected navigation solution suitable for input into an inertial navigation system (INS).

8. The method of claim 7, wherein the Kalman filter fuses readings from a barometric sensor in addition to the matched feature observations and the IMU and GPS sensor readings to arrive at the corrected navigation solution.

9. The method of claim 1, wherein the threshold is dynamically computed as an Otsu threshold statistically derived from a set of peak widths determined from the plurality of regions, the set of peak widths having a bimodal distribution.

10. A system for navigation of an aerial vehicle comprising:

a camera configured to acquire an aerial image frame;

an image processor comprising;

a partitioner configured to partition an acquired aerial image frame into a plurality of regions;

a classifier configured to classify one or more of the plurality of regions as featureless or feature-rich by computing an autocorrelation function to determine at least one peak width, and testing the at least one peak width against a threshold; and

a feature extractor configured to extract features only for those regions classified as feature-rich and not for those regions classified as featureless; and

navigation controls configured to navigate the aerial vehicle based at least in part on the extracted features.

11. The system of claim 10, wherein the classifier is configured to classify the regions in parallel, with each region classified in an independent software thread.

12. The system of claim 10, further comprising a feature matcher configured to generate matched feature observations by matching features extracted from the regions by the feature extractor to corresponding features in a map feature database.

13. The system of claim 12, further comprising a data fusion engine configured to generate a corrected navigation solution by combining the matched feature observations with inertial measurement unit (IMU) and global positioning system (GPS) sensor readings using a Kalman filter, the corrected navigation solution suitable for input into an inertial navigation system (INS).

14. The system of claim 13, wherein the Kalman filter fuses readings from a barometric sensor in addition to the matched feature observations and the IMU and GPS sensor readings to arrive at the corrected navigation solution.

15. The system of claim 10, wherein the classifier dynamically computes the threshold as an Otsu threshold statistically derived from a set of peak widths determined from the plurality of regions, the set of peak widths having a bimodal distribution.

16. A method of navigation of an aerial vehicle comprising:

acquiring an aerial image frame from a camera on the aerial vehicle;

partitioning the acquired aerial image frame into regions;

classifying one or more regions as featureless or feature-rich using a model that has been trained using a machine learning method applied to a training dataset comprising a plurality of training images of featureless regions and a plurality of training images of feature-rich regions;

extracting features for those regions classified as feature-rich and not for those regions classified as featureless; and

navigating the aerial vehicle based at least in part on the extracted features.

17. The method of claim 16, wherein the machine learning method uses a support vector machine.

18. The method of claim 16, wherein the model has been trained using a machine-learning method applied to features extracted from the training images prior to acquiring the aerial image frame.

19. The method of claim 16, further comprising matching features following the extracting features and prior to the navigating the aerial vehicle, wherein matched feature observations are generated by matching features extracted from the regions in the feature extraction step to corresponding features in a map feature database.

20. The method of claim 19, further comprising performing data fusion following the matching features and prior to the navigating the aerial vehicle, wherein the matched feature observations are combined with inertial measurement unit (IMU) and global positioning system (GPS) sensor readings using a Kalman filter to arrive at a corrected navigation solution suitable for input into an inertial navigation system (INS).