SYSTEM AND METHOD PROVIDING AUTOMATIC ALIGNMENT OF AERIAL/SATELLITE IMAGERY TO KNOWN GROUND FEATURES

• US 20180096451A1
• Filed: 09/30/2016
• Published: 04/05/2018
• Est. Priority Date: 09/30/2016
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: MISRA, Avishkar, HERENA ULLOA, Zazhil Ha, REYES MARTINEZ, Juan Carlos, RAVADA, Siva
• Granted Patent: US 10,089,712 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 2207/10036 Multispectral image; Hyperspectral image
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/30181 Earth observation
• CPC Class Codes: G06T 3/14 Transformations for image registration, e.g. adjusting or mapping for alignment of images
• CPC Class Codes: G06T 5/30 Erosion or dilatation, e.g. thinning
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: H04N 23/63 by using electronic viewfinders

Abstract

Systems, methods, and other embodiments are disclosed for correcting errors in the geo-spatial locations of acquired image data. In one embodiment, acquired aerial or satellite image data is segmented to generate extracted boundary data. The extracted boundary data represents boundaries of features of a portion of the Earth'"'"'s surface, but at incorrect geo-spatial coordinates. The extracted boundary data is matched to expected boundary data derived from ground truth data. The expected boundary data represents boundaries of the features at correct geo-spatial coordinates. Adjustment parameters are generated that represent a geo-spatial misalignment between the extracted boundary data and the expected boundary data. Metadata in a header of the acquired image data is modified to include the adjustment parameters. The adjustment parameters may be applied to the acquired image data to generate corrected image data at correct geo-spatial coordinates.

Claims

1. A computer-implemented method performed by a computing device where the computing device includes at least a processor for executing instructions from a memory, the method comprising:

segmenting, via at least the processor, acquired image data to generate extracted boundary data, where the acquired image data is one of aerial image data or satellite image data representing features of a portion of the Earth'"'"'s surface at incorrect geo-spatial coordinates, and where the extracted boundary data represents boundaries of the features at the incorrect geo-spatial coordinates;

matching, via at least the processor, the extracted boundary data to expected boundary data derived from a set of ground truth data, where the expected boundary data represents the boundaries of the features at correct geo-spatial coordinates;

estimating, via at least the processor, adjustment parameters representing an amount and a type of a geo-spatial misalignment between the extracted boundary data and the expected boundary data;

applying, via at least the processor, the adjustment parameters to the acquired image data to generate corrected image data at corrected geo-spatial coordinates to mitigate the geo-spatial misalignment;

analyzing, via at least the processor, the corrected image data to determine at least one environmental condition of the portion of the Earth'"'"'s surface;

generating, via at least the processor, control output data based on the at least one environmental condition; and

controlling at least one external mechanism, at least by communicating the control output data to the at least one external mechanism, to modify the at least one environmental condition.

2. The method of claim 1, wherein the segmenting includes:

performing at least a thresholding operation on the acquired image data to generate binary image data;

performing at least a morphological operation on the binary image data to generate noise-reduced image data; and

performing at least an edge detection operation on the noise-reduced image data to generate the extracted boundary data.

3. The method of claim 1, wherein the matching includes performing at least one of a nearest point process, a least-squares process, or a random sample consensus process to match the extracted boundary data to the expected boundary data.

4. The method of claim 1, wherein the controlling at least one external mechanism, at least by communicating the control output data to the at least one external mechanism, to modify the at least one environmental condition includes at least one of:

controlling an irrigation system to apply water to crops at the corrected geo-spatial coordinates in a farm field, as represented by the corrected image data, to modify the environmental condition being that of drought;

controlling an aerial drone to apply fertilizer to crops at the corrected geo-spatial coordinates in a farm field, as represented by the corrected image data, to modify the environmental condition being that of inadequate crop growth;

controlling an aerial drone to apply a pesticide to crops at the corrected geo-spatial coordinates in a farm field, as represented by the corrected image data, to modify the environmental condition being that of insect infestation;

controlling a traffic signal device to control traffic flow at the corrected geo-spatial coordinates on a heavily used road, as represented by the corrected image data, to modify the environmental condition being that of traffic congestion;

controlling a law enforcement vehicle to navigate to the corrected geo-spatial coordinates in a forest, as represented by the corrected image data, to modify the environmental condition being that of illegal deforestation; and

controlling a military aircraft to navigate to the corrected geo-spatial coordinates of an enemy site, as represented by the corrected image data, to modify the environmental condition being that of construction of an enemy military installation.

5. The method of claim 1, wherein the geo-spatial misalignment is due to a spatial translation error in the incorrect geo-spatial coordinates of the acquired image data.

6. The method of claim 1, wherein the geo-spatial misalignment is due to a spatial scaling error in the incorrect geo-spatial coordinates of the acquired image data.

7. The method of claim 1, wherein the geo-spatial misalignment is due to a spatial rotational error in the incorrect geo-spatial coordinates of the acquired image data.

8. The method of claim 1, wherein the geo-spatial misalignment is due to a spatial skewing error in the incorrect geo-spatial coordinates of the acquired image data.

9. The method of claim 1, further comprising modifying, via at least the processor, metadata in a header of the acquired image data to include the adjustment parameters.

10. The method of claim 1, wherein the incorrect geo-spatial coordinates of the acquired image data are one of global positioning system coordinates, latitude and longitude coordinates, or universal transverse mercator coordinates.

11. A computer system, comprising:

a processor;

a data cache configured to store;

acquired image data, where the acquired image data is one of aerial image data or satellite image data representing features of a portion of the Earth'"'"'s surface at incorrect geo-spatial coordinates, and

a set of ground truth data, where the set of ground truth data represents the features of the portion of the Earth'"'"'s surface at correct geo-spatial coordinates;

a segmentation module stored in a non-transitory computer-readable medium including instructions that when executed cause the processor to segment the acquired image data to generate extracted boundary data, where the extracted boundary data represents boundaries of the features at the incorrect geo-spatial coordinates;

a matching module stored in the non-transitory computer-readable medium including instructions that when executed cause the processor to match the extracted boundary data to expected boundary data derived from the set of ground truth data, where the expected boundary data represents the boundaries of the features at the correct geo-spatial coordinates;

an adjustment module stored in the non-transitory computer-readable medium including instructions that when executed cause the processor to estimate adjustment parameters representing an amount and a type of geo-spatial misalignment between the extracted boundary data and the expected boundary data; and

a modification module stored in the non-transitory computer-readable medium including instructions that when executed cause the processor to modify metadata in a header of the acquired image data to include the adjustment parameters.

12. The computer system of claim 11, wherein the segmentation module includes:

a thresholding module stored in the non-transitory computer-readable medium including instructions that when executed cause the processor to perform a thresholding operation on the acquired image data to generate binary image data;

a morphological module stored in the non-transitory computer-readable medium including instructions that when executed cause the processor to perform a morphological operation on the binary image data to generate noise-reduced image data; and

an edge detection module stored in the non-transitory computer-readable medium including instructions that when executed cause the processor to perform at least an edge detection operation on the noise-reduced image data to generate the extracted boundary data.

13. The computer system of claim 11, wherein the matching module is configured to perform at least one of a nearest point process, a least-squares process, or a random sample consensus process to match the extracted boundary data to the expected boundary data.

14. The computer system of claim 11, further comprising:

a correction module stored in the non-transitory computer-readable medium including instructions that when executed cause the processor to apply the adjustment parameters to the acquired image data to generate corrected image data at corrected geo-spatial coordinates to mitigate the geo-spatial misalignment;

an analysis module stored in the non-transitory computer-readable medium including instructions that when executed cause the processor to analyze the corrected image data to determine at least one environmental condition of the portion of the Earth'"'"'s surface; and

a control module stored in the non-transitory computer-readable medium including instructions that when executed cause the processor to;

generate control output data based on the at least one environmental condition, and

control at least one external mechanism, at least by communicating the control output data to the at least one external mechanism, to modify the at least one environmental condition.

15. The computer system of claim 14, wherein the at least one external mechanism includes one of:

an irrigation system configured to apply, in response to the control output data, water to crops at the corrected geo-spatial coordinates in a farm field, as represented by the corrected image data, to modify the environmental condition being that of drought;

an aerial drone configured to apply, in response to the control output data, fertilizer to crops at the corrected geo-spatial coordinates in a farm field, as represented by the corrected image data, to modify the environmental condition being that of inadequate crop growth;

an aerial drone configured to apply, in response to the control output data, a pesticide to crops at the corrected geo-spatial coordinates in a farm field, as represented by the corrected image data, to modify the environmental condition being that of insect infestation;

a traffic signal device configured to control, in response to the control output data, traffic flow at the corrected geo-spatial coordinates on heavily used roads, as represented by the corrected image data, to modify the environmental condition being that of traffic congestion;

a law enforcement vehicle configured to navigate to, in response to the control output data, the corrected geo-spatial coordinates in a forest, as represented by the corrected image data, to modify the environmental condition being that of illegal deforestation; and

a military aircraft configured to navigate to, in response to the control output data, the corrected geo-spatial coordinates of an enemy site, as represented by the corrected image data, to modify the environmental condition being that of construction of an enemy military installation.

16. The computer system of claim 11, further comprising a visual user interface module stored in the non-transitory computer-readable medium including instructions that when executed cause the processor to provide a graphical user interface that controls user selection of the acquired image data from the data cache in response to user commands.

17. The computer system of claim 16, further comprising a display screen configured to display and facilitate user interaction with at least the graphical user interface provided by the visual user interface module.

18. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a computing device, cause the computing device to at least:

segment, via the one or more processors, acquired image data to generate extracted boundary data, where the acquired image data is one of aerial image data or satellite image data representing features of a portion of the Earth'"'"'s surface at incorrect geo-spatial coordinates, and where the extracted boundary data represents boundaries of the features at the incorrect geo-spatial coordinates;

match, via the one or more processors, the extracted boundary data to expected boundary data derived from a set of ground truth data, where the expected boundary data represents the boundaries of the features at correct geo-spatial coordinates;

estimate, via the one or more processors, adjustment parameters representing an amount and a type of a geo-spatial misalignment between the extracted boundary data and the expected boundary data; and

modify, via the one or more processors, metadata in a header of the acquired image data to include the adjustment parameters.

19. The non-transitory computer-readable medium of claim 18, wherein the instructions further comprise instructions that, when executed by the one or more processors, cause the computing device to at least:

apply, via the one or more processors, the adjustment parameters to the acquired image data to generate corrected image data at corrected geo-spatial coordinates to mitigate the geo-spatial misalignment;

analyze, via the one or more processors, the corrected image data to determine at least one environmental condition of the portion of the Earth'"'"'s surface;

generate, via the one or more processors, control output data based on the at least one environmental condition; and

control, via the control output data, at least one external mechanism to modify the at least one environmental condition.

20. The non-transitory computer-readable medium of claim 18, wherein the incorrect geo-spatial coordinates of the acquired image data are one of global positioning system coordinates, latitude and longitude coordinates, or universal transverse Mercator coordinates.