Detection, classification, and tracking of surface contacts for maritime assets

• US 10,055,648 B1
• Filed: 04/15/2016
• Issued: 08/21/2018
• Est. Priority Date: 04/16/2015
• Status: Active Grant

• Current Assignee: BAE Systems Information and Electronic Systems Integration Incorporated (BAE Systems Plc)
• Original Assignee: BAE Systems Information and Electronic Systems Integration Incorporated (BAE Systems Plc)
• Inventors: Grigsby, Will R., Louchard, Eric, Schorr, David J., Nord, Mitchell J., McGaughey, Joseph, Mathur, Somit S.
• Primary Examiner(s): Yang, Weiwen
• Application Number:
  US15/130,227
• Time in Patent Office: 858 Days
• Field of Search: None
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 2201/07 Target detection

Abstract

Techniques are provided for detection, classification and tracking of surface contacts based on multispectral image data from sensors on maritime assets. An example system may include an anomaly detection circuit configured to detect an object based on anomalies in the variance of pixels of water surface image data. The system may also include an object motion tracking circuit configured to analyze motion of the object relative to water surface waves. The analysis may compensate for motion of the asset and sensors. The system may further include an object classification circuit configured to classify the object as an object of interest, based on the analyzed motion of the object, and as a threat, further based on size estimation, edge detection, surface texture analysis, and volume analysis. The range from the asset to the object may be estimated based on the relation of the object to a detected or estimated horizon line.

Claims

1. A processor-implemented method for object detection and tracking, the method comprising:

receiving, by a processor, water surface image data from one or more imaging sensors associated with a maritime asset;

detecting, by the processor, an object based on anomalies in variance of pixels in the image data;

detecting, by the processor, a horizon line in the image data and estimating a range from the maritime asset to the object based on the location of the object in the image data relative to the horizon line;

analyzing, by the processor, motion of the object relative to water surface waves, the analyzing further comprising compensation for motion of the asset and associated sensors; and

classifying, by the processor, the object as an object of interest based on the estimated range and the analyzed motion of the object.

2. The method of claim 1, further comprising classifying the object of interest as a threat based on one or more of size estimation, edge detection, surface texture analysis, and volume analysis of the object of interest.

3. The method of claim 1, wherein the image data is multispectral and the sensors include one or more of:

electro-optic sensors, infrared sensors, radar sensors and laser sensors.

4. The method of claim 1, wherein the motion of the asset and associated sensors is determined based on data from one or more of a global positioning system, an inertial navigation system, and a sensor pointing vector generator.

5. The method of claim 1, further comprising performing adaptive contrast enhancement on the image data.

6. The method of claim 1, further comprising performing pixel registration on the image data to spatially align pixels between the image data from the imaging sensors.

7. The method of claim 1, further comprising estimating kinematic states and generating tracks for each of the objects.

8. A system for object detection and tracking, the system comprising:

a variance anomaly detection circuit to detect an object based on anomalies in variance of pixels in water surface image data received from one or more imaging sensors associated with a maritime asset;

the variance anomaly detection circuit further to detect a horizon line in the image data and estimate a range from the maritime asset to the object based on the location of the object in the image data relative to the horizon line;

an object motion tracking circuit to analyze motion of the object relative to water surface waves, the analysis further comprising compensation for motion of the asset and associated sensors; and

an object classification circuit to classify the object as an object of interest based on the estimated range and the analyzed motion of the object.

9. The system of claim 8, wherein the object classification circuit is further to classify the object of interest as a threat based on one or more of size estimation, edge detection, surface texture analysis, and volume analysis of the object of interest.

10. The system of claim 8, further comprising a multispectral image acquisition circuit to receive the image data in a multispectral format;

and the sensors include one or more of;

electro-optic sensors, infrared sensors, radar sensors and laser sensors.

11. The system of claim 8, further comprising an object geolocation circuit to determine the motion of the asset and associated sensors based on data from one or more of a global positioning system, an inertial navigation system and a sensor pointing vector generator.

12. The system of claim 8, further comprising a contrast enhancement circuit to perform adaptive contrast enhancement on the image data.

13. The system of claim 8, further comprising a pixel registration circuit to spatially align pixels between the image data from the imaging sensors.

14. The system of claim 8, further comprising a multi-sensor fusion and tracking circuit to estimate kinematic states and generate tracks for each of the objects.

15. At least one non-transitory computer readable storage medium having instructions encoded thereon that, when executed by one or more processors, result in the following operations for object detection and tracking, the operations comprising:

receiving water surface image data from one or more imaging sensors associated with a maritime asset;

detecting an object based on anomalies in variance of pixels in the image data;

detecting a horizon line in the image data and estimating a range from the maritime asset to the object based on the location of the object in the image data relative to the horizon line;

analyzing motion of the object relative to water surface waves, the analyzing further comprising compensation for motion of the asset and associated sensors; and

classifying the object as an object of interest based on the estimated range and the analyzed motion of the object.

16. The computer readable storage medium of claim 15, the operations further comprising classifying the object of interest as a threat based on one or more observations inferred from the image data.

17. The computer readable storage medium of claim 15, wherein the image data is multispectral and the sensors include one or more of:

electro-optic sensors, infrared sensors, radar sensors and laser sensors.

18. The computer readable storage medium of claim 15, wherein the motion of the asset and associated sensors is determined based on data from one or more of a global positioning system, an inertial navigation system and a sensor pointing vector generator.

19. The computer readable storage medium of claim 15, the operations further comprising performing adaptive contrast enhancement on the image data.

20. The computer readable storage medium of claim 15, the operations further comprising performing pixel registration on the image data to spatially align pixels between the image data from the imaging sensors.

21. The computer readable storage medium of claim 15, the operations further comprising estimating kinematic states and generating tracks for each of the objects.