Systems and Methods for Recognizing Objects in Radar Imagery

• US 20180260688A1
• Filed: 05/11/2018
• Published: 09/13/2018
• Est. Priority Date: 07/16/2014
• Status: Active Grant

• Current Assignee: General Dynamics Mission Systems Inc. (General Dynamics Corporation)
• Original Assignee: Deep Learning Analytics, LLC (General Dynamics Corporation)
• Inventors: KAUFHOLD, John Patrick
• Granted Patent: US 10,643,123 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 13/90 using synthetic aperture techniques , e.g. synthetic aperture radar [SAR] techniques
• CPC Class Codes: G01S 13/9027 Pattern recognition for feature extraction
• CPC Class Codes: G01S 13/9029 specially adapted for moving target detection within a single SAR image or within multiple SAR images taken at the same time
• CPC Class Codes: G01S 13/904 SAR modes
• CPC Class Codes: G01S 7/417 involving the use of neural networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/0463 Neocognitrons

Abstract

The present invention is directed to systems and methods for detecting objects in a radar image stream. Embodiments of the invention can receive a data stream from radar sensors and use a deep neural network to convert the received data stream into a set of semantic labels, where each semantic label corresponds to an object in the radar data stream that the deep neural network has identified. Processing units running the deep neural network may be collocated onboard an airborne vehicle along with the radar sensor(s). The processing units can be configured with powerful, high-speed graphics processing units or field-programmable gate arrays that are low in size, weight, and power requirements. Embodiments of the invention are also directed to providing innovative advances to object recognition training systems that utilize a detector and an object recognition cascade to analyze radar image streams in real time. The object recognition cascade can comprise at least one recognizer that receives a non-background stream of image patches from a detector and automatically assigns one or more semantic labels to each non-background image patch. In some embodiments, a separate recognizer for the background analysis of patches may also be incorporated. There may be multiple detectors and multiple recognizers, depending on the design of the cascade. Embodiments of the invention also include novel methods to tailor deep neural network algorithms to successfully process radar imagery, utilizing techniques such as normalization, sampling, data augmentation, foveation, cascade architectures, and label harmonization.

Claims

1. A system for recognizing objects in a radar image stream, comprising:

a flying vehicle;

a synthetic aperture radar (“

SAR”

) sensor on the flying vehicle;

a portable processor on the flying vehicle;

an object recognizer module loaded on the portable processor, the object recognizer module comprising a deep learning network trained on a plurality of SAR image chips, wherein each SAR image chip has been paired with at least one of a plurality of semantic labels;

the portable processor configured to receive from the SAR sensor a stream of SAR image data reflected from a target object, the processor further configured to provide the received stream of SAR image data to the object recognizer module;

the object recognizer module programmed to recognize the target object in the received stream of SAR image data by invoking the trained deep learning network to process the received stream of SAR image data into a recognized semantic label corresponding to the target object, where the recognized semantic label is one of the plurality of semantic labels; and

the portable processor further configured to provide the recognized semantic label to an operational control module onboard the flying vehicle.

2. The system of claim 1, wherein each SAR image chip is a two-dimensional set of pixels comprising magnitude and phase information derived from radar return signals.

3. The system of claim 1, wherein each SAR image chip has been paired with at least one of the plurality of semantic labels according to a wrangling process.

4. The system of claim 1, wherein the portable processor includes a high-speed graphics processing unit and at least a portion of the trained deep learning network executes on the high-speed graphics processing unit.

5. The system of claim 1, wherein the portable processor includes a field-programmable gate array and at least a portion of the trained deep learning network executes on the high-speed graphics processing unit.

6. The system of claim 1, wherein the portable processor is further configured to normalize the received stream of SAR image data.

7. The method of claim 1, wherein the trained deep learning network further comprises an object recognition cascade.

8. The method of claim 7, wherein the object recognition cascade comprises a neural network with more than three layers.

9. The method of claim 8, wherein one of the three layers comprises a convolutional layer.

10. The method of claim 8, wherein one of the three layers comprises a locally connected layer.

13. The method of claim 1, wherein the portable processor includes a field-programmable gate array and at least a portion of the trained deep learning network executes on the high-speed graphics processing unit.

11. A method for recognizing objects in a synthetic aperture radar (“

SAR”

) image, comprising;

receiving a stream of radar data at a SAR sensor on a flying vehicle, where the stream of radar data is reflected from a target object;

forming a two-dimensional SAR image from the stream of radar data;

providing the SAR image to an object recognizer executing on a portable processor, the object recognizer comprising a deep learning network that has been trained on a plurality of SAR image chips, where each SAR image chip has been paired with at least one of a plurality of semantic labels;

the object recognizer recognizing the target object in the SAR image by invoking the trained deep learning network to process the SAR image into a recognized semantic label corresponding to the target object, where the recognized semantic label is one of the plurality of semantic labels; and

providing the recognized semantic label to an operational control module onboard the flying vehicle.

12. The method of claim 11, wherein the portable processor includes a high-speed graphics processing unit and at least a portion of the trained deep learning network executes on the high-speed graphics processing unit.

14. The method of claim 11, wherein each SAR image chip has been paired with at least one of the plurality of semantic labels according to a wrangling process.

15. The method of claim 11, wherein the forming step includes normalizing the received stream of radar data.

16. The method of claim 11, wherein the trained deep learning network comprises an object recognition cascade.

17. The method of claim 16, wherein the object recognition cascade comprises a neural net with more than three layers.

18. The method of claim 17, wherein one of the three layers comprises a convolutional layer.

19. The method of claim 17, wherein one of the three layers comprises a locally connected layer.