Machine vision enabled swarm guidance technology

• US 10,073,454 B2
• Filed: 03/17/2016
• Issued: 09/11/2018
• Est. Priority Date: 03/17/2016
• Status: Active Grant

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Inventors: Aboutalib, Omar, Evans, David W., Larson, Gregory L., Nguyen, Annie D., Ramroth, Richard R.
• Primary Examiner(s): Gregory, Bernarr E
• Application Number:
  US15/073,403
• Publication Number:
  US 20180164820A1
• Time in Patent Office: 908 Days
• CPC Class Codes: B64U 2101/15 for conventional or electronic warfare
• CPC Class Codes: B64U 2101/18 for dropping bombs; for firing ammunition
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/102 adapted for flying in formations
• CPC Class Codes: B64U 2201/104 using satellite radio beacon positioning systems, e.g. GPS
• CPC Class Codes: B64U 30/12 Variable or detachable wings, e.g. wings with adjustable sweep
• CPC Class Codes: B64U 50/19 using electrically powered motors
• CPC Class Codes: B64U 70/20 for releasing or capturing UAVs in flight by another aircraft
• CPC Class Codes: B64U 80/82 Airborne vehicles
• CPC Class Codes: G05D 1/00 Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/0808 specially adapted for aircraft
• CPC Class Codes: G05D 1/104 involving a plurality of aircrafts, e.g. formation flying traffic control systems for aircraft G08G5/00
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30212 Military
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 2201/07 Target detection
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A system and method for controlling a swarm of UAVs that are stored on and released from an airborne platform, fly to and destroy a target, where the UAVs download target information from the airborne platform before being released therefrom, do not communicate with each other or the airborne platform while in flight, and do not depend of the presence of GPS. Each UAV includes a vision sensor that provides image data, a navigation module that receives the image data and causes the UAV to navigate and fly towards the target, and a target destruction module that receives the image data and causes the UAV to destroy the target.

Claims

1. A target swarm system comprising:

an airborne platform; and

a plurality of unmanned aerial vehicles (UAVs) configured to be launched from the platform, each UAV including a vision sensor providing image data after the UAV is launched from the platform, a navigation module receiving the image data and causing the UAV to fly towards a target, a target destruction module receiving the image data and causing the UAV to engage the target, and munitions to destroy the target,

wherein the airborne platform includes a weapon sensor model that identifies the vision sensor on each UAV and provides target type and state and environment information to each UAV, and a reduced dimension feature extraction model that constructs grids and a target contour dome over the target.

2. The system according to claim 1 wherein the UAVs download target information from the airborne platform before being launched therefrom.

3. The system according to claim 1 wherein the navigation module includes a sensing layer having weapons navigation sensors and weapons databases that identify terrain and landmarks around the target, a processing layer that provides sensor processing and environment feature detection, feature correlation and tracking, geo-registration and sensor resource management, a measurement and abstraction layer that extracts measurement data, and a navigation and fusion layer that provides UAV state information of position, velocity, attitude and error estimates.

4. The system according to claim 1 wherein the target destruction module includes an optimal detection morphology filter that uses downloaded target information to improve detection and segmentation of the target and reduce a target search base, a reduced-dimension and global feature matching module that identifies invariant target features, provides target feature matching, provides a pose estimate of elevation angle and target aspect angle, and provides target verification and tracking, a reduced-dimension contour matching module that provides contour extraction and matching, target verification and tracking and target interpolation, and a terminal local feature matching and tracking module that provides component matching, identification verification, aim-point selection, local feature tracking and robust-to-partial occlusions.

5. The system according to claim 1 wherein the airborne platform is a B2 bomber.

6. The system according to claim 1 wherein the vision sensor is an imaging sensor, a visible light camera, or an electro-optical short/medium wave or long wave infrared camera.

7. The system according to claim 1 wherein each UAV includes retractable wings that are deployed after the UAV is launched from the platform.

8. The system according to claim 1 wherein the plurality of UAVs is hundreds of UAVs.

9. A target swarm system comprising:

an airborne platform; and

a plurality of unmanned aerial vehicles (UAVs) configured to be launched from the platform, each UAV including a vision sensor providing image data after the UAV is launched from the platform and a navigation module receiving the image data and causing the UAV to fly towards a target, wherein the UAVs download target information from the airborne platform before being released therefrom, and do not communicate with each other or the airborne platform after the UAVs are released from the platform, wherein the navigation module includes a sensing layer having weapons navigation sensors and weapons databases that identify terrain and landmarks around the target, a processing layer that provides sensor processing and environment feature detection, feature correlation and tracking, geo-registration and sensor resource management, a measurement and abstraction layer that extracts measurement data, and a navigation and fusion layer that provides UAV state information of position, velocity, attitude and error estimates.

10. The system according to claim 9 wherein the plurality of UAVs is hundreds of UAVs.

11. The system according to claim 9 wherein the vision sensor is an imaging sensor, a visible light camera, or an electro-optical short/medium wave or long wave infrared camera.

12. The system according to claim 9 wherein each UAV includes retractable wings that are deployed after the UAV is launched from the platform.

13. A target swarm system comprising:

an airborne platform; and

a plurality of unmanned aerial vehicles (UAVs) configured to be launched from the platform, wherein the UAVs download target information from the airborne platform before being launched therefrom, each UAV including a vision sensor providing image data after the UAV is launched from the platform, a navigation module receiving the image data and causing the UAV to fly towards a target, a target destruction module receiving the image data and causing the UAV to engage the target, and munitions to destroy the target, wherein the airborne platform includes a weapon sensor model that identifies the vision sensor on each UAV and provides target type and state and environment information to each UAV, and a reduced dimension feature extraction model that constructs grids and a target contour dome over the target, the navigation module includes a sensing layer having weapons navigation sensors and weapons databases that identify terrain and landmarks around the target, a processing layer that provides sensor processing and environment feature detection, feature correlation and tracking, geo-registration and sensor resource management, a measurement and abstraction layer that extracts measurement data, and a navigation and fusion layer that provides UAV state information of position, velocity, attitude and error estimates, and the target destruction module includes an optimal detection morphology filter that uses the downloaded target information to improve detection and segmentation of the target and reduce a target search base, a reduced-dimension and global feature matching module that identifies invariant target features, provides target feature matching, provides a pose estimate of elevation angle and target aspect angle, and provides target verification and tracking, a reduced-dimension contour matching module that provides contour extraction and matching, target verification and tracking and target interpolation, and a terminal local feature matching and tracking module that provides component matching, identification verification, aim-point selection, local feature tracking and robust-to-partial occlusions.

14. The system according to claim 13 wherein the airborne platform is a B2 bomber.

15. The system according to claim 13 wherein the vision sensor is an imaging sensor, a visible light camera, or an electro-optical short/medium wave or long wave infrared camera.

16. The system according to claim 13 wherein each UAV includes retractable wings that are deployed after the UAV is launched from the platform.

17. The system according to claim 13 wherein the plurality of UAVs is hundreds of UAVs.

18. A target swarm system comprising:

an airborne platform; and

a plurality of unmanned aerial vehicles (UAVs) configured to be launched from the platform, each UAV including a vision sensor providing image data after the UAV is launched from the platform, a navigation module receiving the image data and causing the UAV to fly towards a target, a target destruction module receiving the image data and causing the UAV to engage the target, and munitions to destroy the target,

wherein the target destruction module includes an optimal detection morphology filter that uses downloaded target information to improve detection and segmentation of the target and reduce a target search base, a reduced-dimension and global feature matching module that identifies invariant target features, provides target feature matching, provides a pose estimate of elevation angle and target aspect angle, and provides target verification and tracking, a reduced-dimension contour matching module that provides contour extraction and matching, target verification and tracking and target interpolation, and a terminal local feature matching and tracking module that provides component matching, identification verification, aim-point selection, local feature tracking and robust-to-partial occlusions.