System and methods for autonomous tracking and surveillance

• US 8,718,838 B2
• Filed: 12/14/2007
• Issued: 05/06/2014
• Est. Priority Date: 12/14/2007
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Kokkeby, Kristen L., Lutter, Robert P., Munoz, Michael L., Cathey, Frederick W., Hilliard, David J., Olson, Trevor L.
• Primary Examiner(s): MUSTAFA, IMRAN K
• Application Number:
  US11/956,711
• Publication Number:
  US 20090157233A1
• Time in Patent Office: 2,335 Days
• Field of Search: 701/3, 701/301
• US Class Current: 701/3
• CPC Class Codes: G01S 13/723 by using numerical data
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target

Abstract

A system and methods for autonomously tracking and simultaneously providing surveillance of a target from air vehicles. In one embodiment the system receives inputs from outside sources, creates tracks, identifies the targets and generates flight plans for unmanned air vehicles (UAVs) and camera controls for surveillance of the targets. The system uses predictive algorithms and aircraft control laws. The system comprises a plurality of modules configured to accomplish these tasks. One embodiment comprises an automatic target recognition (ATR) module configured to receive video information, process the video information, and produce ATR information including target information. The embodiment further comprises a multi-sensor integrator (MSI) module configured to receive the ATR information, an air vehicle state input and a target state input, process the inputs and produce track information for the target. The embodiment further comprises a target module configured to receive the track information, process the track information, and produce predicted future state target information. The embodiment further comprises an ownship module configured to receive the track information, process the track information, and produce predicted future state air vehicle information. The embodiment further comprises a planner module configured to receive the predicted future state target information and the predicted future state air vehicle information and generate travel path information including flight and camera steering commands for the air vehicle.

Claims

1. A method of autonomously tracking a moving target from an airborne air vehicle, the method comprising the steps of:

receiving video information input to an automatic target recognition (ATR) module comprising first hardware, the video information comprising target state information of motion of the target including at least speed and acceleration of the target, processing the video information, and producing ATR information;

receiving the ATR information, air vehicle state information, and target state information as inputs to a multi-sensor integrator (MSI) comprising second hardware, wherein a first set of sensors detects states related to the vehicle and a second set of sensors detects states related to the target;

integrating in the multi-sensor integrator the video information, the air vehicle state information, and the target state information to produce track information, the track information including kinematic information of the vehicle and target information data;

receiving the track information as an input to a target module comprising third hardware, processing the track information, predicting a future state of the target and producing target information;

receiving the track information as an input to an ownship module comprising fourth hardware, processing the track information, predicting a future state of the air vehicle and producing ownship information, the ownship information including a flight path configured to reduce a likelihood of discovery;

receiving the target information and the ownship information as inputs to a planner module comprising fifth hardware and generating a travel path for the air vehicle;

automatically generating, using a first physical processor, control commands for a camera using the predicted future state of the target and the predicted future state of the air vehicle, the control commands including pointing, focus, and zoom to obtain the video information; and

generating a weave plan for the air vehicle, using a second physical processor, wherein an execution of the weave plan comprises moving back and forth across a path of the target and behind the target.

2. The method of claim 1, further comprising the step of storing the track information as a track file.

3. The method of claim 1, further comprising at least one of:

generating a loiter plan for the air vehicle, the loiter plan being designed to maintain a desired distance between the air vehicle and the target while the target is stationary;

generating a region search plan for the air vehicle, the region search plan being designed to allow the air vehicle to scan an area;

analyzing an anticipated path of travel for the air vehicle and determining whether the path crosses into any restricted airspace or intersects any obstructions;

cataloging information about the target;

calibrating navigational sensors aboard the air vehicle;

predicting a flight path for the air vehicle; and

recording data associated with a tracking operation.

4. The method of claim 1, further comprising the step of predicting, using a camera module, future imaging characteristics for a camera aboard the air vehicle.

5. The method of claim 4, wherein the air vehicle further comprises a camera and a camera command module configured to control pointing, focus, and zoom of the camera.

6. The method of claim 1 wherein, a weave corridor module provides information to a planner to generate the weave plan.

7. The method of claim 1, wherein at least a portion of the boundaries coincide with at least one of a national border, an edge of a no fly zone and a physical obstruction.

8. The method of claim 1, further comprising generating a loiter plan for the air vehicle, the loiter plan being designed to maintain a desired distance between the air vehicle and the target while the target is stationary.

9. The method of claim 8, wherein a loiter circle module provides information to a planner is configured to generate the loiter plan.

10. The method of claim 1, further comprising generating a region search plan for the air vehicle, the region search plan being designed to allow the air vehicle to scan an area.

11. The method of claim 10, wherein a region search module provides information to a planner is configured to generate the region search plan.

12. The method of claim 1, further comprising the step of:

determining whether the path crosses into any restricted airspace or intersects any obstructions based on an analysis of an anticipated path of travel for the air vehicle.

13. The method of claim 1, further comprising the step of:

cataloging information about the target in an automatic target recognition module.

14. The method of claim 1, further comprising the step of:

predicting future imaging characteristics for the camera aboard the air vehicle.

15. The method of claim 1, wherein the air vehicle is an unmanned aircraft.

16. The method of claim 1, wherein the ATR is positioned at a ground station.

17. The method of claim 1, wherein the weave plan is designed to maintain a desired distance between the air vehicle and the target while the target is moving and while maintaining the air vehicle within predetermined boundaries.

18. The method of claim 1, wherein the first hardware, the second hardware, the third hardware, the fourth hardware, and the fifth hardware are all part of one computer system, and wherein the first physical processor and the second physical processor are the same.

19. A method of tracking a moving target from an airborne air vehicle, the method comprising the steps of:

receiving, at a physical processor, video information comprising target state information of motion of the target including at least speed and acceleration of the target;

receiving, at the physical processor, air vehicle state information;

integrating, using the video information, the air vehicle state information, and the target state information to generate track information, the track information including kinematic information of the vehicle and target information data;

predicting, using the physical processor, a future state of the target;

predicting, using the physical processor, a future state of the air vehicle;

automatically generating, using the physical processor, a travel path for the air vehicle, including one of a loiter plan and a weave plan, wherein the travel path includes a flight path configured to reduce a likelihood of discovery of the air vehicle;

generating, using the physical processor, control commands for a camera using the predicted future state of the target and the predicted future state of the air vehicle, the control commands including pointing, focus, and zoom; and

generating, using the physical processor, a region search plan for the air vehicle, the region search plan being designed to allow the air vehicle to scan an area; and

storing the region search plan in a non-transitory computer readable storage medium.

20. The method of claim 19, further comprising analyzing an anticipated path of travel for the air vehicle and determining whether the path crosses into any restricted airspace.