SYSTEM AND METHOD FOR REAL-TIME 3-D OBJECT TRACKING AND ALERTING VIA NETWORKED SENSORS

• US 20100128110A1
• Filed: 02/26/2009
• Published: 05/27/2010
• Est. Priority Date: 11/21/2008
• Status: Active Grant

• Current Assignee: Raytheon Company (Rtx Corporation)
• Original Assignee: Raytheon Company (Rtx Corporation)
• Inventors: Mavromatis, Theofanis
• Granted Patent: US 9,520,040 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G08B 13/19641 Multiple cameras having overlapping views on a single scene
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors

Abstract

The invention is an integrated system consisting of a network of two-dimensional sensors (such as cameras) and processors along with target detection, data fusion, tracking, and alerting algorithms to provide three-dimensional, real-time, high accuracy target tracks and alerts in a wide area of interest. The system uses both target kinematics (motion) and features (e.g., size, shape, color, behavior, etc.) to detect, track, display, and alert users to potential objects of interest.

Claims

1. An object detection and tracking system comprising:

two or more spatially separated two-dimensional (2-D) sensors with overlapping fields of view for detecting and three-dimensional (3-D) tracking of moving objects through observations;

at least one image processor for converting each sensor'"'"'s observations into 2-D spatial characteristics and extracting features of the different objects; and

at least one tracking processor for fusing the 2-D spatial characteristics and feature data from the respective sensors into 3-D kinematic and feature data of the objects.

2. The system of claim 1 further comprising at least one behavior analysis and alert processor for analyzing the 3-D kinematic and feature data and generating appropriate alerts corresponding to behaviors of interest.

3. The system of claim 1 further comprising at least one display presenting 3-D kinematic and feature data.

4. The system of claim 2 further comprising at least one display presenting 3-D kinematic, feature, and alert data.

5. The system of claim 1 wherein the sensors are passive electro-optical/infrared sensors.

6. The system of claim 1 with the addition of one or more pan-tilt-zoom (PTZ) cameras for object identification, wherein the PTZ cameras can be activated automatically or from operator input to track a particular object.

7. The system of claim 1 with at least one spatially oriented and time-synchronized 3-D sensor detecting moving objects through observations, wherein the 3-D observation data is fed directly to the tracking processors.

8. The system of claim 1 wherein the 2-D sensors are spatially oriented and time-synchronized.

9. The system of claim 1 wherein the system processes the data in real time.

10. The system of claim 1 also comprising a network connecting the multiple sensors and processors.

11. An object detection and tracking method comprising:

detecting and 3-D tracking of moving objects through observations from two or more spatially separated 2-D sensors with overlapping fields of view;

converting each sensor'"'"'s observations into 2-D spatial characteristics and extracting features of the different objects; and

fusing the 2-D spatial characteristics and feature data from the respective sensors into 3-D kinematic and feature data of the objects.

12. The method of claim 11 further comprising analyzing the 3-D kinematic and feature data and generating appropriate alerts corresponding to behaviors of interest.

13. The method of claim 11 further comprising displaying 3-D kinematic and feature data.

14. The method of claim 12 further comprising displaying 3-D kinematic, feature, and alert data.

15. The method of claim 11 wherein the 3-D tracking further comprises:

creating 3-D tracks from new observations and existing 3-D tracks;

evaluating 3-D tracks for likelihood based on consistent 3-D kinematic and feature data; and

pruning the least likely 3-D tracks.

16. The method of claim 15 wherein the 3-D tracking further comprises:

clustering 3-D tracks that share observations and

identifying incompatible 3-D tracks.

17. The method of claim 16 wherein the 3-D tracking further comprises multiple hypothesis testing, the multiple hypothesis testing comprising:

creating hypotheses from combinations of compatible tracks;

evaluating hypotheses for likelihood based on likelihood of underlying tracks; and

pruning unlikely hypotheses.

18. The method of claim 17 wherein the 3-D tracking further comprises:

reevaluating 3-D tracks for likelihood based on likelihood of corresponding hypotheses and

pruning unlikely 3-D tracks.

19. The method of claim 18 wherein the 3-D tracking further comprises 3-D track filtering, the 3-D track filtering comprising:

identifying similar 3-D tracks likely to represent a same object and

merging the similar 3-D tracks into a single 3-D track that represents the same object.

20. The method of claim 11 further comprising processing the data in real-time.