REAL-TIME MULTISTAGE INFRARED IMAGE-BASED TRACKING SYSTEM

• US 20060291693A1
• Filed: 05/23/2003
• Published: 12/28/2006
• Est. Priority Date: 02/23/1999
• Status: Active Grant

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Slaski, James Joseph, Olson, Teresa Lorae Pace
• Granted Patent: US 7,177,447 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors

Abstract

A multi-stage infrared (IR) image-based tracking system is provided that accurately identifies and tracks targets across a variety of different situations and environmental conditions. The system includes primary and secondary trackers. The secondary tracker supports the primary tracker by identifying and tracking targets when the primary tracker loses or fails to identify one or more targets, and helps the primary tracker reacquire valid targets. In an exemplary embodiment of the invention, the primary tracker is a correlation-based tracker, and the secondary tracker is a feature-based tracker. A prescreener is also included, which operates concurrently with the correlation-based and feature-based trackers to generate a list of possible targets. The list can be used to provide a confidence level for the probable targets identified by each of the primary and secondary trackers.

Claims

1. A multi-stage image-based tracking system, comprising:

a first tracker for identifying at least one target within at least one image provided to the system; and

a second tracker for identifying the at least one target within the at least one image, wherein the second tracker is a feature-based tracker, wherein the second tracker is initialized during tracking by the first tracker;

wherein when the first tracker fails to identify the at least one target, the second tracker provides identification of the at least one target to the first.

2. The system of claim 1, further comprising a prescreener for processing the at least one image to provide a list of possible targets, wherein the first and second trackers identify the at least one target based on the at least one image and the list.

3. The system of claim 2, wherein the list is used to corroborate target identification of the first and second trackers.

4. The system of claim 2, wherein the list is used to provide a quality measure of a target identification of at least one of the first and second trackers.

5. The system of claim 1, wherein the prescreener generates the list by integrating edge information in the at least one image.

6. The system of claim 1, wherein the image is an infrared image.

7. The system of claim 1, wherein the first tracker is a correlation-based tracker.

8. The system of claim 2, further comprising:

an automatic target recognition module that receives the list of possible targets from the prescreener, identifies a target from the list of possible targets, and provides the identified target to the first and second trackers when the first and second trackers both lose tracking of an initial target.

9. The system of claim 8, wherein the identified target is the same as the initial target.

10. The system of claim 8, wherein a device guided by the tracking system continues based upon an identification of the at least one target by the first tracker or the second tracker when both the first and second trackers lose tracking of an initial target.

11. The system of claim 1, wherein the at least one image is provided by an image pickup on a missile.

12. The system of claim 1, wherein whether the first tracker fails to identify the at least one target is based on a confidence value associated with the at least one target identified by the first tracker.

13. The system of claim 12, wherein the confidence value is determined based upon an identification of the at least one target by the first tracker in a predetermined number of images.

14. The system of claim 1, wherein when the second tracker fails to identify the at least one target, the first tracker provides identification of the at least one target to the second tracker.

35. The multi-stage image-based tracking system of claim 1, wherein the first and second trackers are reinitialized when a confidence value associated with the tracking of the second tracker falls below a second predetermined minimum value.

37. The multi-stage image-based tracking system of claim 1, wherein the first tracker and the second tracker track a target in parallel.

15. A method for identifying targets within at least one image using a primary tracker and a secondary tracker, comprising the steps of:

tracking at least one target in the at least one image using the primary tracker;

evaluating a tracking performance of the primary tracker;

initializing the secondary tracker during tracking by the primary tracker;

switching tracking control from the primary tracker to the secondary tracker when the tracking performance of the primary tracker falls below a first predetermined minimum value;

acquiring and tracking the at least one target using the secondary tracker, the secondary tracker is a feature-based tracker, which calculates a predetermined number of features of the at least one target within the at least one image;

switching tracking control from the secondary tracker to the primary tracker after acquiring the at least one target using the secondary tracker.

16. The method of claim 15, further comprising the steps of:

generating a list indicating possible targets in the at least one image using a prescreener; and

providing the list to the primary and secondary trackers to enhance the tracking performance of the primary and secondary trackers.

17. The method of claim 16, wherein the step of generating the list comprises the steps of:

calculating strong edge information in the at least one image using an edge operator;

integrating the edge information over the at least one image using a plurality of boxes having different sizes to determine possible targets; and

applying a morphological erode operator to the integrated at least one image to calculate centroids of possible targets within the at least one image, wherein the centroids indicate the possible targets.

18. The method of claim 17, wherein the edge operator is a Sobel edge operator.

19. The method of claim 16, wherein the step of generating comprises:

calculating strong edge information in the at least one image using an edge operator;

evaluating at least an edge magnitude portion of the strong edge information using atmospheric attenuation weighting to compensate for atmospheric effects;

thresholding the compensated strong edge information;

integrating the thresholded information over the at least one image using variable box sizes;

thresholding the integrated information; and

calculating centroids that correspond to potential target objects in the at least one image, based on the thresholded integrated information.

20. The method of claim 19, further comprising:

determining whether each potential target object is too large or too small to be a valid potential target;

based on the step of determining, discarding potential target objects that are too large or too small to be valid potential targets.

21. The method of claim 19, wherein the step of calculating centroids is performed by applying a morphological erode operator to subimages in the thresholded integrated information.

22. The method of claim 21, further comprising:

determining a size of a potential target object represented by a subimage in the thresholded integrated information, by counting a number erodes that the morphological erode operator must perform to reduce the subimage to a single pixel.

23. The method of claim 19, wherein:

the step of thresholding the integrated information generates a processed version of the at least one image; and

the step of calculating centroids comprises calculating centroids of subimages in the processed version of the at least one image, that correspond to potential target objects in the at least one image.

24. The method of claim 15, wherein the at least one image is provided by an image pickup on a missile.

25. The method of claim 15, wherein tracking control is switched from the primary tracker to the secondary tracker when a distance between the primary and secondary trackers and the at least one target is greater than a predetermined distance.

26. The method of claim 15, wherein prior to switching tracking control from the primary tracker to the secondary tracker the secondary tracker is initialized.

27. The method of claim 26, wherein the initialization of the secondary tracker comprises:

determining whether the tracking performance of the primary tracker is above the predetermined minimum;

receiving a list indicating possible targets from a prescreener;

computing features in a number of images;

calculating statistics of the computed features; and

determining whether the secondary tracker identifies valid targets in a predetermined number of the number of images.

36. The method of claim 15, comprising:

reinitializing the primary and secondary trackers using an output from an automatic target recognition module when a value associated with the tracking performance of the secondary tracker falls below a second predetermined minimum value.

38. The method of claim 15, wherein the first tracker and the second tracker track a target in parallel.

28. A method for identifying targets within at least one image comprising the steps of:

tracking at least one target in the at least one image using a primary tracker;

tracking the at least one target using a secondary tracker;

evaluating a tracking performance of the primary tracker;

switching tracking control from the primary tracker to the secondary tracker when a confidence value associated with the tracking of the primary tracker falls below a first predetermined minimum value; and

reinitializing the primary and secondary trackers using an output from an automatic target recognition module when a confidence value associated with the tracking of the secondary tracker falls below a second predetermined minimum value and the confidence value associated with the primary tracker remains below the first predetermined minimum value.

29. The method of claim 28, wherein the at least one image is provided by an image pickup on a missile.

30. The method of claim 28, wherein the primary tracker is a correlation-based tracker and the secondary tracker is a feature-based tracker.

31. The method of claim 28, wherein reinitialization of the primary and secondary trackers comprises:

receiving, by the automatic target recognition module, a list of possible targets from a prescreener;

identifying, by the automatic target recognition module, a target from the list of possible targets; and

providing the identified target to the primary and secondary trackers when the first and second trackers both lose tracking of an initial target.

32. A method for identifying and tracking at least one target within at least one image using a primary tracker and a secondary tracker and an automatic target recognition module, comprising the steps of:

tracking at least one target in the at least one image using the primary tracker;

tracking the at least one target using the secondary tracker;

evaluating a tracking performance of the primary tracker;

evaluating a tracking performance of the secondary tracker;

switching tracking control from the primary tracker to the secondary tracker when the tracking performance of the primary tracker falls below a first predetermined minimum and the tracking performance of the secondary tracker remains above a second predetermined minimum;

switching tracking control to the primary tracker when the tracking performance of the primary tracker rises above the predetermined minimum;

switching tracking control to the automatic target recognition module when the tracking performance of the primary tracker is below the first predetermined minimum and the tracking performance of the secondary tracker is below the second predetermined minimum;

selecting a target in the at least one image, wherein the automatic target recognition module performs the selection; and

identifying the selected target to the primary tracker and to the secondary tracker.

33. The method of claim 32, wherein a prescreener is also used, further comprising the steps of:

providing a list of potential target objects in the at least one image, to each of the primary tracker, the secondary tracker, and the automatic target recognition module;

wherein the target selected in the step of selecting is selected from the list of potential targets.

34. The method of claim 32, wherein the tracking performance of the primary and secondary trackers is evaluated based on tracking in a number of images.