Correlation based in frame video tracker

• US 7,136,506 B2
• Filed: 03/03/2003
• Issued: 11/14/2006
• Est. Priority Date: 03/03/2003
• Status: Expired due to Fees

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Howard, Robert J.
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): MISTRY, O NEAL RAJAN
• Application Number:
  US10/376,283
• Publication Number:
  US 20040175019A1
• Time in Patent Office: 1,352 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

A video correlation tracking system captures a first image frame and identifies one or more reference patches in that image frame. In a successive image frame, the new location of a reference patch is estimated with a dead reckoning algorithm, and a trial patch is placed at that location. The system cross correlates each reference patch with its corresponding trial patch, the results of which are used to make corrections to the dead reckoning algorithm, register successive image frames, and determine at which reference patch processing resources should be concentrated.

Claims

1. A method to track objects in a video processing system from image frame to image frame, comprising the steps of:

capturing a first image;

identifying an object of interest in said first image with a reference patch;

capturing a second image;

estimating a position of said object of interest in said second image through the use of a dead reckoning algorithm, and marking said position with a trial patch;

cross correlating said trial patch and said reference patch;

using said cross correlation to make corrections to said dead reckoning algorithm;

further using said cross correlation to calculate tie points;

using said tie points to establish a common coordinate system between said first and said second image; and

identifying reference patches that have been determined by a classification algorithm, and releasing said reference patches to free up processing resources.

2. The method to track objects in a video processing system according to claim 1, wherein said reference patch is identified through a contrast anomaly.

3. The method to track objects in a video processing system according to claim 1, wherein said corrections to said dead reckoning algorithm are based on the displacement of a correlation peak of said cross correlation, and further wherein said corrections are fed into said dead reckoning algorithm to correct future predictions of trial patch locations.

4. The method to track objects in a video processing system according to claim 1, wherein said tie points are used to register said images.

5. The method to track objects in a video processing system according to claim 1, wherein said reference patch is identified so as to have strong texture.

6. The method to track objects in a video processing system according to claim 1, further comprising the step of identifying a second object of interest with a released reference patch.

7. The method to track objects in a video processing system according to claim 1, wherein said dead reckoning algorithm is corrected by adjusting coefficients within said dead reckoning algorithm.

8. The method to track objects in a video processing system according to claim 1, wherein said tie points are used as coefficients in a bilinear interpolation, said bilinear interpolation resulting in the registration of images in a common coordinate system.

9. The method to track objects in a video recording system according to claim 1, wherein said reference patch identifies an object of interest that is in motion, and further wherein said reference patch that is associated with said object of interest in motion is rejected.

10. An apparatus to track objects in a video processing system from image frame to image frame, comprising:

means for capturing a first image;

means for identifying an object of interest in said first image with a reference patch;

means for capturing a second image;

means for estimating a position of said object of interest in said second image through the use of a dead reckoning algorithm, and marking said position with a trial patch;

means for cross correlating said trial patch and said reference patch;

means for using said cross correlation to make corrections to said dead reckoning algorithm;

further means for using said cross correlation to calculate tie points;

using said tie points to establish a common coordinate system between said first and said second image; and

means for identifying reference patches that have been determined by a classification algorithm, and releasing said reference patches to free up processing resources.

11. The apparatus to track objects in a video processing system according to claim 10, wherein said reference patch is identified through a contrast anomaly.

12. The apparatus to track objects in a video processing system according to claim 10, wherein said corrections to said dead reckoning algorithm are based on the displacement of a correlation peak of said cross correlation, and further wherein said corrections are fed into said dead reckoning algorithm to correct future predictions of trial patch locations.

13. The apparatus to track objects in a video processing system according to claim 10, wherein said tie points are used to register said images.

14. The apparatus to track objects in a video processing system according to claim 10, wherein said reference patch is identified so as to have strong texture.

15. The apparatus to track objects in a video processing system according to claim 10, further comprising the step of identifying a second object of interest with a released reference patch.

16. The apparatus to track objects in a video processing system according to claim 10, wherein said dead reckoning algorithm is corrected by adjusting coefficients within said dead reckoning algorithm.

17. The apparatus to track objects in a video processing system according to claim 10, wherein said tie points are used as coefficients in a bilinear interpolation, said bilinear interpolation resulting in the registration of images in a common coordinate system.

18. The apparatus to track objects in a video recording system according to claim 10, wherein said reference patch identifies an object of interest that is in motion, and further wherein said reference patch that is associated with said object of interest in motion is rejected.

19. A system to track objects during video processing from image frame to image frame, comprising:

a camera for capturing a first image;

a processor for identifying an object of interest in said first image with a reference patch;

the camera capturing a second image;

processor means for estimating a position of said object of interest in said second image through the use of a dead reckoning algorithm, and marking said position with a trial patch;

processor means for cross correlating said trial patch and said reference patch;

using said cross correlation to make corrections to said dead reckoning algorithm;

further using said cross correlation to calculate tie points;

using said tie points to establish a common coordinate system between said first and said second image; and

means for identifying reference patches that have been determined by a classification algorithm, and releasing said reference patches to free up processing resources.

20. The system to track objects during video processing from image frame to image frame according to claim 19, wherein said reference patch is identified through a contrast anomaly.