Adaptive updating processor for use in an area correlation video tracker

• US 4,227,212 A
• Filed: 09/21/1978
• Issued: 10/07/1980
• Est. Priority Date: 09/21/1978
• Status: Expired due to Term

• Current Assignee: Micron Technology, Inc.
• Original Assignee: Westinghouse Electric Corporation (Paramount Global (f/k/a ViacomCBS Inc.))
• Inventors: Woolfson, Martin G., Bentley, Floyd C.
• Primary Examiner(s): Griffin, Robert L.
• Assistant Examiner(s): Orsino, Jr., Joseph A.
• Application Number:
  US05/944,523
• Time in Patent Office: 747 Days
• Field of Search: 358/125, 358/126, 250/203 CT
• US Class Current: 348/170
• CPC Class Codes: G01S 3/7865 using correlation of the live video image with a stored image

Abstract

In an area correlation video tracker, a reference subarray of picture elements acquired from a selected track window array of picture elements of an acquisition video frame is cross-correlated in accordance with a predetermined metric function with the picture elements of the selected track window array of each of a number of video frames generated subsequent the acquisition video frame. From these cross-correlations, a minimum cross-correlation value is derived and a subarray of picture elements associated therewith is identified in the track window array of the subsequently generated video frames. An adaptive threshold processor performs autocorrelations in accordance with the predetermined metric function with the video information of the acquisition frame to derive a reference threshold correlation value and with the video information of each of the number of subsequently generated frames to correspondingly derive another threshold correlation value each. An update condition is rendered in the video frame having a derived one cross-correlation value greater than the reference threshold correlation value. The update condition causes the values of the picture elements of the reference subarray to be replaced with the values of the picture elements of the identified subarray corresponding to the one cross-correlation value and causes the reference threshold correlation value to take on the value of the another threshold correlation value derived in the video frame associated with the one cross-correlation value, whereby subsequent cross-correlations will be performed with the updated values until another update condition is rendered.

Claims

1. In an area correlation video tracker including an electro-optical sensor which generates frames of video information representative of an intensity pattern of the field of view of said electro-optical sensor corresponding to discrete intervals in time;

means for partitioning a preselected portion of said generated video frames into an array of picture elements, the value of each picture element being representative of the gray level intensity corresponding to its index in said array, said array being additionally partitioned into a predetermined number of indexed subarray zones;

means for acquiring the gray level values of the picture element of a selected first subarray zone corresponding to an acquisition video frame; and

correlation means for cross-correlating, based on a predetermined metric correlation function, the gray level values of the picture elements of said selected first subarray zone with the picture element gray level values of said partitioned subarray zones of each of a number of video frames generated subsequent said acquisition video frame to identify a second subarray zone of picture elements in each of said number of subsequently generated video frames and to derive a cross-correlation metric value associated with each of said identified second subarray zones, the improvement of an adaptive updating means comprising;

means operative to perform an autocorrelation of the gray level values of the picture elements of said selected first subarray zone with the gray level value of the picture elements of an at least one predetermined third subarray zone of said acquisition video frame in accordance with said predetermined metric correlation function to derive a reference threshold correlation metric value and to further perform an autocorrelation of the gray level values of the picture elements of said identified second subarray zone with the gray level values of the picture elements of at least one predetermined fourth subarray zone of the same video frame in accordance with said predetermined metric correlation function to derive another threshold correlation metric value, said another threshold correlation metric value being derived for each of said number of subsequently generated video frames;

means operative to render an update condition based on the results of a comparison between one derived cross-correlation metric value and said reference threshold correlation metric value, said update condition causes the gray level values of the picture elements of said selected first subarray to be replaced with the gray level values of the picture elements of said identified second subarray corresponding to said one cross-correlation metric value and causes said reference threshold correlation metric value to take on the value of said another threshold correlation metric value derived from the video frame corresponding to said one cross-correlation metric value, whereby subsequent cross-correlation processing will be performed with the updated gray level values of the picture elements of the selected first subarray and updated threshold correlation metric value.

2. The adaptive updating means in accordance with claim 1 wherein the at least one predetermined third subarray zone of picture elements is located in the array in a predetermined indexed relation to the selected first subarray zone;

and wherein the at least one predetermined fourth subarray zone of picture elements is located in the array in a similar predetermined indexed relationship to the identified second subarray zone for each of the number of subsequently generated video frames.

3. The adaptive updating means in accordance with claim 1 wherein an autocorrelation metric value is derived from each of the autocorrelations performed between the gray level values of the picture elements of the selected first subarray zone and the gray level values of the picture elements of each of the at least one predetermined third subarray zone(s);

and wherein the maximum of said derived autocorrelation metric value(s) is selected as the reference threshold correlation metric value.

4. The adaptive updating means in accordance with claim 1 wherein an autocorrelation metric value is derived from each of the autocorrelations performed for each of the number of subsequently generated video frames between the gray level values of the picture elements of the identified second subarray zone and the gray level values of the picture elements of each of the at least one predetermined fourth subarray zone(s);

and wherein the maximum of said autocorrelation metric value(s) derived for each of the number of subsequently generated video frames is selected as the another threshold correlation metric value for each of the number of subsequently generated video frames.

5. The adaptive updating means in accordance with claim 1 wherein the update condition is rendered for the one of the number of subsequently generated video frames which has a derived cross-correlation metric value associated with the identified second subarray thereof that is substantially greater than the reference threshold correlation metric value.

6. The adaptive updating means in accordance with claim 1 wherein the predetermined metric correlation function is one of image matching the picture elements of two subarrays being represented by two vectors r and p, each having a dimension of n;

and wherein a correlation metric valued may be derived in accordance with the following expression;

##EQU2## where r_i is the gray level value of the ith picture element of the subarray represented by the vector r and p_i is the gray level value of the corresponding ith picture element of the other subarray represented by the vector p.

7. The adaptive updating means in accordance with claim 6 wherein an image matching cross-correlation is performed between the gray level values of the picture elements r_i of the selected first subarray zone and the gray level values of the corresponding picture elements p_i (Z_k) of each k of the partitioned subarray zones Z_k of a video frame generated subsequent the acquisition of video frame to derive a cross-correlation value d_k for each zone Z_k thereof in accordance with the following expression:

##EQU3## wherein the identified second subarray zone is the subarray zone which corresponds to the minimum d_r of the cross-correlation values d_k which may be mathematically expressed as;

##EQU4## wherein d_n is the cross-correlation metric value which is derived for each one of the number of subsequently generated video frames and compared with the reference threshold correlation metric value to establish the subsequent video frame during which an update is performed.

8. The adaptive updating means in accordance with claim 6 wherein an image matching autocorrelation is performed between the gray level values of the picture elements r_i of the selected first subarray zone and the gray level values of the corresponding picture elements p_i (z_k) of each k of the at least one predetermined third subarray zone(s) z_k of the acquisition video frame to derive an autocorrelation value d_k for each predetermined third subarray zone z_k in accordance with the following expression:

##EQU5## wherein the maximum d_n of the autocorrelation values d_k, which may be expressed mathematically as;

##STR3## is the reference threshold correlation metric value for the acquisition video frame.

9. The adaptive updating means in accordance with claim 6 wherein an image matching autocorrelation is performed between the gray level values of the picture elements r_i of the identified second subarray zone and the gray level values of the corresponding picture elements p_i (z_k) of each k of the at least one predetermined fourth subarray zone to derive an autocorrelation value d_k for each predetermined fourth subarray zone z_k in accordance with the following expression:

##EQU6## wherein the maximum d_n of the autocorrelation values d_k which may be expressed mathematically as;

##EQU7## is the another threshold correlation metric value derived for each of the number of subsequently generated video frames.

10. In an area correlation video tracker including an electro-optical sensor which generates frames of video information representative of an intensity pattern of the field of view of said electro-optical sensor corresponding to discrete intervals in time;

means for partitioning a preselected portion of said generated video frames into an array of picture elements, the value of each picture element being representative of the gray level intensity corresponding to its index in said array, said array additionally being partitioned into a predetermined number of indexed subarray zones;

means for acquiring the gray level values of the picture elements of a selected first subarray zone corresponding to an acquisition video frame; and

correlation means for cross-correlating, based on a predetermined metric correlation function, the gray level values of the picture elements of said selected first subarray zone with the picture element gray level values of said additional subarray zones of each of a number of video frames generated subsequent said acquisition video frame to identify a second subarray zone of picture elements in each of said number of subsequently generated video frames and to derive a cross-correlation metric value associated with each of said identified second subarray zones, the improvement of an adaptive updating means comprising;

means operative to perform autocorrelations in accordance with a predetermined metric function between

(a) the gray level values of the picture elements of said selected first subarray zone and the gray levels of corresponding picture elements of said predetermined subarray zones in said preselected array of said acquisition frame to derive an autocorrelation metric value for each autocorrelation, and between

(b) the gray level values of the picture elements of said identified second subarray zone and the gray level values of corresponding picture elements of said predetermined subarray zones in said preselected array of a video frame generated subsequent said acquisition frame to derive an autocorrelation metric value for each autocorrelation, whereby a group of autocorrelation metric values is derived for each of said number of subsequently generated frames corresponding to the subarray video information generated in each of said number of frames;

means for segregating from the autocorrelation metric values derived from said acquisition video frame an initial set of autocorrelation metric values based on a first predetermined relationship between the indexes of the autocorrelated subarray zones and the index of said selected first subarray zone in the preselected array of said acquisition frame, and for selecting one autocorrelation metric value from said segregated initial set to be a reference threshold correlation metric value; and

for segregating from each of said groups of autocorrelation metric values correspondingly derived from the video information of each of said number of subsequently generated frames another set of autocorrelation metric values based on a second predetermined relationship between the index of said identified second subarray zone and the indexes of the autocorrelated subarray zones corresponding to the video frame from which said second subarray was identified, and for selecting one autocorrelation metric value from each of said another sets to each be an another threshold correlation metric value; and

means for generating an update signal corresponding to the one of said number of subsequently generated video frames which has a derived cross-correlation metric value substantially greater than said reference threshold correlation metric value, said update signal causing the gray level values of the picture elements of said selected first subarray zone to be replaced with the gray level values of the picture elements of said second subarray zone identified from said one subsequently operated video frame and causes said reference threshold correlation value to take on the value of said selected another threshold correlation metric value corresponding to said one subsequently generated video frame, whereby subsequent cross-correlation processing will be performed with the updated gray level values of said first subarray zone and the updated threshold correlation metric value.

11. An adaptive updating means in accordance with claim 10 wherein the index of the selected first subarray zone in the preselected array of picture elements of the acquisition frame is denoted by the cartesian coordinates X_N, Y_M, where N and M are integer number of picture elements along an X-axis and Y-axis, respectively, both measured with respect to a prespecified origin of the generated video frames;

and wherein the first predetermined relationship of the subarray zone indexes corresponding to the segregated initial set of autocorrelation metric values is expressed in cartesian coordinates with respect to the index X_N, Y_M of the selected first subarray zone as follows;

X_N, X_M-1, X_N+1, Y_M ;

X_N-1, Y_M ; and

X_N, Y_M+1.

12. An adaptive updating means in accordance with claim 11 wherein the identified second subarray zone for each of the number of subsequently generated frames is similarly indexed in cartesian coordinates X_N, Y_M as that of the selected first subarray zone;

and wherein the second predetermined relationship of the subarray zone indexes corresponding to the segregated another set of autocorrelation metric values for each of the number of subsequently generated frames is expressed in cartesian coordinates with respct to the index of the identified second subarray zone as follows;

X_N, Y_M-1 ;

X_N+1, Y_M ;

X_N-1, Y_M ; and

X_N, Y_M+1.

13. The adaptive updating means in accordance with claim 10 wherein the predetermined metric correlation function is one of image matching the picture elements of two subarrays being represented by two vectors r and p, each having a dimension of n;

and wherein a correlation metric valued may be derived in accordance with the following expression;

##EQU8## where r_i is the gray level value of the ith picture element of the subarray represented by the vector r and p_i is the gray level value of the corresponding ith picture element of the other subarray represented by the vector p.

14. The adaptive updating means in accordance with claim 13 wherein an image matching cross-correlation is performed between the gray level values of the picture elements r_i of the selected first subarray zone and the gray level values of the corresponding picture elements p_i (Z_k) of each one k of the partitioned subarray zones Z_k of a video frame generated subsequent the acquisition video frame to derive a cross-correlation value d_k for each zone Z_k thereof in accordance with the following expression:

##EQU9## wherein the identified second subarray zone is the subarray zone which corresponds to the minimum d_n of the cross-correlation values d_k which may be mathematically expressed as;

##EQU10## wherein d_n is the cross-correlation metric value which is derived for each of the number of subsequently generated video frames and compared with the reference threshold correlation metric value to establish the subsequent video frame during which an update is performed.

15. The adaptive updating means in accordance with claim 13 wherein an image matching autocorrelation is performed between the gray level values of the picture element r_i of the selected first subarray zone and the gray level values of the corresponding picture element p_i (Z_k) of each one k of the partitioned subarray zones Z_k of the acquisition video frame to derive an autocorrelation metric value d_k for each corresponding subarray zone Z_k in accordance with the following expression:

##EQU11## wherein the maximum d_k of the segregated initial set of autocorrelation values is the one selected as the reference threshold correlation metric value for the acquisition video frame.

16. The adaptive updating means in accordance with claim 13 wherein an image matching autocorrelation is performed between the gray level values of the picture elements r_i of the identified second subarray zone and the gray level values of the corresponding picture elements p_i (Z_k) of each one k of the partitioned subarray zones Z(k) for each of the number of subsequently generated video frames to derive an autocorrelation metric value d_k for each corresponding subarray zone Z_k in accordance with the following expression:

##EQU12## wherein the maximum d_k from each of the segregated another set of autocorrelation metric values is the one selected as the another threshold correlation metric value associated with each of the number of subsequently generated video frames.