Moving object tracking

• US 5,878,151 A
• Filed: 10/31/1996
• Issued: 03/02/1999
• Est. Priority Date: 10/31/1996
• Status: Expired due to Fees

• Current Assignee: XinAoMDT Technology Co., Ltd.
• Original Assignee: Combustion Engineering Incorporated (ABB Limited)
• Inventors: Tang, Qing, Katz, Jeffrey Sherman, Fitzgibbons, Lance Terrance, Gan, Zhongxue
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US08/740,702
• Time in Patent Office: 852 Days
• Field of Search: 382/210, 382/293, 382/103, 382/291, 382/278, 382/260, 395/94, 395/95, 395/96, 348/94, 348/95, 348/129, 348/125, 348/169, 348/82, 348/83
• US Class Current: 382/103
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 2201/06 Recognition of objects for industrial automation

Abstract

An image of the object remains in the field of view of the camera, thereby generating a continuously changing image signal. The images are captured and digitized in a continual sequence S of discrete images of the scenes at a respective sequence T of points in time. During this capture sequence, a first plurality of reference signals are established corresponding to a plurality of respective captured images. In this manner, a plurality of reference images are generated on-line, during movement of the object or cameras. A second plurality of detected signals corresponding to a second plurality of captured images, are individually correlated with the respective next previous reference signal of the sequence, thereby establishing a relative position of the object or camera, for each detected signal. The absolute position of the camera or object, is determined by accumulating these relative positions.

Claims

1. A method for operating a computer vision system with associated moving camera to track the position of the camera relative to a plurality of spaced apart stationary objects of known size, comprising:

focusing the camera having a field of view which defines a scene, so that at least one of said objects remains in the field of view as the camera moves in time, thereby generating a continuously changing image signal;

delivering the continuous image signal to said computer vision system;

in said computer vision system, capturing and digitizing a continual sequence S of discrete images of the scene with object, at a respective sequence T of points in time;

storing a first set S1 of discrete images S1,i (where i=1,2, . . . N) from said sequence S corresponding to a first set T1 of points from time sequence T;

creating a digital filter H1 from discrete image S1,1;

comparing each of the other discrete images S1,2;

S1,3;

. . . S1,N of set S1 with said filter H1;

from said comparison, determining the change in position of the object in the scene, relative to the position of the object at the point in time corresponding to the capture of image S1,1;

recording the position of the object as determined from the first set S1 of discrete images;

storing a second set S2 of discrete images S2,i (where i=1,2, . . . N) from said sequence S corresponding to a second set T2 of time points following the first set T1 of time points;

creating a digital filter H2 from discrete image S2,1;

comparing each of the other discrete images S2,2;

S2,3;

. . . S2,N of set S2 with said filter H2

from said comparison, determining the change in position of the object in the scene, relative to the position of the object at the point in time corresponding to the capture of image S2,1;

recording the position of the object as determined from the second set S2 of discrete images; and

repeating the foregoing for at least one additional set S3 of said sequence S and set T3 of said sequence T.

2. The method of claim 1, wherein

the step of comparing each of the other discrete images of a particular set with the digital filter for that particular set includes computing a correlation value for each comparison,

the step of determining the change in position includes selecting the peak correlation value for each comparison, and

the step of recording the position includes recording the position determined from each said peak correlation value, as the change in position of the object during each time interval corresponding to the capture of said particular set of images.

3. The method of claim 2, wherein the last digitized image Sk,N of set K is also the first digitized image Sk+1,1 of set K+1.

4. The method of claim 3, wherein the digital filters H1 and H2 are the phase conjugations of the Fast Fourier transforms for images S1,1 and S2,1 respectively.

5. The method of claim 1, wherein the digital filters H1 and H2 are the phase conjugations of the Fast Fourier transforms for images S1,1 and S2,1 respectively.

6. A method for operating a computer vision system with associated camera to track the position of a camera moving in parallel relative to a tube sheet having a multiplicity of tube openings of uniform shape, each opening centered on a coordinate of a two-dimensional grid having a regular matrix and spaced a distance from each other, comprising:

focusing the camera having a field of view which defines a scene, so that images of tube openings remain in the field of view as the camera moves in time, thereby generating a continuously changing image signal;

delivering the continuous image signal to said computer vision system;

in said computer vision system, capturing and digitizing a continual sequence S of discrete images of the scene at a respective sequence T of points in time;

establishing a first plurality of reference signals corresponding to a plurality of respective images at a respective plurality of points in time in the sequences S and T;

establishing a second plurality of detected signals corresponding to a second plurality of respective images at a corresponding second plurality of points in time in the sequences S and T; and

correlating each of the detected signals with the next previous reference signal of the sequence S, thereby establishing a relative position of the camera for each detected signal; and

determining the absolute position of the camera by accumulating the relative positions.

7. The method of claim 6, including the step of estimating the relative position of the camera for future points in time, from the time dependent rate of change of the camera position during the time sequence T.

8. The method of claim 6, including

storing a first set S1 of discrete images S1,i (where i=1,2, . . . N) from said sequence S corresponding to a first set T1 of points from time sequence T;

creating a digital filter H1 from discrete image S1,1;

comparing each of the other discrete images S1,2;

S1,3;

. . . S1,N of set S1 with said filter H1;

from said comparison, determining the change in position of the tube opening in the scene, relative to the position of the opening at the point in time corresponding to the capture of image S1,1;

recording the position of the object as determined from the first set S1 of discrete images;

storing a second set S2 of discrete images S2,i (where i=1,2, . . . N) from said sequence S corresponding to a second set T2 of time points following the first set T1 of time points;

creating a digital filter H2 from discrete image S2,1;

comparing each of the other discrete images S2,2;

S2,3;

. . . S2,N of set S2 with said filter H2

from said comparison, determining the change in position of the object in the scene, relative to the position of the object at the point in time corresponding to the capture of image S2,1;

recording the position of the object as determined from the second set S2 of discrete images; and

repeating the foregoing for at least one additional set S3 of said sequence S and set T3 of said sequence T.

9. The method of claim 8, wherein

the step of comparing each of the other discrete images of a particular set with the digital filter for that particular set includes computing a correlation value for each comparison,

the step of determining the change in position includes selecting the peak correlation value for each comparison, and

the step of recording the position includes recording the position determined from each said peak correlation value, as the change in position of the object during each time interval corresponding to the capture of said particular set of images.

10. The method of claim 8, wherein the digital filters H1 and H2 are the phase conjugations of the Fast Fourier transforms for images S1,1 and S2,1 respectively.

11. A method for operating a computer vision system for tracking the position of an object of known size moving in a scene, comprising:

focusing a camera having a field of view which defines a scene, so that said object remains in the field of view as the object moves in time through the scene, thereby generating a continuously changing image signal;

delivering the continuous image signal to said computer vision system;

in said computer vision system, capturing and digitizing a continual sequence S of discrete images of the scene with object, at a respective sequence T of points in time;

storing a first set S1 of discrete images S1,i (where i=1,2, . . . N) from said sequence S corresponding to a first set T1 of points from time sequence T;

creating a digital filter H1 from discrete image S1,1;

comparing each of the other discrete images S1,2;

S1,3;

. . . S1,N of set S1 with said filter H1;

from said comparison, determining the change in position of the object in the scene, relative to the position of the object at the point in time corresponding to the capture of image S1,1;

recording the position of the object as determined from the first set S1 of discrete images;

storing a second set S2 of discrete images S2,i (where i=1,2, . . . N) from said sequence S corresponding to a second set T2 of time points following the first set T1 of time points;

creating a digital filter H2 from discrete image S2,1;

comparing each of the other discrete images S2,2;

S2,3;

. . . S2,N of set S2 with said filter H2

from said comparison, determining the change in position of the object in the scene, relative to the position of the object at the point in time corresponding to the capture of image S2,1;

recording the position of the object as determined from the second set S2 of discrete images; and

repeating the foregoing for at least one additional set S3 of said sequence S and set T3 of said sequence T.

12. The method of claim 11, wherein

the step of comparing each of the other discrete images of a particular set with the digital filter for that particular set includes computing a correlation value for each comparison,

the step of determining the change in position includes selecting the peak correlation value for each comparison, and

the step of recording the position includes recording the position determined from each said peak correlation value, as the change in position of the object during each time interval corresponding to the capture of said particular set of images.

13. The method of claim 12, wherein the last digitized image Sk,N of set K is also the first digitized image Sk+1,1 of set K+1.

14. The method of claim 11, wherein the digital filters H1 and H2 are the phase conjugations of the Fast Fourier transforms for images S1,1 and S2,1 respectively.