Method and apparatus for tracking moving objects in real time using contours of the objects and feature paths

• US 5,764,283 A
• Filed: 12/29/1995
• Issued: 06/09/1998
• Est. Priority Date: 12/29/1995
• Status: Expired due to Term

• Current Assignee: AT&T Corporation (AT&T, Inc.)
• Original Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Inventors: Pingali, Sarma VGK, Segen, Jakub
• Primary Examiner(s): Lee, Richard
• Assistant Examiner(s): VINCENT, DAVID ROBERT
• Application Number:
  US08/586,012
• Time in Patent Office: 893 Days
• Field of Search: 348/699, 348/416, 348/169, 348/399, 348/411, 348/26, 382/190, 382/242, 382/243
• US Class Current: 348/169
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments

Abstract

A method and apparatus for tracking moving objects, such as people, in real time by grouping existing feature paths from a current video frame with preexisting clusters from current video frames. Grouping is accomplished by selecting, as a candidate cluster to a specified feature path, the cluster having the closest relative distance to the specified feature path. The relative distance is calculated according to a function having a first component and a second component. The first component is representative of a maximum displacement between the select feature path and the candidate cluster, and the second component is representative of the difference between velocity vectors defining the select feature path motion and velocity vectors defining the candidate cluster motion. The method and apparatus also provide for updating the candidate cluster with parameters obtained from the select feature path and for merging overlapping clusters.

Claims

1. A method for tracking, in real time, a moving object represented by a video signal comprised of prior frames and a current frame and having an object image with features associated with the object, said method comprising the steps of:

matching a select feature path defined by x and y component values representing motion of the features of an object with a candidate cluster defined by x and y component values chosen from a plurality of preexisting clusters representing motion of the object, said select feature path having a portion concurrent in time with some of the clusters in said plurality of clusters, said candidate cluster selected as the cluster closest in distance to the select feature path, the distance being calculated according to a function having a first component and a second component, the first component being representative of a maximum displacement between the select feature path and the candidate cluster, and the second component being representative of differences between velocity vectors defining motion of the select feature path and velocity vectors defining motion of the candidate cluster wherein the select feature path is comprised of path points, wherein the candidate cluster is comprised of cluster points, and wherein the second component of the distance function is computed according to the following formula;

##EQU11## wherein P represents the select path, Z represents the candidate cluster, 1 represents the overlap length between P and Z, and wherein represents a first instant of overlap between the select path P and the candidate cluster Z.

2. The method of claim 1, wherein the distance is calculated according to the formula:

##EQU12## wherein p_x, p_y and p_r are weighting factors for the x, y and A components, wherein

space="preserve" listing-type="equation">xd=(β

.sub.x -Z.sub.bx).sup.2

space="preserve" listing-type="equation">yd=(β

.sub.y -Z.sub.by).sup.2 ##EQU13## wherein (Z.sub.bx, Z.sub.by) represent an existing mean value of components of the candidate cluster points, wherein (β

.sub.x, β

.sub.y) represent a mean displacement of the select path P from the candidate cluster Z over their overlap length, wherein (x.sub.d, y.sub.d) represent the square of a difference between the existing mean value (Z.sub.bx, Z.sub.by) and the mean displacement (β

.sub.x, β

.sub.y), wherein (x.sub.d, y.sub.d represent the square of a difference between the existing mean value (Z.sub.bx, Z.sub.by) and the mean displacement (β

.sub.x, β

.sub.y) for the displacement of the paths contributing to the cluster Z, wherein (γ

.sub.x, γ

.sub.y) represent mean values for path displacements from the candidate cluster Z if the select path P is merged with cluster Z, and wherein (xx, yy) represent new variance values in path displacements from the candidate cluster Z if P is merged with Z.

3. A method for tracking, in real time, a moving object represented by a video signal comprised of prior frames and a current frame and having an object image with features associated with the object, said method comprising the steps of:

matching a select feature path defined by x and y component values presenting motion of the features of an object with a candidate cluster defined by x and y component values chosen from a plurality of preexisting clusters representing motion of the object, said select feature path having a portion concurrent in time with some of the clusters in said plurality of clusters, said candidate cluster selected as the cluster closest in distance to the select feature path, the distance being calculated according to a function having a first component and a second component, the first component being representative of a maximum displacement between the select feature path and the candidate cluster, and the second component being representative of differences between velocity vectors defining motion of the select feature path and velocity vectors defining motion of the candidate cluster, wherein the select feature path is comprised of path points, wherein the candidate cluster is comprised of cluster points, and wherein the second component of the distance function is computed according to the following formula;

##EQU14## wherein P represents the select path, Z represents the candidate cluster, 1 represents the overlap length between P and Z, and wherein P_t.sbsb.a =Z_t.sbsb.r represents a first instant of overlap between the select path P and the candidate cluster Z.

4. The method of claim 3, wherein the distance is calculated according to the formula:

##EQU15## wherein p_x, p_y and p_r are weighting factors for the x, y and Δ

distances, and wherein

space="preserve" listing-type="equation">xd=(β

.sub.x -Z.sub.bx).sup.2

space="preserve" listing-type="equation">yd=(β

.sub.y -Z.sub.by).sup.2 ##EQU16## wherein (Z.sub.bx, Z.sub.by) represent an existing mean value of components of the candidate cluster points, wherein (β

.sub.x, β

.sub.y) represent a mean displacement of the select path P from the candidate cluster Z over their overlap length, wherein (x.sub.d, y.sub.d) represent the square of a difference between the existing mean value (Z.sub.bx, Z.sub.by) and the mean displacement (β

.sub.x, β

.sub.y), wherein (x.sub.d, y.sub.d) represent the square of a difference between the existing mean value (Z.sub.bx, Z.sub.by and the mean displacement (β

.sub.x, β

.sub.y) for the displacement of the paths contributing to the cluster Z, wherein (γ

.sub.x, γ

.sub.y) represent mean values for path displacements from the candidate cluster Z if the select path P is merged with cluster Z, and wherein (xx, yy) represent new variance values in path displacements from the candidate cluster Z if P is merged with Z.

5. The method of claim 4, wherein said updating step further comprises updating the parameters of the candidate cluster according to the formulas:

##EQU17## wherein Z'"'"' represents the updated component values for the candidate cluster, and wherein P_cx and P_cy are derived from;

##EQU18## wherein P_cx, P_cy represent the mean displacement of the path components comprising a portion of the candidate cluster.

6. A method for tracking, in real time, a moving object represented by a video signal derived from prior frames and a current frame, the video signal having an object image and a background image, the object image having a bounding contour with extrema of curvature, said method comprising the steps of:

(a) isolating, from the current frame, the object image from the background image;

(b) detecting the bounding contour of the isolated object image and the extrema of curvature of the bounding contour;

(c) identifying a plurality of points on the detected bounding contour, some points in said plurality defined as feature points corresponding to a location of a feature of the object and represented by the extrema of curvature, each point in said plurality having coordinates associated therewith;

(d) comparing the coordinates of some of said identified points of the object image of the current frame with coordinates of corresponding previously identified points of an object image of an immediately preceding frame for correlating at least some of said current identified points with said corresponding ones of said previously identified points, each of said previously identified points having a previously defined feature path associated therewith, each feature path having components representing motion of said feature paths;

(e) assigning at least some of said identified points to a select one of the previously identified feature paths based on the comparison from said comparing step, and updating the components of the select feature path with the components of the associated points;

(f) updating parameters of a select one of a plurality of preexisting clusters, at least one of which represents motion of the moving object, with the components of the updated select feature path if the updated select feature path is associated with the select cluster; and

(g) selecting a candidate cluster from said plurality of preexisting clusters if said updated select feature path is not associated with any of said preexisting clusters, and updating the parameters of the candidate cluster with the components of the select path, the candidate cluster being chosen based on a smallest distance value of a distance calculation between the select path and each of the preexisting clusters in said plurality of preexisting clusters.

7. The method of claim 6, wherein said distance calculation is derived according to a function having a first component and a second component, said first component being representative of a maximum displacement between the select feature path and the candidate cluster, and the second component being representative of the difference between velocity vectors defining motion of the select feature path and velocity vectors defining motion of the candidate cluster.

8. The method of claim 7, wherein said selecting step (g) further comprises forming a new cluster from said updated path if the smallest distance value between said updated path and each cluster in the plurality of preexisting clusters is greater than a predetermined threshold distance.

9. The method of claim 6, wherein the coordinates of the plurality of points includes location (x, y), orientation and curvature values and wherein said comparing step (d) further comprises calculating the distances between a selected previously identified feature point and the identified feature points of the object image of the current frame located within a neighborhood area containing a corresponding location of said selected previously identified feature point according to the formula:

space="preserve" listing-type="equation">(dx.sup.2 +dy.sup.2)/σ

.sub.x.sup.2 +do.sup.2 /σ

.sub.o.sup.2 +dc.sup.2 /σ

.sub.c.sup.2,

wherein dx, dy, do and dc, respectively, correspond to a difference in the location coordinates (x, y), orientation and curvature values between selected previously identified feature points and the identified feature point, and wherein σ

_x, σ

_o, and σ

_c are weighting values; and

selecting, as a candidate to the previous feature point, a current feature point in said neighborhood corresponding to the minimum distance calculation.

10. A device for tracking, in real time, a moving object represented by a video signal derived from prior frames and a current frame, the video signal having an object image and a background image, the object image having a bounding contour with extrema of curvature, said device comprising:

means for isolating, from the current frame, the object image from the background image;

means for detecting the bounding contour of the isolated object image and the extrema of curvature of the bounding contour;

means for identifying a plurality of points on the detected bounding contour, some points in said plurality defined as feature points corresponding to the location of a feature of the object and represented by the extrema of curvature, each point in said plurality having coordinates associated therewith;

means for comparing the coordinates of some of said identified points of the object image of the current frame with coordinates of corresponding previously identified points of an object image of an immediately preceding frame for correlating at least some of said current identified points with said corresponding ones of said previously identified points, each of said previously identified points having a previously defined feature path associated therewith, each feature path having components representing motion of said feature paths;

means for assigning at least some of said identified points to a select one of the previously identified feature paths and updating the components of the select feature path with the components of the associated points;

means for updating parameters of a select one of a plurality of preexisting clusters, at least one of which representing motion of the moving object, with the components of the select feature path if the select path is associated with the select cluster; and

means for selecting a candidate cluster if said updated select path is not associated with the select cluster, and updating parameters of the candidate cluster with the components of the select path, the candidate cluster being chosen based on a smallest distance value of a distance calculation between the select path and each cluster in the plurality of preexisting clusters.

11. The device of claim 10, wherein said selecting means further comprises calculating means for calculating the distance between the select path and each cluster in the plurality of preexisting clusters, said distance calculation being derived according to a function having a first component and a second component, the first component being representative of a maximum displacement between the select feature path and the candidate cluster, and the second component being representative of the difference between velocity vectors defining motion of the select feature path and velocity vectors defining motion of the candidate cluster.

12. The method of claim 11, further comprising means for forming a new cluster from said updated path if the smallest distance value between said updated path and each cluster in the plurality of preexisting clusters is greater than a predetermined threshold distance.