Method and apparatus for tracking a number of objects or object parts in image sequences

• US 7,965,867 B2
• Filed: 07/05/2007
• Issued: 06/21/2011
• Est. Priority Date: 07/10/2006
• Status: Active Grant

• Current Assignee: Fondazione Bruno Kessler
• Original Assignee: Fondazione Bruno Kessler
• Inventors: Lanz, Oswald
• Primary Examiner(s): Bella, Matthew C
• Assistant Examiner(s): Nakhjavan, Shervin
• Application Number:
  US11/773,483
• Publication Number:
  US 20080031492A1
• Time in Patent Office: 1,447 Days
• Field of Search: 382/103, 382/106, 348/169, 348/171
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A method for tracking a number of objects or object parts in image sequences utilizes a Bayesian-like approach to object tracking, computing, at each time a new image is available, a probability distribution over all possible target configurations for that time. The Bayesian-like approach to object tracking computes a probability distribution for the previous image, at time (t−1), is propagated to the new image at time (t) according to a probabilistic model of target dynamics, obtaining a predicted distribution at time (t). The Bayesian-like approach to object tracking also aligns the predicted distribution at time (t) with the evidence contained in the new image at time (t) according to a probabilistic model of visual likelihood.

Claims

1. A method for tracking a number of objects or object parts in image sequences, comprising following a Bayesian-like approach to object tracking, computing, at each time a new image is available, a probability distribution over target configurations for that time, said Bayesian-like approach to object tracking comprising:

computing a probability distribution for the previous image, at time (t−

1), and propagating the probability distribution to the new image, at time (t), according to a probabilistic model of target dynamics, to obtain a predicted, or prior, distribution at time (t); and

aligning the predicted distribution, at time (t), with the evidence contained in the new image, at time (t), according to a probabilistic model of visual likelihood, to obtain a posterior distribution at time (t);

wherein aligning the predicted distribution, at time (t), of a hypothetic target configuration associated with a new frame by performing image analysis includes,

identifying the image portion in which the target under consideration is expected to be visible under the current configuration, by using a shape rendering function,

assigning a probability value to each pixel of the identified image portion, which is computed as a probability of the target under consideration being expected to be visible under the current configuration being occluded by another target, the probability of the target under consideration being expected to be visible under the current configuration being occluded by another target being derived from prior distributions and shape models of other targets,

computing a degree of dissimilarity between visual features extracted from the identified image portion and a corresponding characterization, or appearance model, of the target, an importance, or weight, of the different pixels being calculated by the probability of the target under consideration being expected to be visible under the current configuration being occluded,

adding, to the value of degree of dissimilarity, a further dissimilarity term per each other tracked target, each further dissimilarity term being computed in form of an expectation, under a prior distribution, of the dissimilarity value, the expectation of dissimilarity value being calculated on configurations that map closer to an image recording device than the target under analysis wherein the target, currently under consideration, being neglected, and

calculating a distribution value, assigned to the hypothetic target configuration of the target under analysis, by multiplying a prior value with a negative exponential of an overall dissimilarity score.

2. The method according to claim 1, wherein said distribution value assigned to the hypothetic target configuration of the target under analysis is calculated recursively according to:

q(x_t^k|z<sub>1;

t</sub>)∝

ƒ

(z_t|x_t^k,z<sub>1;

t−

1</sub>)q(x_t^k|z<sub>1;

t−

1</sub>)

where;

x_t^k is the configuration of a scene at time t for a target k;

z_t is the image at time (t);

q(x_t^k|z<sub>1;

1</sub>) is a posterior distribution for each single target k, when multiple targets are present, at time (t); and

q(x_t^k|z<sub>1;

t−

1</sub>) is the prior distribution at time (t);

ƒ

(z_t|x_t^k, z<sub>1;

t−

1</sub>) is a multi-target likelihood function which implements the occlusion process, according;

3. The method according to claim 2, wherein said foreground term, L^k(z_t|w^k), calculates a dissimilarity score between the image part where target k is expected to be visible, and its appearance model by the following steps:

the image region identified by the shape model for the hypothesis x_t^k under analysis is first calculated;

pixel weights w^k are computed within the image area;

weighted appearance features are extracted from that region and compared to the appearance model of the target; and

the comparison being made in terms of a distance function between extracted and modeled appearance features and defining the value of L^k(z_t|w^k).

4. The method according to claim 2, wherein the background term, ∫

L^m(z_t|w^m|k)q(x_t^m|z<sub>1;

t−

1</sub>)dx_t^m,takes into account the occlusion evidence derived from the presence of target m≠

k, and is computed as the expectation of the foreground term of m under prior q(x_t^m|z<sub>1;

t−

1</sub>) when the configuration of k is blocked to hypothesis X_t^k under analysis; and

in that, for each x_t^m, a foreground score, L^m(z_t|w^m|k), is computed, using weight maps w^m|k;

the scores are modulated with the prior q(x_t^m|z<sub>1;

t−

1</sub>) and integrated.

5. The method according to claim 2, wherein the pixel weights and weight maps w^k, w^m|k are scalar images computed by attributing to each pixel of z a probability value in the range [0,1], the probability value being interpreted as the importance of that pixel when computing the likelihood score for target k.

6. The method according to claim 5, characterized in that the pixel weights and weight maps w^k, w^m|k are computed for each pixel u as follows:

7. The method according to claim 1, wherein the shape rendering function is a user-defined shape model of the target that is rendered onto the image for the specific configuration under analysis.

8. The method according to claim 1, wherein the computing of the degree of dissimilarity is user defined.

9. An apparatus for tracking a number of objects or object parts in image sequences comprising:

a processor;

said processor computing a probability distribution for the previous image, at time (t−

1 ), and propagating the probability distribution to the new image, at time (t), according to a probabilistic model of target dynamics, to obtain a predicted, or prior, distribution at time (t); and

said processor aligning the predicted distribution, at time (t), with the evidence contained in the new image, at time (t), according to a probabilistic model of visual likelihood, to obtain a posterior distribution at time (t);

said processor aligning the predicted distribution, at time (t), of a hypothetic target configuration associated with a new frame by,

identifying the image portion in which the target under consideration is expected to be visible under the current configuration, by using a shape rendering function,

assigning a probability value to each pixel of the identified image portion, which is computed as a probability of the target under consideration is expected to be visible under the current configuration being occluded by another target, the probability of the target under consideration being expected to be visible under the current configuration being occluded by another target being derived from prior distributions and shape models of other targets,

computing a degree of dissimilarity between visual features extracted from the identified image portion and a corresponding characterization, or appearance model, of the target, the importance, or weight, of the different pixels in this calculation being calculated by the probability of the target under consideration being expected to be visible under the current configuration being occluded,

adding, to the value of degree of dissimilarity, a further dissimilarity term per each other tracked target, each further dissimilarity term being computed in form of an expectation, under a prior distribution, of the dissimilarity values, the expectation of dissimilarity values being calculated on configurations that map closer to an image recording device than the target under analysis wherein the target, currently under consideration, being neglected, and

calculating a distribution value, assigned to the hypothetic target configuration of the target under analysis, by multiplying a prior value with the negative exponential of the overall dissimilarity score.

10. A recordable computer readable medium having a program recorded thereon, adapted to cause a process to be executed on a computer, the process comprising:

computing a probability distribution for the previous image, at time (t−

1 ), and propagating the probability distribution to the new image, at time (t), according to a probabilistic model of target dynamics, to obtain a predicted, or prior, distribution at time (t); and

aligning the predicted distribution, at time (t), with the evidence contained in the new image, at time (t), according to a probabilistic model of visual likelihood, to obtain a posterior distribution at time (t);

wherein aligning the predicted distribution, at time (t), of a hypothetic target configuration associated with a new frame by performing image analysis includes,

identifying the image portion in which the target under consideration is expected to be visible under the current configuration, by using a shape rendering function,

assigning a probability value to each pixel of the identified image portion, which is computed as a probability of the target under consideration is expected to be visible under the current configuration being occluded by another target, the probability of the target under consideration being expected to be visible under the current configuration being occluded by another target being derived from prior distributions and shape models of other targets,

computing a degree of dissimilarity between visual features extracted from the identified image portion and a corresponding characterization, or appearance model, of the target, the importance, or weight, of the different pixels in this calculation being calculated by the probability of the target under consideration being expected to be visible under the current configuration being occluded,

adding, to the value of degree of dissimilarity, a further dissimilarity term per each other tracked target, each further dissimilarity term being computed in form of an expectation, under a prior distribution, of the dissimilarity values, the expectation of dissimilarity values being calculated on configurations that map closer to an image recording device than the target under analysis wherein the target, currently under consideration, being neglected, and

calculating a distribution value, assigned to the hypothetic target configuration of the target under analysis, by multiplying a prior value with the negative exponential of the overall dissimilarity score.