Object tracker for visually tracking object motion

• US 7,940,957 B2
• Filed: 05/31/2007
• Issued: 05/10/2011
• Est. Priority Date: 06/09/2006
• Status: Active Grant

• Current Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Ikenoue, Shoichi, Morisada, Hidehiko, Sakamoto, Takayuki
• Primary Examiner(s): Tabatabai; Abolfazl
• Application Number:
  US11/755,870
• Publication Number:
  US 20080063236A1
• Time in Patent Office: 1,440 Days
• Field of Search: 382/103, 382/107, 382/236, 382/287, 348/169, 348/170, 348/171, 348/172, 348/94, 348/154, 348/155, 348/208.1, 348/208.2, 348/208.14, 348/208.16, 348/352
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image

Abstract

A tracking start-and-stop determining unit of a tracking processing unit determines whether to start or stop tracking based on a predetermined condition. During tracking, a sampling unit 29 creates or eliminates particles using a probability density distribution estimated for the previous image frame. A first to n^th model tracking units transit the particles according to motion models respectively assigned, and observe the likelihood of a candidate curve corresponding to each particle with respect to an edge image generated by an image processing unit. A tracking integration unit determines a motion model which causes the highest likelihood, and estimates a probability density distribution using an observation likelihood based on the motion model.

Claims

1. An object tracker comprising:

a sampling unit which generates particles based on an estimated existence probability distribution of a target object in a first image frame, the particles being used for a particle filter in a parameter space where candidate contours of the target object are defined and each particle defines a candidate contour, the first image frame being contained in a video stream together with a second image frame;

a plurality of model tracking units which correspond respectively to multiple motion models and observe likelihoods of the candidate contours with respect to the target object in the second image frame, the candidate contours being defined by respective particles transited according to each motion model; and

a tracking integration unit which chooses one of the motion models based on a comparison of the likelihoods observed by the model tracking units, and estimates an existence probability distribution of the target object in the second image frame based on the likelihood of a candidate contour which is defined according to the motion model chosen.

2. The object tracker according to claim 1, wherein the tracking integration unit chooses a motion model which causes the highest of the likelihoods observed by the plurality of model tracking units.

3. The object tracker according to claim 1, wherein the plurality of model tracking units include a plurality of Gaussian motion model tracking units which transit the particles generated by the sampling unit so that the particles show a Gaussian distribution centered around each coordinate in the parameter space, and

wherein each of the Gaussian motion model tracking units transits the particles so that the particles show a Gaussian distribution with different standard deviation.

4. The object tracker according to claim 1, wherein the plurality of model tracking units include autoregression motion model tracking units which perform autoregression prediction of a motion of the target object based on multiple image frames contained in the video stream, and transit, based on a result of the prediction, the particles generated by the sampling unit, and Gaussian motion model tracking units which transit the particles generated by the sampling unit so that the particles show a Gaussian distribution centered around each coordinate in the parameter space, and

wherein a distribution region of the particles showing the Gaussian distribution does not overlap with at least a part of a distribution region of the particles transited by the autoregression motion model tracking unit.

5. The object tracker according to claim 1, further comprising an image processing unit which extracts an edge of the target object from the second image frame,

wherein the model tracking unit sets for each of a plurality of points including in a candidate curve being the candidate contour, as a search area, an area having predetermined shape and size and centered around the point, detects an edge of the target object closest to the point within the search area to calculate a distance between the point and the edge, and calculates the likelihood using the distance for each candidate curve, the search area being set based on a size of the candidate curve.

6. The object tracker according to claim 1, wherein the tracking integration unit chooses a motion model which causes a highest likelihood, for each of the particles generated by the sampling unit.

7. The object tracker according to claim 1, wherein the tracking integration unit chooses a motion model which causes a maximum sum of the likelihoods of the respective particles generated by the sampling unit.

8. The object tracker according to claim 1, wherein

the model tracking unit observes the likelihood for each of multiple target objects, and

wherein the tracking integration unit estimates an existence probability distribution for each of the multiple target objects, and generates image frame data of a tracking result by combining a plurality of existence probability distributions obtained.

9. The object tracker according to claim 1, wherein the tracking integration unit detects an occurrence of occlusion of the target object, and

wherein the sampling unit stops generating particles for a target object hiding due to the occlusion when a predetermined time has elapsed from the occurrence of occlusion detected by the tracking integration unit.

10. The object tracker according to claim 1, wherein the candidate contour is described by a B-spline curve which is defined by control points and unit points.

11. The object tracker according to claim 1, further comprising a storage unit which stores in advance a plurality of curves each describing the candidate contour and having shapes of expected target objects,

wherein one curve is chosen as the candidate contour according to a shape of an actual target object, from the storage unit when starting tracking.

12. The object tracker according to claim 1, wherein the sampling unit further places the particles uniformly over a predetermined area at predetermined time intervals,

wherein the model tracking unit further observes a likelihood for each of the particles based on an edge image of an image frame immediately after the particles are placed, the image frame being contained in the video stream, and

wherein the object tracker further comprises a tracking start-and-stop determining unit which determines that a target object appears when the likelihoods, observed by the model tracking unit, for the particles uniformly placed are more than a predetermined threshold.

13. An object tracking method performed by a computer, the method comprising:

storing a received video stream in a memory;

reading a first image frame in the video stream from the memory, and applying multiple hypothetical motion models to multiple candidate contours of a target object derived from an existence probability distribution of the target object in the first image frame;

comparing the candidate contours after applying the multiple hypothetical motion models with true contour of the target object in a second image frame read from the memory, and choosing a motion model with the highest likelihood from the multiple hypothetical motion models based on the result of comparison; and

estimating an existence probability distribution of the target object in the second image frame based on the chosen motion model.

14. A non-transitory computer-readable medium having stored a computer program product causing a computer to perform an object tracking, comprising:

a program code module for storing a received video stream in a memory;

a program code module for reading a first image frame in the video stream from the memory, and applying multiple hypothetical motion models to multiple candidate contours of a target object derived from an existence probability distribution of the target object in the first image frame;

a program code module for comparing the candidate contours after applying the multiple hypothetical motion models with true contour of the target object in a second image frame read from the memory, and choosing a motion model with the highest likelihood from the multiple hypothetical motion models based on a result of comparison; and

a program code module for estimating an existence probability distribution of the target object in the second image frame based on the chosen motion model.