Method and system for multi-modal component-based tracking of an object using robust information fusion

• US 7,072,494 B2
• Filed: 02/16/2005
• Issued: 07/04/2006
• Est. Priority Date: 02/20/2004
• Status: Expired due to Fees

• Current Assignee: Siemens Corp. (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Georgescu, Bogdan, Zhou, Xiang Sean, Rao, R. Bharat, Comaniciu, Dorin
• Primary Examiner(s): Dang, Duy M.
• Application Number:
  US11/058,784
• Publication Number:
  US 20050185826A1
• Time in Patent Office: 503 Days
• Field of Search: 382/103, 382/100, 382/107, 382/236, 382/238, 348/143, 348/155
• US Class Current: 382/103
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 10/754 involving a deformation of the sample pattern or of the reference pattern; Elastic matching

Abstract

A system and method for tracking an object is disclosed. A video sequence including a plurality of image frames are received. A sample based representation of object appearance distribution is maintained. An object is divided into one or more components. For each component, its location and uncertainty with respect to the sample based representation are estimated. Variable-Bandwidth Density Based Fusion (VBDF) is applied to each component to determine a most dominant motion. The motion estimate is used to determine the track of the object.

Claims

1. A method for tracking an object comprising the steps of:

receiving a video sequence comprised of a plurality of image frames;

maintaining a sample based nonparametric representation of object appearance distribution, each sample acquired at a different time instance during object tracking or any past object appearance instance;

dividing an object into one or more components for each image frame;

for each object component, estimating its motion measurement and uncertainty with respect to the sample based representation;

computing fused motion estimates by robustly fusing all motion measurements and uncertainties for each object component to determine a most dominant motion; and

using the fused motion estimates to determine the track of the object in subsequent image frames.

2. The method of claim 1 wherein the tracking of the object is performed on each subsequent image frame containing the object, each subsequent image frame being representative of a different time instance.

3. The method of claim 1 wherein the relative location of the components can be constrained using global parametric motion.

4. The method of claim 1 wherein the residual error associated with the components motion measurement is measured relative to model templates.

5. The method of claim 4 wherein if the residual error is below a predetermined threshold, the current frame is added to the model templates.

6. The method of claim 5 wherein a high residual error indicates that the object is at least partially occluded.

7. The method of claim 5 wherein a high residual error indicates that the object is affected by a change in illumination conditions.

8. The method of claim 1 wherein the object being tracked is a face.

9. The method of claim 1 wherein the object being tracked is a human body.

10. The method of claim 1 wherein the sample based representation of object appearance distribution comprises model templates obtained from past object appearance instances of the image frames.

11. The method of claim 1 wherein the sample based representation of object appearance distribution comprises model templates obtained during object tracking of the image frames.

12. The method of claim 1 wherein the relative location of the components can be constrained using a shape model.

13. The method of claim 1 wherein the step of computing fused motion estimates uses Variable Bandwidth Density based Fusion.

14. A method for tracking a candidate object in a medical video sequence comprising a plurality of image frames, the object being represented by a plurality of labeled control points in each image frame, each control point being acquired at a different time instance during object tracking or any past object appearance instance the method comprising the steps of:

estimating motion measurement and uncertainty for each control point;

maintaining multiple appearance models;

comparing each control point to one or more models;

using a VBDF estimator to determine a most likely current location of each control point in a given image frame;

concatenating coordinates for all of the control points;

fusing the set of control points with a model that most closely resemble the set of control points; and

using the fused set of control points to determine the track of the obiect in subsequent image frames.

15. The method of claim 14 wherein the control points are associated with a contour of an endocardial wall.

16. The method of claim 15 wherein the method tracks deformable motion of the endocardial wall.

17. A system for tracking an object comprising:

at least one camera for capturing a video sequence of image frames;

a processor associated with the at least one camera, the processor performing the following steps;

i). maintaining a sample based nonparametric representation of object appearance distribution, each sample acquired at a different time instance during object tracking or any past object appearance instance;

ii). dividing an object into one or more components for each image frame;

iii). for each object component, estimating its motion measurement and uncertainty with respect to the sample based representation;

iv). computing fused motion estimates by robustly fusing all motion measurements and uncertainty for each object component to determine a most dominant motion; and

v). using the fused motion estimate to determine the track of the object in subsequent image frames.

18. The system of claim 17 wherein the tracking of the object is performed on each subsequent image frame containing the object, each subsequent image frame being representative of a different time instance.

19. The system of claim 17 wherein the relative location of the components can be constrained using global parametric motion.

20. The system of claim 17 wherein the residual error associated with the components motion measurement is measured relative to model templates.

21. The system of claim 20 wherein if the residual error is below a predetermined threshold, the current frame is added to the model templates.

22. The system of claim 21 wherein a high residual error indicates that the object is at least partially occluded.

23. The system of claim 21 wherein a high residual error indicates that the object is affected by a change in illumination conditions.

24. The system of claim 17 wherein the object being tracked is a face.

25. The system of claim 17 wherein the object being tracked is a human body.

26. The system of claim 17 wherein the sample based representation of object appearance distribution comprises model templates obtained from past object appearance instances of the image frames.

27. The system of claim 17 wherein the sample based representation of object appearance distribution comprises model templates obtained during object tracking of the image frames.

28. The system of claim 17 wherein the relative location of the components can be constrained using a shape model.

29. The method of claim 17 wherein the step of computing fused motion estimates uses Variable Bandwidth Density based Fusion.