Method and system for multi-modal component-based tracking of an object using robust information fusion

• US 20050185826A1
• Filed: 02/16/2005
• Published: 08/25/2005
• Est. Priority Date: 02/20/2004
• Status: Active Grant

• Current Assignee: Siemens Corp. (Siemens AG)
• Original Assignee: Siemens Medical Solutions USA Incorporated (Siemens AG)
• Inventors: Georgescu, Bogdan, Zhou, Xiang Sean, Rao, R. Bharat, Comaniciu, Dorin
• Granted Patent: US 7,072,494 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 10/754 involving a deformation of the sample pattern or of the reference pattern; Elastic matching

Abstract

A system and method for tracking an object is disclosed. A video sequence including a plurality of image frames are received. A sample based representation of object appearance distribution is maintained. An object is divided into one or more components. For each component, its location and uncertainty with respect to the sample based representation are estimated. Variable-Bandwidth Density Based Fusion (VBDF) is applied to each component to determine a most dominant motion. The motion estimate is used to determine the track of the object.

Claims

1. A method for tracking an object comprising the steps of:

receiving a video sequence comprised of a plurality of image frames;

maintaining a sample based representation of object appearance distribution;

dividing an object into one or more components;

for each component, estimating its location and uncertainty with respect to the sample based representation;

applying Variable-Bandwidth Density Based Fusion (VBDF) to each component to determine a most dominant motion; and

using the motion estimate to determine the track of the object.

2. The method of claim 1 wherein the tracking of the object is performed on each subsequent image frame containing the object.

3. The method of claim 1 wherein the relative location of the components can be constrained using global parametric motion.

4. The method of claim 1 wherein the residual error associated with the components location is measured relative to model templates.

5. The method of claim 4 wherein if the residual error is below a predetermined threshold, the current frame is added to the model templates.

6. The method of claim 5 wherein a high residual error indicates that the object is at least partially occluded.

7. The method of claim 5 wherein a high residual error indicates that the object is affected by a change in illumination conditions.

8. The method of claim 1 wherein the object being tracked is a face.

9. The method of claim 1 wherein the object being tracked is a human body.

10. The method of claim 1 wherein the sample based representation of object appearance distribution comprises model templates obtained offline from the image frames;

11. The method of claim 1 wherein the sample based representation of object appearance distribution comprises model templates obtained online from the image frames.

12. The method of claim 1 wherein the relative location of the components can be constrained using a shape model.

13. A method for tracking a candidate object in a medical video sequence comprising a plurality of image frames, the object being represented by a plurality of labeled control points, the method comprising the steps of:

estimating a location and uncertainty for each control point;

maintaining multiple appearance models;

comparing each control point to one or more models;

using a VBDF estimator to determine a most likely current location of each control point;

concatenating coordinates for all of the control points; and

fusing the set of control points with a model that most closely resemble the set of control points.

14. The method of claim 13 wherein the control points are associated with a contour of an endocardial wall.

15. The method of claim 14 wherein the method tracks deformable motion of the endocardial wall.

16. A system for tracking an object comprising:

at least one camera for capturing a video sequence of image frames;

a processor associated with the at least one camera, the processor performing the following steps;

i). maintaining a sample based representation of object appearance distribution;

ii). dividing an object into one or more components;

iii). for each component, estimating its location and uncertainty with respect to the sample based representation;

iv). applying Variable-Bandwidth Density Based Fusion (VBDF) to each component to determine a most dominant motion; and

v). using the motion estimate to determine the track of the object.

17. The system of claim 16 wherein the tracking of the object is performed on each subsequent image frame containing the object.

18. The system of claim 16 wherein the relative location of the components can be constrained using global parametric motion.

19. The system of claim 16 wherein the residual error associated with the components location is measured relative to model templates.

20. The system of claim 19 wherein if the residual error is below a predetermined threshold, the current frame is added to the model templates.

21. The system of claim 20 wherein a high residual error indicates that the object is at least partially occluded.

22. The system of claim 20 wherein a high residual error indicates that the object is affected by a change in illumination conditions.

23. The system of claim 16 wherein the object being tracked is a face.

24. The system of claim 16 wherein the object being tracked is a human body.

25. The system of claim 16 wherein the sample based representation of object appearance distribution comprises model templates obtained offline from the image frames;

26. The system of claim 16 wherein the sample based representation of object appearance distribution comprises model templates obtained online from the image frames.

27. The system of claim 16 wherein the relative location of the components can be constrained using a shape model.