Morphological postprocessing for object tracking and segmentation

• US 6,674,925 B1
• Filed: 02/08/2000
• Issued: 01/06/2004
• Est. Priority Date: 02/08/2000
• Status: Expired due to Fees

• Current Assignee: University of Washington
• Original Assignee: University of Washington
• Inventors: Schoepflin, Todd, Kim, Yongmin
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US09/500,259
• Time in Patent Office: 1,428 Days
• Field of Search: 382/103, 382/173, 382/190, 382/212, 382/213, 382/215, 382/216, 382/217, 382/256, 382/257, 382/263, 382/266, 382/268, 382/275, 382/308, 348/170, 348/155, 348/661, 348/169, 345/563
• US Class Current: 382/308
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/155 involving morphological operators
• CPC Class Codes: G06T 7/251 involving models

Abstract

An object mask and a set of control points defined for a given frame are subjected to morphological processing to remove false edge points and provide a more robust mask for use in tracking the object in a next frame. The morphological processing is performed on a frame by frame basis corresponding to object tracking so that errors added in by the object tracker do not accumulate, and instead are filtered out. Rapidly moving objects which are troublesome for edge-based object trackers are more readily tracked. Also, regions of low contrast or regions locked onto in the background when trying to identify the object are more readily distinguished and eliminated from the object mask.

Claims

1. A method for tracking an object among a plurality of image frames, comprising the steps of:

altering a template using a first set of constraints to define a search area for locating a boundary of the object within a first image frame of the plurality of image frames, wherein the template comprises a set of control points;

determining an estimated location and estimated shape of the object in said first image frame and revising the template to correspond to the estimated location and estimated shape of the object, the revised template comprising a first mask and said set of control points;

morphologically eroding the first mask to remove extraneous areas having less than a prescribed width;

deriving a final mask from the morphologically eroded first mask;

revising the set of control points to eliminate control points that are not on the boundary of the final mask; and

using the revised set of control points and the final mask to define a second image frame template for tracking the object in a second image frame of the plurality of image frames.

2. The method of claim 1, in which said eroding comprises eliminating any pixel in the first mask which does not have all surrounding pixels in the first mask.

3. The method of claim 1, further comprising adding to the morphologically eroded first mask any pixel which has any surrounding pixel within the morphologically eroded first mask to achieve a morphologically dilated mask.

4. A method for tracking an object among a plurality of image frames, comprising the steps of:

altering a template using a first set of constraints to define a search area for locating a boundary of the object within a first image frame of the plurality of image frames, wherein the template comprises a set of control points;

determining an estimated location and estimated shape of the object in said first image frame and revising the template to correspond to the estimated location and estimated shape of the object, the revised template comprising a first mask and said set of control points;

morphologically eroding the first mask to remove extraneous features based on a first test criteria;

deriving a final mask from the morphologically eroded first mask;

revising the set of control points to eliminate control points that are not on the boundary of the final mask; and

using the revised set of control points and the final mask to define a second image frame template for tracking the object in a second image frame of the plurality of image frames;

in which the morphologically eroded first mask comprises a plurality of unconnected regions, and wherein the final mask is derived from only one region of the plurality of unconnected regions.

5. The method of claim 4 in which said one region is selected for encompassing a highest number of control points among the plurality of unconnected regions.

6. An apparatus for tracking an object among a plurality of image frames, comprising:

an object segmenter which derives a template of an object to be tracked, the template comprising a mask and a set of control points;

an object tracker for altering the template using a first set of constraints to define a search area for locating a boundary of the object within a first image frame of the plurality of image frames, the object tracker determining an estimated location and estimated shape of the object in said first image frame and revising the template to correspond to the estimated location and estimated shape of the object, the revised template comprising a first mask and said set of control points; and

a morphological postprocessor which processes the first mask and said set of control points to define a revised mask and revised set of control points which define the template for a second image frame of the plurality of image frames, the revised mask, relative to the first mask, allowing the object tracker to more accurately track the object among the plurality of frames for a scenario in which the object either one or both of moves or is occluded;

in which the template is a first image frame template, and in which the morphological postprocessor comprises a processor which

morphologically erodes the first mask to remove extraneous areas having less than a prescribed width;

derives a final mask from the morphologically eroded first mask;

revises the set of control points to eliminate control points that are not on the boundary of the final mask, wherein said final mask is the revised mask; and

uses the revised set of control points and the final mask to define a second image frame template for tracking the object in a second image frame of the plurality of image frames.

7. The apparatus of claim 8, in which said eroding eliminates any pixel in the first mask which does not have all surrounding pixels in the first mask.

8. The apparatus of claim 6, in which the morphological postprocessor further performs a function of expanding the morphologically eroded first mask by adding to the morphologically eroded first mask any pixel which has any surrounding pixel within the morphologically eroded first mask to achieve a morphologically dilated mask.

9. An apparatus for tracking an object among a plurality of image frames, comprising:

an object segmenter which derives a template of an object to be tracked, the template comprising a mask and a set of control points;

an object tracker for altering the template using a first set of constraints to define a search area for locating a boundary of the object within a first image frame of the plurality of image frames, the object tracker determining an estimated location and estimated shape of the object in said first image frame and revising the template to correspond to the estimated location and estimated shape of the object, the revised template comprising a first mask and said set of control points; and

a morphological postprocessor which processes the first mask and said set of control points to define a revised mask and revised set of control points which define the template for a second image frame of the plurality of image frames, the revised mask, relative to the first mask, allowing the object tracker to more accurately track the object among the plurality of frames for a scenario in which the object either one or both of moves or is occluded;

in which the morphologically eroded first mask comprises a plurality of unconnected regions, and wherein the final mask is derived from only one region of the plurality of unconnected regions.

10. The apparatus of claim 9, in which said one region is selected for encompassing a highest number of control points among the plurality of unconnected regions.

11. A method for tracking an object among a plurality of image frames, comprising the steps of:

identifying a first mask which is an initial estimate of a shape and location of the object within a current frame, the first mask including a set of control points along an outer boundary of the first mask;

morphologically eroding the first mask to remove mask areas having less than a prescribed width to achieve an eroded mask;

morphologically dilating the eroded mask to achieve a dilated mask;

deriving a final mask from the dilated mask, the final mask being an estimate of the object shape and location for the current frame;

revising the set of control points to eliminate control points that are not on the boundary of the final mask; and

using the revised set of control points and the final mask as the first mask and set of control points for a subsequent frame.

12. The method of claim 11, in which said eroding comprises eliminating any pixel in the first mask which does not have all surrounding pixels in the first mask.

13. The method of claim 11, in which said dilating comprises adding to the morphologically eroded first mask any pixel which has any surrounding pixel within the morphologically eroded first mask.

14. The method of claim 11, in which the dilated mask includes a plurality of unconnected areas, and said deriving comprises selecting one of said plurality of unconnected areas as the final mask.

15. The method of claim 14, in which said selected one area encompasses a greatest number of control points among the plurality of unconnected areas.