APPROACH FOR RESOLVING OCCLUSIONS, SPLITS AND MERGES IN VIDEO IMAGES

• US 20080240496A1
• Filed: 03/26/2007
• Published: 10/02/2008
• Est. Priority Date: 03/26/2007
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Senior, Andrew W.
• Granted Patent: US 8,086,036 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/251 involving models

Abstract

Aspects of the present invention provide a solution for resolving an occlusion in a video image. Specifically, an embodiment of the present invention provides an environment in which portions of a video image in which occlusions have occurred may be determined and analyzed to determine the type of occlusion. Furthermore, regions of the video image may be analyzed to determine which object in the occlusion the region belongs to. The determinations and analysis may use such factors as pre-determined attributes of an object, such as color or texture of the object and/or a temporal association of the object, among others.

Claims

1. A method for resolving an occlusion in a video image, comprising:

detecting a plurality of foreground regions in a series of video images;

associating a foreground region with one or more tracked objects;

determining whether any object of the plurality of objects may occlude or be occluded by another object of the plurality of objects by comparing current attributes and object models of each of the objects with the video image and plurality of foreground regions;

assigning pixels overlapped by either object model to one or other of the object and the another object; and

performing an analysis of an unresolved region to associate the unresolved region with one of the objects or to assign the region with designation ambiguous region.

2. The method of claim 1, wherein the associating assigns the unresolved region based on at least one of:

assigning the unresolved region to the object if the unresolved region is encompassed by regions that are assigned to the object;

assigning the unresolved region to the object if the unresolved region is adjacent to a region that is assigned to the object;

assigning the unresolved region to the object if the unresolved region is closer in proximity to the object than to the occluding object;

3. The method of claim 1, wherein the associating assigns the unresolved region to the object if it is adjacent to a previously assigned region.

4. The method of claim 1, wherein the associating assigns the unresolved region to the object if current attributes of the unresolved region are more consistent with attributes in the object model of the object than attributes in the object model of the occluding object.

5. The method of claim 1, wherein the associating assigns the unresolved region to the object based on use of a Markov Random Field to determine a global optimum assignment of the unresolved region based on individual region likelihoods and a boundary penalty.

6. The method of claim 1, wherein the attributes include at least one of:

color, texture, or anticipated position within the video image.

7. A system for resolving an occlusion in a video image, comprising:

an object detector for detecting a plurality of foreground regions in a series of video images and associating a foreground regions with one or more tracked objects;

an occlusion determinator for determining whether any object of the plurality of objects may occlude or be occluded another object by comparing current attributes and object models of each of the objects with the video image and plurality of the foreground regions; and

a region analyzer for assigning regions overlapped by either object model to one of the object and the another object and for performing an analysis of an unresolved region to associate the unresolved region with one of the object of the another object or to assign the region with designation ambiguous region.

8. The system of claim 7, wherein the associating assigns the unresolved region based on at least one of:

assigning the unresolved region to the object if the unresolved region is encompassed by regions that are assigned to the object;

assigning the unresolved region to the object if the unresolved region is adjacent to a region that is assigned to the object;

assigning the unresolved region to the object if the unresolved region is closer in proximity to the object than to the occluding object;

9. The system of claim 7, wherein the associating assigns the unresolved region to the object if it is adjacent to a previously assigned region.

10. The system of claim 7, wherein the associating assigns the unresolved region to the object if current attributes of the unresolved region are more consistent with attributes in the object model of the object than attributes in the object model of the occluding object.

11. The system of claim 7, wherein the associating assigns the unresolved region to the object based on use of a Markov Random Field to determine a global optimum assignment of the unresolved region based on individual region likelihoods and a boundary penalty.

12. The system of claim 7, wherein the attributes include at least one of:

color, texture, or anticipated position within the video image.

13. A program product stored on a computer readable medium for resolving an occlusion in a video image, the computer readable medium comprising:

program code for detecting a plurality of foreground regions in a series of video images;

program code for associating a foreground regions with one or more tracked objects;

program code for determining whether any object of the plurality of objects may occlude or be occluded by another object of the plurality of objects by comparing current attributes and object models of each of the objects with the video image and plurality of foreground regions;

program code for assigning the unresolved region to the object if the unresolved region is adjacent to a region that is assigned to the object; and

program code for performing an analysis of an unresolved region to associate the unresolved region with one of the objects or to assign the region with designation ambiguous region.

14. The program product of claim 13, wherein the associating assigns the unresolved region based on at least one of:

assigning the unresolved region to the object if the unresolved region is encompassed by regions that are assigned to the object;

assigning the unresolved region to the object if the unresolved region is adjacent to a region that is assigned to the object;

assigning the unresolved region to the object if the unresolved region is closer in proximity to the object than to the occluding object;

15. The program product of claim 13, wherein the associating assigns the unresolved region to the object if it is adjacent to a previously assigned region.

16. The program product of claim 13, wherein the associating assigns the unresolved region to the object if current attributes of the unresolved region are more consistent with attributes in the object model of the object than attributes in the object model of the occluding object.

17. The program product of claim 13, wherein the associating assigns the unresolved region to the object based on use of a Markov Random Field to determine a global optimum assignment of the unresolved region based on individual region likelihoods and a boundary penalty.

18. The program product of claim 13, wherein the attributes include at least one of:

color, texture, or anticipated position within the video image.

24. The method of claim 17, wherein the evidence includes at least one of:

whether a comparison, in the case that one of the two objects is determined to be an object of a particular type, of a size of the object with the size of the object of the particular type falls outside of a known object type threshold;

whether a gap between the multiple objects remains constant within a gap threshold; and

whether the temporal association of each of the multiple objects is similar within a temporal association threshold.

19. A method for deploying a system for resolving an occlusion in a video image, comprising:

providing a computer infrastructure being operable to;

detect a plurality of foreground regions in a series of video images;

associate a foreground region with one or more tracked objects;

determine whether any object of the plurality of objects may occlude or be occluded by another object of the plurality of objects by comparing current attributes and object models of each of the objects with the video image and plurality of foreground regions;

assign pixels overlapped by either object model to one or other of the object and the another object; and

perform an analysis of an unresolved region to associate the unresolved region with one of the objects or to assign the region with designation ambiguous region.

20. The method of claim 19, wherein the performing of the analysis assigns the unresolved region based on at least one of:

assigning the unresolved region to the object if the unresolved region is encompassed by regions that are assigned to the object;

assigning the unresolved region to the object if the unresolved region is adjacent to a region that is assigned to the object;

assigning the unresolved region to the object if the unresolved region is closer in proximity to the object than to the occluding object;

assigning the unresolved region to the object if current attributes of the unresolved region are more consistent with attributes in the object model of the object than attributes in the object model of the occluding object;

or

assigning the unresolved region to the object based on use of a Markov Random Field to determine a global optimum assignment of the unresolved region based on individual region likelihoods and a boundary penalty.

21. A method for determining, if an object has split, whether the object should be classified into multiple objects, comprising:

creating a fission model that tracks a tendency of the object to include multiple non-contiguous regions;

tracking, in association with the object, a region that is in close proximity with the object;

accumulating, based on the fission model, evidence that the region and the object should be separate;

making, based on the accumulated evidence, a determination that the region and the object should be classified into multiple objects; and

tracking the region and the object as separate objects.

22. The method of claim 21 wherein the evidence includes at least one of:

whether a gap between the multiple objects has exceeded a gap threshold;

whether the gap between the multiple objects is increasing;

or

whether a time period that the object has remained split has exceeded a threshold time period.

23. A method for determining whether multiple objects should be classified as a single object, comprising:

creating a fusion model that tracks a tendency of the multiple objects to include behave in a similar manner;

tracking, in association with an object, one or more other objects that are in close proximity with the object;

accumulating, based on the fusion model, evidence that the multiple objects should be classified as one object;

making, based on the accumulated evidence, a determination that the objects should be classified as one object; and

tracking the objects as one object.