Enhanced background model employing object classification for improved background-foreground segmentation

• US 7,190,809 B2
• Filed: 06/28/2002
• Issued: 03/13/2007
• Est. Priority Date: 06/28/2002
• Status: Expired due to Fees

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Trajkovic, Miroslav, Gutta, Srinivas, Colmenarez, Antonio J.
• Primary Examiner(s): Mehta; Bhavesh M.
• Assistant Examiner(s): LAVIN, CHRISTOPHER L
• Application Number:
  US10/184,517
• Publication Number:
  US 20040001612A1
• Time in Patent Office: 1,719 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A method and apparatus are disclosed for generating and maintaining enhanced background models for use in background-foreground segmentation. Background models are modified to contain an indication of objects that are typically stationary. Thereafter, if an object moves and has been previously identified as an object that is typically stationary, the object will not unnecessarily be identified as part of the foreground during background-foreground segmentation. In an exemplary implementation, moving objects are classified into two sets. A first set includes objects that typically move independently and a second set includes objects that are typically stationary. Generally, once an object is assigned to the second (stationary object) set, the object will remain in the background, even if the object is moved (normally, movement of the object would cause the object to become part of the foreground).

Claims

1. A method, comprising:

retrieving a plurality of images from a location that is substantially stationary relative to a scene, each image of the plurality of images of said scene comprising a plurality of pixels;

obtaining a background model of said scene, wherein obtaining the background model comprises determining at least one probability distribution corresponding to pixels of each image of the plurality of images, the step of determining performed by using a model wherein at least some pixels in each image of the plurality of images are modeled as being dependent on other pixels, further wherein said background model comprises (i) a term representing a probability of a global state of the scene and (ii) a term representing a probability of pixel appearances conditioned to the global state of the scene, wherein the global state of the scene is other than a global motion state of the scene; and

providing two indications in said background model for moving objects, a first indication for objects that typically move independently relative to said scene and a second indication for objects that are typically stationary relative to said scene, wherein the background model further comprises an object classification process that modifies probability tables for relevant pixels of an image of the plurality of images of the background model to contain the second indication.

2. The method of claim 1, further comprising the step of transferring pixel information associated with a moved object to a new location of said object.

3. The method of claim 1, further comprising the step of maintaining an object with said second indication in said background even if said object is moved independent of said scene.

4. The method of claim 3, wherein said motion of an object in said background model is detected using an optical flow method.

5. The method of claim 4, wherein said optical flow method indicates a new location in said image of said moved object.

6. The method of claim 1, wherein said second indication that an object in said background model is typically stationary is provided even when said object is moved by more than a specified threshold.

7. The method of claim 1, wherein the method further comprises the steps of:

providing a training image to the model;

determining parameters of the model; and

performing the step of providing a training image and determining parameters for a predetermined number of training images.

8. A system comprising:

a memory that stores computer-readable code; and

a processor operatively coupled to said memory, said processor configured to implement said computer-readable code, said computer-readable code configured to;

retrieve a plurality of images from a location that is substantially stationary relative to a scene, each image of the plurality of images of said scene comprising a plurality of pixels;

obtain a background model of said scene, wherein obtaining the background model comprises determining at least one probability distribution corresponding to pixels of each image of the plurality of images, the step of determining performed by using a model wherein at least some pixels in each image of the plurality of images are modeled as being dependent on other pixels, further wherein said background model comprises (i) a term representing a probability of a global state of the scene and (ii) a term representing a probability of pixel appearances conditioned to the global state of the scene, wherein the global state of the scene is other than a global motion state of the scene; and

provide two indications in said background model for moving objects, a first indication for objects that typically move independently relative to said scene and a second indication for objects that are typically stationary relative to said scene, wherein the background model further comprises an object classification process that modifies probability tables for relevant pixels of an image of the plurality of images of the background model to contain the second indication.

9. The system of claim 8, wherein said processor is further configured to transfer pixel information associated with a moved object to a new location of said object.

10. The system of claim 8, wherein said processor is further configured to maintain an object with said second indication in said background even if said object is moved independent of said scene.

11. The system of claim 10, wherein said motion of an object in said background model is detected using an optical flow system.

12. The system of claim 11, wherein said optical flow system indicates a new location in said image of said moved object.

13. The system of claim 8, wherein said second indication that an object in said background model is typically stationary is provided even when said object is moved by more than a specified threshold.

14. A computer-readable medium having computer-readable code embodied thereon, said computer-readable program code comprising:

a step to retrieve a plurality of images from a location that is substantially stationary relative to a scene, each image of the plurality of images of said scene comprising a plurality of pixels;

a step to obtain a background model of said scene, wherein obtaining the background model comprises determining at least one probability distribution corresponding to pixels of each image of the plurality of images, the step of determining performed by using a model wherein at least some pixels in each image of the plurality of images are modeled as being dependent on other pixels, further wherein said background model comprises (i) a term representing a probability of a global state of the scene and (ii) a term representing a probability of pixel appearances conditioned to the global state of the scene, wherein the global state of the scene is other than a global motion state of the scene; and

a step to provide two indications in said background model for moving objects, a first indication for objects that typically move independently relative to said scene and a second indication for objects that are typically stationary relative to said scene, wherein the background model further comprises an object classification process that modifies probability tables for relevant pixels of an image of the plurality of images of the background model to contain the second indication.

15. The computer readable medium of claim 14, wherein the global state of said scene is one of a global illumination and a global color of said scene.

16. A method, comprising:

retrieving a plurality of images from a location that is substantially stationary relative to a scene, each image of the plurality of images of said scene comprising a plurality of pixels;

obtaining a background model of said scene, wherein obtaining the background model comprises determining at least one probability distribution corresponding to pixels of each image of the plurality of images, the step of determining performed by using a model wherein at least some pixels in each image of the plurality of images are modeled as being dependent on other pixels, further wherein said background model comprises (i) a term representing a probability of a global state of the scene and (ii) a term representing a probability of pixel appearances conditioned to the global state of the scene, wherein the global state of the scene is other than a global motion state of the scene; and

providing two indications in said background model for moving objects, a first indication for an object that typically moves independently relative to said scene and a second indication for an object that is an inanimate object relative to said scene, wherein the background model further comprises an object classification process that modifies probability tables for relevant pixels of an image of the plurality of images of the background model to contain the second indication.

17. The method of claim 16, further comprising the step of maintaining said inanimate object in said background even if said inanimate object is moved independent of said scene.