Foreground background separation in a scene with unstable textures

• US 8,837,823 B2
• Filed: 10/25/2011
• Issued: 09/16/2014
• Est. Priority Date: 10/29/2010
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Partis, Ashley, Gupta, Amit Kumar, Pakulski, Peter Jan, Lin, Qianlu
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US13/281,113
• Publication Number:
  US 20120106837A1
• Time in Patent Office: 1,057 Days
• Field of Search: None
• US Class Current: 382/165
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes

Abstract

Disclosed herein are a system and method for performing foreground/background separation on an input image. The method pre-classifies (1010, 1020) an input visual element in the input image as one of a first element type and a second element type, dependent upon a predetermined characteristic. The method performs a first foreground/background separation (1030) on the input visual element that has been pre-classified as the first element type, wherein the first foreground/background separation step is based on color data and brightness data of the input visual element. The method performs a second foreground/background separation (1040) on the input visual element that has been pre-classified as the second element type, wherein the second foreground/background separation step is based on color data, brightness data, and texture of the input visual element.

Claims

1. A computer-implemented method for performing foreground/background separation on an input image, said method comprising the steps of:

pre-classifying a plurality of input visual elements in said input image as at least one of a first element type and a second element type, said at least one first element type having unstable texture characteristics and said second element type having stable texture characteristics, dependent upon a predetermined characteristic;

performing a first foreground/background separation on said at least one input visual element pre-classified as the first element type, said first foreground/background separation being based on at least one of colour data and brightness data of said input visual element without texture data of said input visual element; and

performing a second foreground/background separation on said at least one input visual element pre-classified as the second element type, said second foreground/background separation being based on texture of said input visual element and at least one of colour data and brightness data of said input visual element.

2. The method according to claim 1, wherein:

(a) the input visual element having a first characteristic relating to an average of colour data and brightness data of the input visual element satisfying a first threshold is pre-classified as the first element type; and

(b) the input visual element having a second characteristic relating to texture of image data in the input visual element satisfying a second threshold is pre-classified as the second element type.

3. The method according to claim 1, wherein colour data and brightness data corresponds to the DC components of the input visual element.

4. The method according to claim 1, wherein the texture data corresponds to the AC components of the input visual element.

5. The method according to claim 1, wherein an input visual element is identified as the unstable texture region when a counter associated with the input visual element satisfies a third predetermined threshold.

6. The method according to claim 1, wherein the predetermined characteristics is a first visual distance of texture data between an input visual element and a corresponding local element model which is stored information relating to a scene and a second visual distance of at least one of colour data and brightness data between the input visual element and the corresponding local element model which is stored information relating to the scene.

7. The method according to claim 6, wherein each of the first visual distance and the second visual distance is a weighted sum of differences between a coefficient value in the input visual element and a corresponding coefficient of the element model.

8. The method according to claim 7, wherein a weight for the weighted sum used to calculate the visual distance for the input visual element pre-classified as the first element type is different from a corresponding weight for the weighted sum used to calculate the visual distance for the input visual element pre-classified as the second element type.

9. A computer-implemented method for detecting foreground in an input image, said method comprising the steps of:

pre-classifying a plurality of input visual elements in said input image as at least one of a first element type and a second element type, said at least one first element type having unstable texture characteristic and said second element type having stable texture characteristics, dependent upon a predetermined characteristic;

classifying at least one said pre-classified input visual element by applying a first classifier if said one pre-classified input visual element is of the first element type and applying a second classifier if said one pre-classified input visual element is of the second element type, the classifiers operating based on texture of said one pre-classified input visual element and at least one of colour data and brightness data of said one pre-classified input visual element; and

detecting said input visual element as foreground dependent upon said classifying step.

10. The method according to claim 9, wherein said first element type corresponds to an unstable textured visual element and said second element type corresponds to a stable textured visual element.

11. The method according to claim 9, wherein said first classifier is configured for said predetermined characteristic and said first and second classifiers are different.

12. The method according to claim 11, wherein said predetermined characteristic is unstable texture, said first classifier is configured for unstable textured visual elements, and said second classifier is configured for stable textured visual elements.

13. A non-transitory tangible computer readable storage medium having a computer program recorded thereon, the program being executable by computer apparatus to detect foreground in an input image, said computer program comprising:

code for pre-classifying a plurality of input visual elements in said input image as at least one of a first element type and a second element type, said at least one first element type having unstable texture characteristics and said second element type having stable texture characteristics, dependent upon a predetermined characteristic;

code for classifying at least one said pre-classified input visual element by applying a first classifier if said one pre-classified input visual element is of the first element type and applying a second classifier if said one pre-classified input visual element is of the second element type, the classifiers operating based on texture of said one pre-classified input visual element and at least one of colour data and brightness data of said one pre-classified input visual element; and

code for detecting said input visual element as foreground dependent upon said classifying step.

14. Apparatus for detecting foreground in an input image, said apparatus comprising:

a memory storage device storing a computer program; and

a processor for executing the computer program, said program comprising code executable by the processor for;

pre-classifying a plurality of input visual elements in said input image as at least one of a first element type and a second element type, said at least one first element type having unstable texture characteristics and said second element type having stable texture characteristics, dependent upon a predetermined characteristic;

classifying at least one said pre-classified input visual element by applying a first classifier if said one pre-classified input visual element is of the first element type and applying a second classifier if said one pre-classified input visual element is of the second element type, the classifiers being based on texture of said one pre-classified input visual element and at least one of colour data and brightness data of said one pre-classified input visual element; and

detecting said input visual element as foreground dependent upon said classifying step.

15. The apparatus according to claim 14, wherein said storage device and processor are components of one of a camera and a general purpose computer.

16. A non-transitory tangible computer-readable storage medium having recorded thereon a program, the program being executable by computerised apparatus to perform foreground/background separation on an input image, said program comprising:

code for pre-classifying a plurality of input visual elements in said input image as at least one of a first element type and a second element type, said at least one first element type having unstable texture characteristics and said second element type having stable texture characteristics, dependent upon a predetermined characteristic;

code for performing a first foreground/background separation on said at least one input visual element pre-classified as the first element type, said first foreground/background separation being based on at least one of colour data and brightness data of said input visual element without texture data of said input visual element without texture data of said input visual element; and

code for performing a second foreground/background separation on said at least one input visual element pre-classified as the second element type, said second foreground/background separation being based on texture of said input visual element and at least one of colour data and brightness data of said input visual element.

17. Computerised apparatus for performing foreground/background separation on an input image, said apparatus comprising:

a processor; and

a memory coupled to the processor and having a program recorded therein, the program being executable by the processor to perform the foreground/background separation and comprising;

code for pre-classifying a plurality of input visual elements in said input image as at least one of a first element type and a second element type, said at least one first element type having unstable texture characteristics and said second element type having stable texture characteristics, dependent upon a predetermined characteristic;

code for performing a first foreground/background separation on said at least one input visual element pre-classified as the first element type, said first foreground/background separation being based on at least one of colour data and brightness data of said input visual element without texture data of said input visual element; and

code for performing a second foreground/background separation on said at least one input visual element pre-classified as the second element type, said second foreground/background separation being based on texture of said input visual element and at least one of colour data and brightness data of said input visual element.