FOREGROUND BACKGROUND SEPARATION IN A SCENE WITH UNSTABLE TEXTURES

• US 20120106837A1
• Filed: 10/25/2011
• Published: 05/03/2012
• Est. Priority Date: 10/29/2010
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Gupta, Amit Kumar, Pakulski, Peter Jan, Partis, Ashley, Lin, Qianlu
• Granted Patent: US 8,837,823 B2
• Time in Patent Office: Days
• US Class Current: 382/165
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes

Abstract

Disclosed herein are a system and method for performing foreground/background separation on an input image. The method pre-classifies (1010, 1020) an input visual element in the input image as one of a first element type and a second element type, dependent upon a predetermined characteristic. The method performs a first foreground/background separation (1030) on the input visual element that has been pre-classified as the first element type, wherein the first foreground/background separation step is based on colour data and brightness data of the input visual element. The method performs a second foreground/background separation (1040) on the input visual element that has been pre-classified as the second element type, wherein the second foreground/background separation step is based on colour data, brightness data, and texture of the input visual element.

Claims

1. A computer-implemented method for performing foreground/background separation on an input image, said method comprising the steps of:

pre-classifying an input visual element in said input image as one of a first element type and a second element type dependent upon a predetermined characteristic, said first element type having unstable texture characteristics and said second element type having stable texture characteristics;

performing a first foreground/background separation on said input visual element pre-classified as the first element type, said first foreground/background separation step based on at least one of colour data and brightness data of said input visual element without texture data of said input visual element; and

performing a second foreground/background separation on said input visual element pre-classified as the second element type, said second foreground/background separation step based on colour data, brightness data, and texture of said input visual element.

2. The method according to claim 1, wherein:

(a) the input visual element having a first characteristic relating to an average of colour data and brightness data of the input visual element satisfying a first threshold is pre-classified as the first element type; and

(b) the input visual element having a second characteristic relating to texture of image data in the input visual element satisfying a second threshold is pre-classified as the second element type.

3. The method according to claim 1, wherein colour data and brightness data corresponds to the DC components of the input visual element.

4. The method according to claim 1, wherein the texture data corresponds to the AC components of the input visual element.

5. The method according to claim 1, wherein an input visual element is identified as the unstable texture region when a counter associated with the input visual element satisfies a third predetermined threshold.

6. The method according to claim 1, wherein the detecting steps include calculating a visual distance between an input visual element and an element model of the scene.

7. The method according to claim 6, wherein the visual distance is a weighted sum of a difference between a coefficient value in the input visual element and a coefficient of the element model.

8. The method according to claim 7, wherein a weight for the weighted sum used to calculate the visual distance for the input visual element pre-classified as the first element type is different from a corresponding weight for the weighted sum used to calculate the is visual distance for the input visual element pre-classified as the second element type.

9. A computer-implemented method for detecting foreground in an input image, said method comprising the steps of:

pre-classifying an input visual element in said input image as one of a first element type and a second element type dependent upon a predetermined characteristic;

classifying said pre-classified input visual element by applying a first classifier if said pre-classified input visual element is of the first element type and applying a second classifier if said pre-classified input visual element is of the second element type; and

detecting said input visual element as foreground dependent upon said classifying step.

10. The method according to claim 9, wherein said first element type corresponds to an unstable textured visual element and said second element type corresponds to a stable textured visual element.

11. The method according to claim 9, wherein said first classifier is configured for said predetermined characteristic and said first and second classifiers are different.

12. The method according to claim 11, wherein said predetermined characteristic is unstable texture, said first classifier is configured for unstable textured visual elements, and said second classifier is configured for stable textured visual elements.

13. A computer readable storage medium having a computer program recorded thereon, the program being executable by computer apparatus to detect foreground in an input image, said computer program comprising:

code for pre-classifying an input visual element in said input image as one of a first element type and a second element type dependent upon a predetermined characteristic;

code for classifying said pre-classified input visual element by applying a first classifier if said pre-classified input visual element is of the first element type and applying a second classifier if said pre-classified input visual element is of the second element type; and

code for detecting said input visual element as foreground dependent upon said classifying step.

14. Apparatus for detecting foreground in an input image, said apparatus comprising:

a memory storage device storing a computer program; and

a processor for executing the computer program, said program comprising code executable by the processor for;

pre-classifying an input visual element in said input image as one of a first element type and a second element type dependent upon a predetermined characteristic;

classifying said pre-classified input visual element by applying a first classifier if said pre-classified input visual element is of the first element type and applying a second classifier if said pre-classified input visual element is of the second element type; and

detecting said input visual element as foreground dependent upon said classifying step.

15. The apparatus according to claim 14, wherein said storage device and processor are components of one of a camera and a general purpose computer.

16. A computer-readable storage medium having recorded thereon a program, the program being executable by computerised apparatus to perform foreground/background separation on an input image, said program comprising:

code for pre-classifying an input visual element in said input image as one of a first element type and a second element type dependent upon a predetermined characteristic, said first element type having unstable texture characteristics and said second element type having stable texture characteristics;

code for performing a first foreground/background separation on said input visual element pre-classified as the first element type, said first foreground/background separation step based on at least one of colour data and brightness data of said input visual element without texture data of said input visual element; and

code for performing a second foreground/background separation on said input visual element pre-classified as the second element type, said second foreground/background separation step based on colour data, brightness data, and texture of said input visual element.

17. Computerised apparatus for performing foreground/background separation on an input image, said apparatus comprising:

a processor; and

a memory coupled to the processor and having a program recorded therein, the program being executable by the processor to perform the foreground/background separation and comprising;

code for pre-classifying an input visual element in said input image as one of a first element type and a second element type dependent upon a predetermined characteristic, said first element type having unstable texture characteristics and said second element type having stable texture characteristics;

code for performing a first foreground/background separation on said input visual element pre-classified as the first element type, said first foreground/background separation step based on at least one of colour data and brightness data of said input visual element without texture data of said input visual element; and

code for performing a second foreground/background separation on said input visual element pre-classified as the second element type, said second foreground/background separation step based on colour data, brightness data, and texture of said input visual element.