IMAGE PROCESSING DEVICE, IMAGE PROCESSING METHOD, AND PROGRAM

• US 20110243383A1
• Filed: 03/08/2011
• Published: 10/06/2011
• Est. Priority Date: 03/30/2010
• Status: Abandoned Application

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: OYAIZU, Hideki
• Application Number:
  US13/042,527
• Publication Number:
  US 20110243383A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/248 involving reference images or patches

Abstract

An image processing device includes a reference background storage unit that stores a reference background image, an estimation unit that detects an object from an input image and estimates an approximate position and an approximate shape of the object that is detected, a background difference image generation unit that generates a background difference image obtained based on a difference value between the input image and the reference background image, a failure determination unit that determines whether a failure occurs in the background difference image based on a comparison between the background difference image that is generated by the background difference image generation unit and the object that is estimated by the estimation unit, a failure type identification unit that identifies a type of the failure, and a background image update unit that updates the reference background image in a manner to correspond to the type of the failure.

Claims

1. An image processing device, comprising:

a reference background storage means for storing a reference background image;

an estimation means for detecting an object from an input image and estimating an approximate position and an approximate shape of the object that is detected;

a background difference image generation means for generating a background difference image obtained based on a difference value between the input image and the reference background image;

a failure determination means for determining whether a failure occurs in the background difference image based on a comparison between the background difference image that is generated by the background difference image generation means and the object that is estimated by the estimation means;

a failure type identification means for identifying a type of the failure; and

a background image update means for updating the reference background image in a manner to correspond to the type of the failure.

2. The image processing device according to claim 1, wherein the failure determination means compares the object to the background difference image so as to determine whether the failure occurs based on whether a ratio of a region of the background difference image with respect to a region of the object is larger than a predetermined ratio.

3. The image processing device according to claim 1, further comprising:

a change amount calculation means for calculating a change amount between pixels, the pixels corresponding to each other, in regions excluding region of the object, the object being estimated by the estimation means, of the reference background image and the background difference image;

wherein

in a case where the change amount is larger than a predetermined value, the failure type identification means identifies a failure type as a color failure based on a color change, and in a case where the change amount is free from being larger than the predetermined value, the failure type identification means identifies the failure type as a displacement failure based on displacement of an image pickup direction of the input image.

4. The image processing device according to claim 3, further comprising:

a motion vector calculation means for comparing the input image and the reference background image so as to obtain displacement of the image pickup direction of the input image as a motion vector;

a motion compensation means for performing motion compensation with respect to the reference background image based on the motion vector so as to generate a motion compensation background image;

a calculation means for calculating a relational formula of pixel values between pixels, the pixels corresponding to each other, in the reference background image and the region excluding the region of the object, the object being estimated by the estimation means, in the background difference image; and

a conversion means for converting the pixel value of the reference background image based on the relational formula so as to generate a pixel value conversion background image;

wherein

when the failure type identified by the failure type identification means is the displacement failure, the background image update means substitutes the reference background image with the motion compensation background image so as to update the reference background image, and

when the failure type identified by the failure type identification means is the color failure, the background image update means substitutes the reference background image with the pixel value conversion background image so as to update the reference background image.

5. The image processing device according to claim 4, wherein when the failure determination means determines that there is no occurrence of a failure, the background image update means keeps the reference background image as it is.

6. The image processing device according to claim 4, wherein the motion vector calculation means compares the region excluding the region of the object in the reference background image to the region excluding the region of the object in the input image so as to obtain a motion vector by which a sum of difference absolute values between corresponding pixels of the images becomes the minimum.

7. The image processing device according to claim 1, wherein an object detection means includes a person detection means for detecting a person as an object, an animal detection means for detecting an animal as an object, and a vehicle detection means for detecting a vehicle as an object.

8. The image processing device according to claim 7, wherein the person detection means includes a face detection means for detecting a face image of the person from the input image, and a body mask estimation means for estimating a body mask from a position and a size in which a body of the person, the body of the person being estimated based on the face image that is detected by the face detection means, exists.

9. An image processing method of an image processing device, the image processing device including,

a reference background storage means for storing a reference background image,

an estimation means for detecting an object from an input image and estimating an approximate position and an approximate shape of the object that is detected,

a background difference image generation means for generating a background difference image obtained based on a difference value between the input image and the reference background image,

a failure determination means for determining whether a failure occurs in the background difference image based on a comparison between the background difference image that is generated by the background difference image generation means and the object that is estimated by the estimation means,

a failure type identification means for identifying a type of the failure, and

a background image update means for updating the reference background image in a manner to correspond to the type of the failure, the image processing method comprising the steps of:

storing the reference background image, in the reference background storage means;

detecting the object from the input image and estimating the approximate position and the approximate shape of the object that is detected, in the estimation means;

generating the background difference image based on the difference value between the input image and the reference background image, in the background difference image generation means;

determining whether a failure occurs in the background difference image based on the comparison between the background difference image that is generated through processing of the step of generating a background difference image and the object that is estimated through processing of the step of estimating, in the failure determination means;

identifying a type of the failure, in the failure type identification means; and

updating the reference background image in a manner to correspond to the type of the failure, in the background image update means.

10. A program allowing a computer that controls an image processing device, the image processing device including

a reference background storage means for storing a reference background image,

an estimation means for detecting an object from an input image and estimating an approximate position and an approximate shape of the object that is detected,

a background difference image generation means for generating a background difference image obtained based on a difference value between the input image and the reference background image,

a failure determination means for determining whether a failure occurs in the background difference image based on a comparison between the background difference image that is generated by the background difference image generation means and the object that is estimated by the estimation means,

a failure type identification means for identifying a type of the failure, and

a background image update means for updating the reference background image in a manner to correspond to the type of the failure, the program allowing the computer to execute processing comprising the steps of:

storing the reference background image, in the reference background storage means;

detecting the object from the input image and estimating the approximate position and the approximate shape of the object that is detected, in the estimation means;

generating the background difference image based on the difference value between the input image and the reference background image, in the background difference image generation means;

determining whether a failure occurs in the background difference image based on the comparison between the background difference image that is generated through processing of the step of generating a background difference image and the object that is estimated through processing of the step of estimating, in the failure determination means;

identifying a type of the failure, in the failure type identification means; and

updating the reference background image in a manner to correspond to the type of the failure, in the background image update means.

11. An image processing device, comprising:

a reference background storage unit configured to store a reference background image;

an estimation unit configured to detect an object from an input image and estimate an approximate position and an approximate shape of the object that is detected;

a background difference image generation unit configured to generate a background difference image obtained based on a difference value between the input image and the reference background image;

a failure determination unit configured to determine whether a failure occurs in the background difference image based on a comparison between the background difference image that is generated by the background difference image generation unit and the object that is estimated by the estimation unit;

a failure type identification unit configured to identify a type of the failure; and

a background image update unit configured to update the reference background image in a manner to correspond to the type of the failure.