Image processing apparatus and image processing method

• US 9,811,888 B2
• Filed: 01/30/2015
• Issued: 11/07/2017
• Est. Priority Date: 03/14/2014
• Status: Active Grant

• Current Assignee: Omron Corporation
• Original Assignee: Omron Corporation
• Inventors: Matsunaga, Jumpei, Kobayashi, Masayuki
• Primary Examiner(s): Roz, Mark
• Application Number:
  US14/610,267
• Publication Number:
  US 20150262342A1
• Time in Patent Office: 1,012 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/20172 Image enhancement details
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611

Abstract

An image processing apparatus configured to correct at least part of a face image has an image acquisition unit that acquires an input image, a region extraction unit that extracts a pupil region, which is a region corresponding to an eye of a person, from the input image, a pixel extraction unit that extracts a pixel from the pupil region whose brightness value is higher than a threshold value, a determination unit that determines, based on the result of extraction performed by the pixel extraction unit, whether or not halation has occurred in the pupil region, and a correction unit that performs, based on the result of determination performed by the determination unit, processing for correcting halation on the pupil region. The pixel extraction unit sets the threshold value based on a bias of a brightness distribution of the pixels included in the pupil region.

Claims

1. An image processing apparatus configured to correct at least part of a face image, comprising:

an image acquisition unit that acquires an input image;

a region extraction unit that extracts a pupil region, which is a region corresponding to an eye of a person, from the input image;

a pixel extraction unit that extracts a pixel from the pupil region whose brightness value is higher than a threshold value;

a determination unit that determines, based on the result of extraction performed by the pixel extraction unit, whether or not halation has occurred in the pupil region; and

a correction unit that performs, based on the result of determination performed by the determination unit, processing for correcting halation on the pupil region,

wherein the pixel extraction unit sets the threshold value based on a bias of a brightness distribution of the pixels included in the pupil region,

wherein the threshold value is a value according to which the brightness distribution of the pixels included in the pupil region is separated into a high brightness class and a low brightness class,

wherein the determination unit calculates a degree of separation, which is a ratio between an intra-class variance and an inter-class variance in the brightness distribution of the pixels included in the pupil region and sets the threshold value to a value according to which the degree of separation is maximized, and

wherein, if the degree of separation is less than or equal to a predetermined value and the average of the brightness values of the pixels included in the pupil region is greater than or equal to the predetermined value, the correction unit performs halation correction processing regardless of the result of the determination performed by the determination unit.

2. The image processing apparatus according to claim 1,

wherein, if the pixels extracted by the pixel extraction unit make up a predetermined percentage or more of the pixels in the pupil region, the determination unit determines that halation has occurred in the pupil region.

3. The image processing apparatus according to claim 1,

wherein, if the degree of separation is less than or equal to a predetermined value and the average of the brightness values of the pixels included in the pupil region and is less than or equal to the predetermined value, the correction unit does not perform halation correction processing regardless of the result of the determination performed by the determination unit.

4. The image processing apparatus according to claim 1,

wherein the correction unit stores a pupil image corresponding to a pupil region, and

wherein, if halation occurs in a pupil region, the correction unit performs halation correction by pasting the corresponding pupil image on the target pupil region.

5. The image processing apparatus according to claim 4,

wherein the correction unit deforms the pupil image into a shape matching that of a target pupil region and thereafter pastes it on the pupil region.

6. The image processing apparatus according to claim 4,

wherein the correction unit corrects the pupil image to a tone matching that of a target pupil region and thereafter pastes it on the pupil region.

7. The image processing apparatus according to claim 1,

wherein, if halation occurs in a first pupil region and halation does not occur in a second pupil region that is a pupil region of an eye opposite to that of the first pupil region, the correction unit performs halation correction by pasting a pupil image acquired from the second pupil region on the first pupil region.

8. An image processing apparatus configured to correct halation that has occurred in a face image, the image processing apparatus comprising:

an image acquisition unit that acquires an input image;

a region extraction unit that extracts a pupil region, which is a region corresponding to an eye of a person, from the input image;

a determination unit that determines whether or not halation has occurred in the pupil region; and

a correction unit that performs, in the case where halation has occurred in the pupil region, correction by pasting a corresponding pupil image on the target pupil region,

wherein threshold value is a value according to which brightness distribution of pixels included in the pupil region is separated into a high brightness class and a low brightness class,

wherein the determination unit calculates a degree of separation, which is a ratio between an intra-class variance and an inter-class variance in the brightness distribution of the pixels included in the pupil region and sets the threshold value to a value according to which the degree of separation is maximized, and

wherein, if the degree of separation is less than or equal to a predetermined value and the average of the brightness values of the pixels included in the pupil region is greater than or equal to the predetermined value, the correction unit performs halation correction processing regardless of the result of the determination performed by the determination unit.

9. An image processing method that corrects at least part of a face image, comprising:

an image acquisition step of acquiring an input image;

a region extraction step of extracting a pupil region, which is a region corresponding to an eye of a person, from the input image;

a pixel extraction step of extracting a pixel from the pupil region whose brightness value is higher than a threshold value;

a determination step of determining whether or not halation has occurred in the pupil region based on the result of extraction performed in the pixel extraction step; and

a correction step of performing processing for correcting halation with respect to the pupil region, based on the result of determination performed in the determination step,

wherein in the determination step, the threshold value is set based on a bias of a brightness distribution of the pixels included in the pupil region,

wherein the threshold value is a value according to which the brightness distribution of the pixels included in the pupil region is separated into a high brightness class and a low brightness class,

wherein the determination step comprises calculating a degree of separation, which is a ratio between an intra-class variance and an inter-class variance in the brightness distribution of the pixels included in the pupil region and setting the threshold value to a value according to which the degree of separation is maximized, and

wherein, if the degree of separation is less than or equal to a predetermined value and the average of the brightness values of the pixels included in the pupil region is greater than or equal to the predetermined value, the correction step includes performing halation correction processing regardless of the result of the determination performed by the determination step.

10. A non-transitory computer readable storing medium recording an image processing program for causing a computer to perform a method comprising:

an image acquisition step of acquiring an input image;

a region extraction step of extracting a pupil region, which is a region corresponding to an eye of a person, from the input image;

a pixel extraction step of extracting a pixel from the pupil region whose brightness value is higher than a threshold value;

a determination step of determining whether or not halation has occurred in the pupil region based on the result of extraction performed in the pixel extraction step; and

a correction step of performing processing for correcting halation with respect to the pupil region, based on the result of determination performed in the determination step,

wherein in the determination step, the threshold value is set based on a bias of a brightness distribution of the pixels included in the pupil region, wherein the threshold value is a value according to which the brightness distribution of the pixels included in the pupil region is separated into a high brightness class and a low brightness class,

wherein determination step comprises calculating a degree of separation, which is a ratio between an intra-class variance and an inter-class variance in the brightness distribution of the pixels included in the pupil region and setting the threshold value to a value according to which the degree of separation is maximized, and

wherein, if the degree of separation is less than or equal to a predetermined value and the average of the brightness values of the pixels included in the pupil region is greater than or equal to the predetermined value, the correction step includes performing halation correction processing regardless of the result of the determination performed by the determination step.