Detecting human faces and detecting red eyes

• US 20050169520A1
• Filed: 12/29/2004
• Published: 08/04/2005
• Est. Priority Date: 12/29/2003
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Zhang, Yue, Wang, Jianmin, Chen, Xinwu
• Granted Patent: US 7,376,270 B2
• Time in Patent Office: Days
• US Class Current: 382/165
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

The present invention provides a method and apparatus for detecting a red eye in an image. The method comprises steps of: calculating a skin color reference vector for said image; identifying a candidate for red eye region in said image; identifying skin color pixels in said candidate for red eye region; removing skin color pixels from said candidate for red eye region; deciding whether said candidate for red eye region is a red eye or not. The present invention also provides a method and apparatus for detecting a human face in an image. Moreover, the present invention provides corresponding storage medium including instructions for causing a processor to implement the methods. According to the present invention, red eyes and/or human faces can be detected with consideration of skin color pixels. The speed and accuracy of detecting red eyes and human faces are greatly increased.

Claims

1. A method of detecting a red eye in an image, characterized by comprising steps of:

calculating a skin color reference vector for said image;

identifying a candidate for red eye region in said image;

identifying skin color pixels in said candidate for red eye region;

removing skin color pixels from said candidate for red eye region;

deciding whether said candidate for red eye region is a red eye or not.

2. The method according to claim 1, characterized in that said step of calculating a skin color reference vector comprises steps of:

identifying a face rectangle in said image;

identifying a skin color reference portion in said face rectangle; and

calculating said skin color reference vector based on color values of pixels included in said skin color reference portion.

3. The method according to claim 2, characterized in that:

said skin color reference portion is an area between two eyes in said face rectangle;

said skin color reference vector comprises a R value, a G value and a B value, wherein

said R value is an average red color value for pixels included in said skin color reference portion,

said G value is an average green color value for pixels included in said skin color reference portion, and

said B value is an average blue color value for pixels included in said skin color reference portion.

4. The method according to claim 3, characterized in that said step of identifying skin color pixels in said candidate for red eye region comprises steps of:

calculating a distance between said skin color reference vector and a color vector for each pixel included in said candidate for red eye region; and

identifying a pixel as a skin color pixel if the distance of said pixel and the distances of pixels in the neighborhood of said pixel are all less than a first threshold.

5. The method according to claim 4, characterized in that said distance of a pixel is calculated as (x−

{overscore (x)})^TΣ

<sup>−

1</sup>(x−

{overscore (x)}), wherein x represents a color vector of said pixel, {overscore (x)} represents said skin color reference vector, and Σ

represents a covariance matrix for pixels in said skin color reference portion; and

that said first threshold ranges from 5 to 20.

8. A storage medium encoded with machine-readable computer program code for detecting a red eye in an image, the storage medium including instructions for causing a processor to implement the method according to claim 1.

6. An apparatus for detecting a red eye in an image, characterized by comprising:

a calculator adapted to calculate a skin color reference vector for said image;

a candidate identifier circuit adapted to identify a candidate for red eye region in said image;

a candidate processor adapted to identify skin color pixels in said candidate for red eye region, and to remove said skin color pixels from said candidate for red eye region; and

a deciding unit adapted to decide whether said candidate for red eye region output by said candidate processor is a red eye or not.

7. The apparatus according to claim 6, wherein said calculator comprises:

a face rectangle identifier circuit adapted to identify a face rectangle in said image;

means for identifying a skin color reference portion in said face rectangle; and

a vector calculator adapted to calculate said skin color reference vector based on color values of pixels included in said skin color reference portion.

9. A method for detecting a human face in an image, characterized by comprising steps of:

obtaining samples of skin colors of a face candidate;

calculating relative distances, from the color of said samples, of the pixels in said face candidate;

calculating a feature vector for said face candidate based on said relative distances; and

analyzing said face candidate to determine whether it is a human face based on said feature vector.

10. The method according to claim 9, characterized in that:

said samples are obtained from the RGB values of the pixels in a given area of said face candidate;

said relative distances are Mahalanobis distances calculated based on the following equation;

d=(c−

{overscore (c)})^TΣ

_c<sup>−

1</sup>(c−

{overscore (c)})

where d denotes a Mahalanobis distance for a pixel;

{overscore (c)} an average RGB vector consisting of the average RGB values {overscore (c)}_r, {overscore (c)}_g and {overscore (c)}_b of the pixels in the given area;

c a RGB vector consisting of the obtained values C_r, C_g and C_b of the pixel, Σ

_c<sup>−

1</sup> the contrary matrix of the covariance matrix of the pixels in the given area, and “

T”

denotes transformation of a vector or matrix.

11. The method according to claim 9, characterized in that said step of constituting said feature vector comprises the steps of:

dividing said face candidate into m*n blocks;

calculating the average Mahalanobis distances of all the blocks;

calculating the mean values, variance and covariance of the coordinates of the pixels having Mahalanobis distances less than a first threshold; and

constituting said feature vector comprising the average Mahalanobis distances of the m*n blocks as well as the mean values, variance and covariance.

12. The method according to claim 9, characterized in that said step of analyzing said face candidate comprises the steps of:

calculating the inner product of said feature vector and a given weight vector; and

comparing the inner product with a second threshold to determine whether or not the face candidate is a human face.

14. A storage medium encoded with machine-readable computer program code for detecting a red eye in an image, the storage medium including instructions for causing a processor to implement the method according to claim 9.

13. An apparatus for detecting a human face in an image, characterized in that said apparatus comprises:

an obtaining unit adapted to obtain samples of skin colors of a face candidate;

a relative distance calculator adapted to calculate relative distances, from the color of the samples, of the pixels in the face candidate;

a feature vector calculator adapted to calculate a feature vector for the face candidate based on the relative distances; and

an analyzing unit adapted to analyze the face candidate to determine whether or not it is a human face based on said feature vector.