Face detection in color images with complex background

• US 7,035,456 B2
• Filed: 05/31/2002
• Issued: 04/25/2006
• Est. Priority Date: 06/01/2001
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Lestideau, Fabrice
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): Alavi, Amir
• Application Number:
  US10/157,825
• Publication Number:
  US 20030053685A1
• Time in Patent Office: 1,425 Days
• Field of Search: 382/115, 382/118, 382/162, 382/164, 382/173, 382/190, 382/195, 382/199, 382/203, 382/224, 382/225, 382/284, 382/272, 382/276
• US Class Current: 382/164
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

A method (100) of locating human faces, if present, in a cluttered scene captured on a digital image (105) is disclosed. The method (100) relies on a two step process, the first being the detection of segments with a high probability of being human skin in the color image (105), and to then determine a bounday box, or other boundary indication, to border each of those segments. The second step (140) is the analysis of features within each of those boundary boxes to determine which of the segments are likely to be a human face. As human skin is not highly textured, in order to detect segments with a high probability of being human skin, a binary texture map (121) is formed from the image (105), and segments having high texture are discarded.

Claims

1. A method of detecting one or more human faces in a colour image, said method comprising the steps of:

(a) forming human skin coloured segments in said colour image; and

(b) determining for each segment whether said segment contains a face, said determining comprising the sub-steps of;

(b1) identifying features in said segment;

(b2) forming groups of said features; and

(b3) for each group of said features, determining whether said group is bounded by contour lines in the shape of an ellipse, wherein only segments with at least one group of features bounded by contour lines in the shape of an ellipse contain a human face.

2. A method as claimed in claim 1, wherein step (a) comprises the sub-steps of:

(a1) over-segmenting said image into a plurality of first segments, each first segment having a substantially homogenous colour;

(a2) determining, for each first segment, a probability of colours contained therein being human skin colour;

(a3) merging adjacent first segments in which said probability is above a first predetermined threshold; and

(a4) grouping the merged first segments to form said human skin coloured segments.

3. A method as claimed in claim 2, wherein step (a2) comprises the sub-steps of:

(a2i) determining for each pixel of said image a probability of said pixel having a colour of human skin; and

(a2ii) calculating a mean of said probabilities of all pixels within said segment.

4. A method as claimed in claim 1 comprising the further steps of:

(c) determining for pixels in said image a measure of texture in a region of said pixel; and

(d) eliminating segments having said measure of texture above a second predetermined value from said human skin coloured segments.

5. A method as claimed in claim 4, wherein step (c) comprises the further steps of:

(c1) converting said image into a gray scale image;

(c2) determining for said grey scale image wavelet transforms for each of a third predetermined number of levels;

(c3) for at least one level of wavelet transforms, determining for each pixel in detail sub-images of said wavelet transform a variance of brightness intensities of pixels around said pixel to form a plurality of texture maps;

(c4) resizing each said texture map to the size of said image; and

(c5) determining a maximum pixel value of corresponding pixels in said resized texture maps, said maximum pixel value being said measure of texture of said pixel.

6. An apparatus for detecting one or more human faces in a colour image, said apparatus comprising:

means for forming human skin coloured segments in said colour image; and

means for determining for each segment whether said segment contains a face, said means for determining comprising;

means for identifying features in said segment;

means for forming groups of said features; and

means for determining for each group of said features whether said group is bounded by contour lines in the shape of an ellipse, wherein only segments with at least one group of features bounded by contour lines in the shape of an ellipse contain a human face.

7. An apparatus as claimed in claim 6, wherein said means for forming human skin coloured segments comprises:

means for over-segmenting said image into a plurality of first segments, each first segment having a substantially homogenous colour;

means for determining, for each first segment, a probability of colours contained therein being human skin colour;

means for merging adjacent first segments in which said probability is above a first predetermined threshold; and

means for grouping the merged first segments to form said human skin coloured segments.

8. An apparatus as claimed in claim 7, wherein said means for determining a probability comprises:

means for determining for each pixel of said image a probability of said pixel having a colour of human skin; and

means for calculating a mean of said probabilities of all pixels within said segment.

9. An apparatus as claimed in claim 6 further comprising:

means for detecting for pixels in said image a measure of texture in a region of said pixel; and

means for eliminating segments having said measure of texture above a second predetermined value from said human skin coloured segments.

10. An apparatus as claimed in claim 9, wherein said means for determining a measure of texture comprises:

means for converting said image into a grey scale image;

means for determining for said grey scale image wavelet transforms for each of a third predetermined number of levels;

means for, for at least one level of wavelet transforms, determining for each pixel in detail sub-images of said wavelet transform a variance of brightness intensities of pixels around said pixel to form a plurality of texture maps;

means for resizing each said texture map to the size of said image; and

means for determining a maximum pixel value of corresponding pixels in said resized texture maps, said maximum pixel value being said measure of texture of said pixel.

11. A program embodied in a computer-readable memory medium for controlling the operation of a data processing apparatus upon which the program executes to perform a method of detecting one or more human faces in a colour image, said method comprising the steps of:

forming human skin coloured segments in said colour image; and

determining for each segment whether said segment contains a face, the determining step comprising the sub-steps of;

identifying features in said segment;

forming groups of said features; and

determining for each group of said features whether said group is bounded by contour lines in the shape of an ellipse, wherein only segments with at least one group of features bounded by contour lines in the shape of an ellipse contain a human face.

12. A program as claimed in claim 11, wherein the step of forming human skin coloured segments comprises the sub-steps of:

over-segmenting said image into a plurality of first segments, each first segment having a substantially homogenous colour;

determining, for each first segment, a probability of colours contained therein being human skin colour;

merging adjacent first segments in which said probability is above a first predetermined threshold; and

grouping the merged first segments to form said human skin coloured segments.

13. A program as claimed in claim 12, wherein the step of determining a probability comprises the sub-steps of:

determining for each pixel of said image a probability of said pixel having a colour of human skin; and

calculating a mean of said probabilities of all pixels within said segment.

14. A program as claimed in claim 11, wherein the method further comprises the steps of:

determining for pixels in said image a measure of texture in a region of said pixel; and

eliminating segments having said measure of texture above a second predetermined value from said human skin coloured segments.

15. A program as claimed in claim 14, wherein the step of determining a measure of texture comprises the sub-steps of:

converting said image into a grey scale image;

determining for said grey scale image wavelet transforms for each of a third predetermined number of levels;

for at least one level of wavelet transforms, determining for each pixel in detail sub-images of said wavelet transform a variance of brightness intensities of pixels around said pixel to form a plurality of texture maps;

for resizing each said texture map to the size of said image; and

for determining a maximum pixel value of corresponding pixels in said resized texture maps, said maximum pixel value being said measure of texture of said pixel.