Face detection on mobile devices

• US 20070154095A1
• Filed: 12/31/2005
• Published: 07/05/2007
• Est. Priority Date: 12/31/2005
• Status: Active Grant

• Current Assignee: ArcSoft, Inc.
• Original Assignee: ArcSoft, Inc.
• Inventors: Wang, Jin, Cao, Jiangen, Li, Shu, Wang, Guogang
• Granted Patent: US 7,953,253 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06V 10/446 using Haar-like filters, e.g. using integral image techniques
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

A method for detecting a facial area on a color image includes (a) placing a search window on the color image, (b) determining if a center pixel of the search window is a skin color pixel, indicating that the search window is a possible facial area candidate, (c) applying a 3-rectangle filter to the search window to determine if the search window is a possible facial area candidate, (d) applying a 4-rectangle filter to the search window to determine if the search window is a possible facial area candidate, (e) if steps (b), (c), (d) all determine that the search window is a possible facial area candidate, applying an AdaBoost filter to the search window to determine if the search window is a facial area candidate, and (f) if step (e) determines that the search window is a facial area candidate, saving the location of the search window.

Claims

1. A method for detecting an area with a face (“

facial area”

) on a color image, comprising;

(a) placing a search window on the color image;

(b) determining if a center pixel of the search window is a skin color pixel, wherein the search window is a possible facial area candidate if the center pixel of the search window is a skin color pixel;

(c) applying a 3-rectangle filter to the search window to determine if the search window is a possible facial area candidate;

(d) applying a 4-rectangle filter to the search window to determine if the search window is a possible facial area candidate;

(e) if steps (b), (c), (d) all determine that the search window is a possible facial area candidate, applying an AdaBoost filter to the search window to determine if the search window is a facial area candidate; and

(f) if step (e) determines that the search window is a facial area candidate, saving the location of the search window.

2. The method of claim 1, further comprising:

(g) repeating steps (a), (b), (c), (d), (e), and (f) to determine if additional locations of the search window on the color image are facial area candidates;

(h) grouping facial area candidates into groups based on their locations, each group having a group candidate;

(i) for each group, calculating a sum of minimum differences to a threshold of the AdaBoost filter of the facial area candidates in said each group; and

(j) selecting a group candidate of a group with a largest sum of minimum differences as the facial area on the image.

3. The method of claim 2, wherein said grouping the facial area candidates comprises applying an afPartitionSeq( ) function from an Open Source Computer Vision Library, and the facial area has a size of a bounding box of the group candidate.

4. The method of claim 1, wherein steps (b), (c), (d) are performed in order or in a different order.

5. The method of claim 1, prior to step (b), further comprising:

applying a skin color filter to the color image to generate a skin color mask designating pixels in the color image as skin color pixels and non-skin color pixels;

6. The method of claim 5, wherein the skin color filter comprises:

7. The method of claim 5, wherein the skin color filter comprises a boosted skin filter trained from samples.

8. The method of claim 1, wherein said applying a 3-rectangle filter comprises:

for each pixel in the search window, calculating a sum of (1) a first difference between corresponding pixel intensities in a middle rectangle and a top rectangle in the search window and (2) a second difference between corresponding pixel intensities in the middle rectangle and a bottom rectangle in the search window,

wherein the top rectangle is atop the middle rectangle and the middle rectangle is atop the bottom rectangle;

determining if an average of sums of (1) the first difference and (2) the second difference for all pixels in the search window is within a range of thresholds,

wherein the search window is a possible facial area candidate if the average of sums is within the range of threshold.

9. The method of claim 8, prior to step (c), further comprising:

creating an integral image of the color image, wherein said calculating a sum comprises using the integral image to calculate the sum.

10. The method of claim 1, wherein said applying a 4-rectangle comprises:

calculating a first sum of corresponding pixel intensities of an upper left rectangle and an upper right rectangle in the search window, and a second sum of corresponding pixel intensities of a lower left rectangle and a lower right rectangle in the search window, wherein the upper left rectangle is atop the lower left rectangle, the upper right rectangle is atop the lower right rectangle, and the upper and the lower left rectangles are spaced apart from the upper and the lower right rectangles;

determining if the second sum is greater than the first sum, wherein the search window is a possible facial area candidate if the second sum is greater than the first sum.

11. The method of claim 10, prior to step (c), further comprising:

creating an integral image of the color image, wherein said calculating a first sum and a second sum comprises using the integral image to calculate the first sum and the second sum.

12. The method of claim 2, wherein the search window is placed only at locations within a search mask of the color image, the search mask being defined as follows:

d=0.08*W,
l=0.7*W,
h=0.7*H,

where “

d”

is a distance from boundaries of the search mask to boundaries of the color image, “

l” and

“

h”

are width and height of the search mask, respectively, and “

W” and

“

H”

are width and height of the color image, respectively.

13. The method of claim 1, prior to step (b), further comprising:

applying a skin color filter to the color image to generate a skin color mask designating pixels in the color image as skin color pixels and non-skin color pixels;

generate a first histogram along an x direction to represent numbers of skin color pixels in each column of the skin color mask;

generate a second histogram along a y direction to represent numbers of skin color pixels in each row of the skin color mask;

grouping contiguous columns with less than 5% skin color pixels in a first group of sub-areas;

grouping contiguous rows with less than 5% skin color pixels in a second group of sub-areas;

segmenting the skin color mask along vertical and horizontal borders of the first and the second groups of sub-areas into sub-blocks;

determining a group of sub-blocks that have (1) both columns and rows with less than 5% skin color pixels and (2) a height to width ratio greater than a threshold; and

setting all pixels in the group of sub-blocks as non-skin pixels.

14. The method of claim 13, prior to step (b), further comprising:

enlarging remaining sub-blocks not in the group of sub-blocks.