Method, system and non-transitory computer storage medium for face detection

• US 9,530,045 B2
• Filed: 09/25/2014
• Issued: 12/27/2016
• Est. Priority Date: 03/26/2012
• Status: Active Grant

• Current Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Original Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Inventors: Xie, Xiaojing, Wang, Hui
• Primary Examiner(s): Fitzpatrick, Atiba O
• Application Number:
  US14/497,191
• Publication Number:
  US 20150016687A1
• Time in Patent Office: 824 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/169 Holistic features and representations, i.e. based on the facial image taken as a whole
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

In a face detection method, preprocess an image, and extract corners from the preprocessed image. Then, filter and combine the corners to obtain a connected component for the corners. Extract a centroid from the connected component of the corners, and match the centroid with a facial template. Then, calculate a matching probability of the centroid with the facial template, and identify a region formed by centroids having a matching probability greater than or equal to a predetermined value as a candidate face region. With the method described above, the accuracy and efficiency of face detection can be improved. In addition, the present invention provides a face detection system and a computer storage medium.

Claims

1. A method for detecting a face, comprising:

preprocessing an image;

extracting corners from the preprocessed image, and filtering and combining the corners, to obtain a connected component of the corners;

extracting centroids from the connected component of the corners; and

matching the centroids with a facial template, calculating a matching probability of the centroids with the facial template, and identifying regions formed by centroids having a matching probability greater than or equal to a predetermined value as candidate face regions;

wherein extracting the centroids from the connected component of the corners comprises;

selecting a connected component having an area greater than or equal to a third threshold and/or having an aspect ratio within a preset range;

extracting a center of the selected connected component as a centroid; and

calculating a direction of the extracted centroid, and removing the centroid when the perpendicularity of the direction is within a set perpendicularity range.

2. The method according to claim 1, wherein the preprocessing comprises at least one of image level adjustment, auto white balance, normalization and image mosaic.

3. The method according to claim 1, wherein extracting the corners from the preprocessed image comprises:

calculating luminance differences between a current pixel and surrounding pixels based on a predefined 3×

3 template; and

deriving a pixel having a luminance difference greater than or equal to a first threshold as the corner;

wherein the 3×

3 template is a region centered on the current pixel and including pixels to the left, right, upper, lower, top-left, top-right, bottom-left and bottom-right of the current pixel.

4. The method according to claim 1, wherein filtering the corners comprises:

identifying skin color points in the preprocessed image, and filtering out the corners which do not include a skin color point in a preset surrounding range; and

extracting the center of the intersection of a YcgCr color space and a YcbCr color space as a skin color center, calculating values of components Cb, Cg and Cr of the corners, calculating distances between the skin color center and the values of the components Cb, Cg and Cr of the corners, and filtering out the corners having a distance greater than a second threshold from the skin color center.

5. The method according to claim 1, wherein the facial template is a rectangular template including a left eye vertex, a right eye vertex, and at least one third point located on another line parallel with a line where the left eye vertex and the right eye vertex are located.

6. The method according to claim 5, wherein matching the centroids with the facial template, calculating the matching probability of the centroids with the facial template, and identifying the region formed by the centroids having a matching probability greater than or equal to the predetermined value as the candidate face region comprises:

traversing the centroids; and

for each of the centroids, performing the following steps;

searching a second centroid having a distance to the right eye vertex less than or equal to a fourth threshold using a first centroid as a vertex of the facial template;

searching a third centroid having a vertical distance less than or equal to the fourth threshold from another line parallel with a line where the left eye vertex and the right eye vertex are located;

calculating the matching probability based on the distance between the second centroid and the right eye vertex, the vertical distance between the third centroid and the another line, and the shortest distance between the third centroid and the third point; and

determining whether the matching probability is greater than or equal to the predetermined value, and identifying a region formed by the first centroid, the second centroid and the third centroid as the candidate face region if the matching probability is greater than or equal to the predetermined value.

7. The method according to claim 1, further comprising:

after matching the centroids with the facial template, calculating the matching probability of the centroids with the facial template, and identifying the region formed by the centroids having a matching probability greater than or equal to the predetermined value as the candidate face region,

dividing the candidate face region into a set amount of grids, and calculating a skin color proportion of each of the grids; and

selecting a candidate face region which has a skin color proportion meeting a preset skin color proportional distribution as a final face region.

8. A system for detecting a face, comprising:

memory, and one or more processors;

the memory stores one or more program modules configured for execution by the one or more processors;

wherein the one or more program modules comprise;

a preprocessing module, configured to preprocess an image;

a corner extracting module, configured to extract corners from the preprocessed image;

a corner filtering and combining module, configured to filter and combine the corners to obtain a connected component for the corners;

a centroid extracting module, configured to extract a centroid from the connected component of the corners; and

a candidate face region determining module, configured to match the centroids with a facial template, calculate a matching probability of the centroids with the facial template, and identify regions formed by the centroids having a matching probability greater than or equal to a predetermined value as candidate face regions;

wherein the centroid extracting module comprises;

a connected component selecting unit, configured to select a connected component having an area greater than or equal to a third threshold and/or having an aspect ratio within a preset range;

a centroid extracting unit, configured to extract a center of the selected connected component as a centroid;

a centroid removing unit, configured to calculate a direction of the extracted centroid, and remove the centroid when the direction has a perpendicularity within a set perpendicularity range.

9. The system according to claim 8, wherein the preprocessing comprises at least one of image level adjustment, auto white balance, normalization and image mosaicing.

10. The system according to claim 8, wherein the corner extracting module is configured to calculate a luminance difference between a current pixel and surrounding pixels based on a predefined 3×

3 template, and extract the pixel having the luminance difference greater than or equal to a first threshold as the corner;

wherein the 3×

3 template is a region having the current pixel as a center and including pixels to the left, right, upper, lower, top-left, top-right, bottom-left and bottom-right of the current pixel.

11. The system according to claim 8, wherein the corner filtering and combining module is configured to identify skin color points in the preprocessed image, and filter out the corners which do not include a skin color point in a preset surrounding range;

and

the corner filtering and combining module is further configured to extract the center of the intersection of a YcgCr color space and a YcbCr color space as a skin color center, calculate values of components Cb, Cg and Cr of the corners, calculate distances between the values of the components Cb, Cg and Cr of the corners and the skin color center, and filter out corners having a distance from the skin color center greater than a second threshold.

12. The system according to claim 8, wherein the facial template is a rectangular template including a left eye vertex, a right eye vertex, and at least one third point located on another line parallel with a line where the left eye vertex and the right eye vertex are located.

13. The system according to claim 12, wherein the candidate face region determining module comprises:

a searching unit, configured to take each centroid as a first centroid, search a second centroid having a distance less than or equal to a fourth threshold from the right eye vertex using the first centroid to act as a vertex of the facial template, and search a third centroid having a vertical distance less than or equal to the fourth threshold from another line parallel with a line of the left eye vertex and the right eye vertex;

a matching probability calculating unit, configured to calculate the matching probability based on the distance between the second centroid and the right eye vertex, the vertical distance between the third centroid and the another line, and the shortest distance between the third centroid and the third point; and

a region locating unit, configured to determine whether the matching probability is greater than or equal to the predetermined value, and locate regions formed by the first centroid, the second centroid and the third centroid as the candidate face regions if the matching probability is greater than or equal to the predetermined value.

14. The system according to claim 8, further comprising:

a region selecting module, configured to divide the candidate face regions into a set amount of grids, calculate a skin color proportion of each of the grids, and select a candidate face region which has a skin color proportion meeting a skin color proportional distribution as a final face region.

15. A non-transitory computer storage medium comprising computer executable instructions for performing a method for detecting a face, wherein the method comprises:

preprocessing an image;

extracting corners from the preprocessed image, and filtering and combining the corners, to obtain a connected component of the corners;

extracting centroids from the connected component of the corners; and

matching the centroids with a facial template, calculating a matching probability of the centroids with the facial template, and identifying regions formed by centroids having a matching probability greater than or equal to a predetermined value as candidate face regions;

wherein extracting the centroids from the connected component of the corners comprises;

selecting a connected component having an area greater than or equal to a third threshold and/or having an aspect ratio within a preset range;

extracting a center of the selected connected component as a centroid; and

calculating a direction of the extracted centroid, and removing the centroid when the perpendicularity of the direction is within a set perpendicularity range.

16. The non-transitory computer storage medium according to claim 15, wherein the preprocessing comprises at least one of image level adjustment, auto white balance, normalization and image mosaicing.

17. The non-transitory computer storage medium according to claim 15, wherein extracting the corners from the preprocessed image comprises:

calculating luminance differences between a current pixel and surrounding pixels based on a predefined 3×

3 template; and

deriving a pixel having a luminance difference greater than or equal to a first threshold as the corner;

wherein the 3×

3 template is a region centered on the current pixel and including pixels to the left, right, upper, lower, top-left, top-right, bottom-left and bottom-right of the current pixel.

18. The non-transitory computer storage medium according to claim 15, wherein filtering the corners comprises:

identifying skin color points in the preprocessed image, and filtering out the corners which do not include a skin color point in a preset surrounding range; and

extracting the center of the intersection of a YcgCr color space and a YcbCr color space as a skin color center, calculating values of components Cb, Cg and Cr of the corners, calculating distances between the skin color center and the values of the components Cb, Cg and Cr of the corners, and filtering out the corners having a distance greater than a second threshold from the skin color center.

19. The non-transitory computer storage medium according to claim 15, wherein the facial template is a rectangular template including a left eye vertex, a right eye vertex, and at least one third point located on another line parallel with a line where the left eye vertex and the right eye vertex are located.

20. The non-transitory computer storage medium according to claim 19, wherein matching the centroids with the facial template, calculating the matching probability of the centroids with the facial template, and identifying the region formed by the centroids having a matching probability greater than or equal to the predetermined value as the candidate face region comprises:

traversing the centroids; and

for each of the centroids, performing the following steps;

searching a second centroid having a distance to the right eye vertex less than or equal to a fourth threshold using a first centroid as a vertex of the facial template;

searching a third centroid having a vertical distance less than or equal to the fourth threshold from another line parallel with a line where the left eye vertex and the right eye vertex are located;

calculating the matching probability based on the distance between the second centroid and the right eye vertex, the vertical distance between the third centroid and the another line, and the shortest distance between the third centroid and the third point; and

determining whether the matching probability is greater than or equal to the predetermined value, and identifying a region formed by the first centroid, the second centroid and the third centroid as the candidate face region if the matching probability is greater than or equal to the predetermined value.

21. The non-transitory computer storage medium according to claim 15, further comprising:

after matching the centroids with the facial template, calculating the matching probability of the centroids with the facial template, and identifying the region formed by the centroids having a matching probability greater than or equal to the predetermined value as the candidate face region,

dividing the candidate face region into a set amount of grids, and calculating a skin color proportion of each of the grids; and

selecting a candidate face region which has a skin color proportion meeting a preset skin color proportional distribution as a final face region.