Face Detector Training Method, Face Detection Method, and Apparatuses

• US 20150235074A1
• Filed: 02/16/2015
• Published: 08/20/2015
• Est. Priority Date: 02/17/2014
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Zhang, Yu, Chai, Zhenhua, Du, Zhijun
• Granted Patent: US 9,836,640 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 10/7747 Organisation of the process, e.g. bagging or boosting
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A face detector training method, a face detection method, and apparatuses are provided. In the present invention, during a training phase, a flexible block based local binary pattern feature and a corresponding second classifier are constructed, appropriate second classifiers are searched for to generate multiple first classifiers, and multiple layers of first classifiers that are obtained by using a cascading method form a final face detector; and during a detection phase, face detection is performed on a to-be-detected image by using a first classifier or a face detector that is learned during a training process, so that a face is differentiated from a non-face, and a face detection result is combined and output.

Claims

1. A face detector training method, comprising:

performing a training process by;

collecting face and non-face images as a training image sample set;

extracting a flexible block based local binary pattern (FBLBP) feature of the face and non-face images to form an FBLBP feature set; and

using the FBLBP feature and a GentleBoost algorithm to perform training, to obtain a first classifier, wherein the first classifier comprises several optimal second classifiers, and each optimal second classifier is obtained by training by using the GentleBoost algorithm;

repeating the training process to obtain multiple layers of first classifiers; and

cascading the multiple layers of first classifiers to form a face detector.

2. The method according to claim 1, wherein collecting the face and the non-face images as the training image sample set and extracting the FBLBP feature of the face and the non-face images to form the FBLBP feature set comprises:

constructing the FBLBP feature to represent co-occurrence information of a relative change of local grayscale of an image, wherein the FBLBP feature comprises several congruent rectangular blocks, a threshold, and a sign bit, wherein the several congruent rectangular blocks comprise one pivot block and at least one neighbor block, wherein the threshold is θ

₀ when the sign bit is −

1, wherein the threshold is θ

₁ when the sign bit is 1, and wherein θ

₀ and θ

1 are real numbers; and

calculating a sketch value AvgInt_pivotblock of the pivot block in the FBLBP feature and a sketch value AvgInt_{neighborblockk} of each of the neighbor blocks in the FBLBP feature, wherein the sketch value AvgInt_pivotblock of the pivot block is represented by average grayscale of an area in which the pivot block is located, the sketch value AvgInt_{neighborblockk} of each of the neighbor blocks is represented by average grayscale of an area in which each of the neighbor blocks is located separately, and k is a sequence number of the k^th neighbor block;

calculating a differential result according to the sign bit, and comparing and quantizing the differential result and the threshold for each of the neighbor blocks;

converting a binary number that consists of a quantization result of each of the neighbor blocks into a decimal number; and

saving the decimal number to obtain a sketch value of the FBLBP feature.

3. The method according to claim 1, wherein in using the FBLBP feature and the GentleBoost algorithm to perform training, to obtain the first classifier, wherein the first classifier comprises the several optimal second classifiers, and each optimal second classifier is obtained by training using the GentleBoost algorithm, a calculation formula (1) of a second classifier is as follows:

4. The method according to claim 3, wherein the method further comprises obtaining, by calculation, the weight ω

_i of the i^th training image sample according to a formula (3) and a normalization formula (4), wherein;

the formula (3) is ω

_i=ω

_i×

e<sup>−

yifm(xi)</sup>, and the formula (4) is

5. The method according to claim 3, wherein the method further comprises obtaining, by calculation, a sketch value x of an FBLBP feature of each second classifier according to a formula (5), wherein the formula (5) is as follows:

6. The method according to claim 1, wherein in using the FBLBP feature and the GentleBoost algorithm to perform training to obtain the first classifier, wherein the first classifier comprises the several optimal second classifiers, a process of calculating the optimal second classifiers comprises the following substeps:

initially, the FBLBP feature comprises only one pivot block and one neighbor block, wherein an FBLBP feature set of the FBLBP feature that consists of the two rectangular blocks may be obtained by using Brute force and traversal, for each FBLBP feature in the FBLBP feature set, calculating a sketch value of the FBLBP feature and an output value of a corresponding second classifier, substituting the obtained output value of the second classifier into a formula (6) to obtain an error J of the second classifier, and selecting a second classifier having a smallest value of the error J as the optimal second classifier, wherein the formula (6) is as follows;

7. The method according to claim 6, further comprising:

determining a threshold of the first classifier according to a formula (8), wherein the formula (8) is as follows;

8. The method according to claim 2, wherein for each of the neighbor blocks, calculating the differential result according to the sign bit, comparing and quantizing the differential result and the threshold, converting the binary number that consists of the quantization result of each of the neighbor blocks into the decimal number, and saving the decimal number to obtain the sketch value of the FBLBP feature comprises, for each neighbor block, when the sign bit is 1 and a difference between the sketch value AvgInt_pivotblock of the pivot block and a sketch value AvgInt_{neighborblockk} of the k^th neighbor block is not less than the threshold θ

₁, labeling a position of the neighbor block as 1, and otherwise, when the difference between the sketch value AvgInt_pivotblock of the pivot block and the sketch value AvgInt_{neighborblockk} of the k^th neighbor block is less than the threshold θ

₁, labeling the position of the neighbor block as 0, or for each of the neighbor blocks, when an opposite number of a difference between the sketch value AvgInt_pivotblock of the pivot block and a sketch value AvgInt_{neighborblockk} of the k^th neighbor block is not less than the threshold θ

₀, labeling a position of the neighbor block as 1, and otherwise, when the opposite number of the difference between the sketch value AvgInt_pivotblock of the pivot block and the sketch value AvgInt_{neighborblockk} of the k^th neighbor block is less than the threshold θ

₀, labeling the position of the neighbor block as 0.

9. A method for performing face detection by using a face detector, comprising:

traversing a to-be-detected image to obtain a to-be-detected subimage set;

inputting each to-be-detected subimage in the to-be-detected subimage set into the face detector;

calculating, layer by layer, output of a first classifier at each layer in the face detector;

for a to-be-detected subimage, considering that the to-be-detected subimage is a non-face when output of a first classifier at any layer of the face detector is less than a threshold that is of the first classifier and is obtained by training, wherein a subimage to be detected is considered as a face only when a first classifier at all layers determines that the subimage to be detected is a face;

combining all detection results; and

outputting a position of a face in the to-be-detected image.

10. The method according to claim 9, wherein traversing the to-be-detected image to obtain the to-be-detected subimage set comprises:

traversing the to-be-detected image in a detection window with a preset length-width ratio;

enlarging the detection window by the preset length-width ratio according to a preset step to traverse the to-be-detected image, repeating this operation to traverse the to-be-detected image in a different detection window until length of the detection window is greater than or equal to length of the to-be-detected image or until width of the detection window is greater than or equal to width of the to-be-detected image; and

performing normalization processing on subimages that are obtained by traversing the to-be-detected image such that a length-width ratio of each subimage conforms to the preset length-width ratio, and using a set that consists of all normalized subimages as the to-be-detected subimage set.

11. A face detector training apparatus, comprising:

a flexible block based local binary pattern (FBLBP) feature set module configured to collect face and non-face images as a training image sample set, and extract an FBLBP feature of the face and non-face images to form an FBLBP feature set;

a first classifier module configured to perform training, by using the FBLBP feature that is collected by the FBLBP feature set module and by using a GentleBoost algorithm, to obtain a first classifier, wherein the first classifier comprises several optimal second classifiers, and each optimal second classifier is obtained by training by using the GentleBoost algorithm; and

a face detector module configured to repeat a training process of the FBLBP feature set module and the first classifier module to obtain multiple layers of first classifiers, and cascade the multiple layers of first classifiers to form a face detector.

12. The apparatus according to claim 11, wherein the FBLBP feature set module comprises:

a constructing unit configured to construct the FBLBP feature to represent co-occurrence information of a relative change of local grayscale of an image, wherein the FBLBP feature comprises several congruent rectangular blocks, a threshold, and a sign bit, wherein the several congruent rectangular blocks comprise one pivot block and at least one neighbor block, and when the sign bit is −

1, the threshold is θ

₀, and when the sign bit is 1, the threshold is θ

₁, wherein θ

₀ and θ

₁ are real numbers; and

a calculating unit configured to calculate a sketch value AvgInt_pivotblock of the pivot block in the FBLBP feature that is constructed by the constructing unit and a sketch value AvgInt_{neighborblockk} of each of the neighbor block in the FBLBP feature that is constructed by the constructing unit, wherein the sketch value AvgInt_pivotblock of the pivot block is represented by average grayscale of an area in which the pivot block is located, the sketch value AvgInt_{neighborblockk} of each of the neighbor blocks is represented by average grayscale of an area in which each of the neighbor blocks is located separately, and k is a sequence number of the k^th neighbor block, for each of the neighbor blocks, calculate a differential result according to the sign bit, and compare and quantize the differential result and the threshold, and convert a binary number that consists of a quantization result of each of the neighbor blocks into a decimal number, and save the decimal number, to obtain a sketch value of the FBLBP feature.

13. The apparatus according to claim 11, wherein the first classifier module is configured to use the FBLBP feature and the GentleBoost algorithm to perform training;

to obtain the first classifier, wherein the first classifier comprises several optimal second classifiers, and each optimal second classifier is obtained by training by using the GentleBoost algorithm, wherein a calculation formula (1) of a second classifier is as follows;

14. The apparatus according to claim 13, wherein the first classifier module is configured to obtain, by calculation, the weight ω

_i of the i^th training image sample according to a formula (3) and a normalization formula (4), wherein the formula (3) is ω

_i=ω

_i×

e<sup>−

yifm(xi)</sup>, and the formula (4) is

15. The apparatus according to claim 13, wherein the first classifier module is configured to obtain, by calculation, a sketch value x of an FBLBP feature of each second classifier in the formula (1) according to a formula (5), wherein, the formula (5) is as follows:

16. The apparatus according to claim 11, wherein the first classifier comprises:

an initial calculation module configured to initially, the FBLBP feature comprises only one pivot block and one neighbor block, wherein an FBLBP feature set of the FBLBP feature that consists of the two rectangular blocks may be obtained by using Brute force and traversal, for each FBLBP feature in the FBLBP feature set, calculate a sketch value of the FBLBP feature and an output value of a corresponding second classifier, substitute the obtained output value of the second classifier into a formula (6) to obtain an error J of the second classifier, and select a second classifier having a smallest value of the error J as the optimal second classifier, wherein the formula (6) is as follows;

17. The apparatus according to claim 16, wherein the apparatus further comprises:

a threshold determining module configured to determine a threshold of the first classifier according to a formula (8), wherein the formula (8) is as follows;

18. The apparatus according to claim 12, wherein the calculating unit is configured to, for each neighbor block, when the sign bit is 1 and a difference between the sketch value AvgInt_pivotblock of the pivot block and a sketch value AvgInt_{neighborblockk} of the k^th neighbor block is not less than the threshold θ

₁, label a position of the neighbor block as 1, and otherwise, when the difference between the sketch value AvgInt_pivotblock of the pivot block and the sketch value AvgInt_{neighborblockk} of the k^th neighbor block is less than the threshold θ

₁, label the position of the neighbor block as 0, or for each of the neighbor blocks, when an opposite number of a difference between the sketch value AvgInt_pivotblock of the pivot block and a sketch value AvgInt_{neighborblockk} of the k^th neighbor block is not less than the threshold θ

₀, label a position of the neighbor block as 1, and otherwise, when the opposite number of the difference between the sketch value AvgInt_pivotblock of the pivot block and the sketch value AvgInt_{neighborblockk} of the k^th neighbor block is less than the threshold θ

₀, label the position of the neighbor block as 0.

19. A face detection apparatus, comprising:

a traversing module configured to traverse a to-be-detected image to obtain a to-be-detected subimage set;

a calculating module configured to bring each to-be-detected subimage in the to-be-detected subimage set that is obtained by traversing by the traversing module into the face detector, and calculate, layer by layer, output of a first classifier at each layer in the face detector;

a determining module configured to, for each to-be-detected subimage, determine the output that is of the first classifier at each layer and is calculated by the calculating module, and consider that the to-be-detected subimage is a non-face when output of a first classifier at any layer of the face detector is less than a threshold that is of the first classifier and is obtained by training, wherein only a to-be-detected subimage that passes determining of classifiers at all layers is considered as a face; and

a combining module configured to combine all detection results that are obtained by the determining module, and output a position of a face in the to-be-detected image.

20. The apparatus according to claim 19, wherein:

the traversing module is configured to traverse the to-be-detected image in a detection window with a preset length-width ratio, enlarge the detection window by the preset length-width ratio according to a preset step to traverse the to-be-detected image, and repeat this operation to traverse the to-be-detected image in a different detection window until length of the detection window is greater than or equal to length of the to-be-detected image or until width of the detection window is greater than or equal to width of the to-be-detected image, and perform normalization processing on subimages that are obtained by traversing the to-be-detected image as described above such that a length-width ratio of each subimage conforms to the preset length-width ratio, and use a set that consists of all normalized subimages as the to-be-detected subimage set.