Method and apparatus for recognizing face

• US 10,685,215 B2
• Filed: 09/14/2018
• Issued: 06/16/2020
• Est. Priority Date: 11/24/2017
• Status: Active Grant

• Current Assignee: Baidu Online Network Technology (Beijing) Co., Ltd (Baidu Incorporated)
• Original Assignee: Baidu Online Network Technology (Beijing) Co., Ltd (Baidu Incorporated)
• Inventors: Zhou, Dingfu, Yang, Ruigang, Zhang, Yanfu, Hong, Zhibin
• Primary Examiner(s): Couso, Yon J
• Application Number:
  US16/131,308
• Publication Number:
  US 20190163959A1
• Time in Patent Office: 641 Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2414 Smoothing the distance, e.g. radial basis function networks [RBFN]
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: H04N 23/21 from near infrared [NIR] radiation
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Embodiments of the present disclosure disclose a method and apparatus for recognizing a face. A specific embodiment of the method includes: acquiring at least two facial images of a to-be-recognized face under different illuminations using a near-infrared photographing device; generating at least one difference image based on a brightness difference between each two of the at least two facial images; determining a facial contour image of the to-be-recognized, face based on the at least one difference image; inputting the at least two facial images, the at least one difference image, and the facial contour image into a pre-trained real face prediction value calculation model to obtain a real face prediction value of the to-be-recognized face; and outputting prompt information for indicating successful recognition of a real face, in response to determining the obtained real face prediction value being greater than a preset threshold.

Claims

1. A method for recognizing a face, the method comprising:

acquiring at least two facial images of a to-be-recognized face under different illuminations using a near-infrared photographing device;

generating at least one difference image based on a brightness difference between each two of the at least two facial images;

determining a facial contour image of the to-be-recognized face based on the at least one difference image;

inputting the at least two facial images, the at least one difference image, and the facial contour image into a pre-trained real face prediction value calculation model to obtain a real face prediction value of the to-be-recognized face, the real face prediction value calculation model being used to represent a corresponding relationship between the at least two facial images, the at least one difference image, the facial contour image and the real face prediction value; and

outputting prompt information for indicating successful recognition of a real face, in response to determining the obtained real face prediction value being greater than a preset threshold.

2. The method according to claim 1, wherein the near-infrared photographing device comprises an illumination lamp;

and

the acquiring at least two facial images of a to-be-recognized face under different illuminations using a near-infrared photographing device, comprises;

acquiring respectively a first facial image of the to-be-recognized face under no illumination of the illumination lamp, a second facial image when the illumination lamp illuminates a first side of the to-be-recognized face, and a third facial image when the illumination lamp illuminates a second side of the to-be-recognized face.

3. The method according to claim 2, wherein the generating at least one difference image based on a brightness difference between each two of the at least two facial images, comprises:

generating a first difference image based on a brightness difference between the first facial image and the second facial image, and generating a second difference image based on a difference between the first facial image and the third facial image.

4. The method according to claim 3, wherein the determining a facial contour image of the to-be-recognized face based on the at least one difference image, comprises:

determining a maximum pixel value in the first difference image and the second difference image;

determining respectively a ratio of the first difference image to the maximum pixel value and a ratio of the second difference image to the maximum pixel value, to obtain a first ratio image and a second ratio image; and

marking a position of a brightness change in the first ratio image and a position of a brightness change in the second ratio image, and defining the marked first ratio image and the marked second ratio image as the facial contour image.

5. The method according to claim 1, wherein the real face prediction value calculation model comprises a convolutional neural network, and the convolutional neural network comprises a convolutional layer and a fully connected layer.

6. The method according to claim 5, wherein the inputting the at least two facial images, the at least one difference image, and the facial contour image into a pre-trained real face prediction value calculation model to obtain a real face prediction value of the to-be-recognized face, comprises:

inputting the at least two facial images, the at least one difference image, and the facial contour image into the convolution layer to obtain a fusion feature vector of the at least two facial images, the at least one difference image, and the facial contour image, wherein the convolution layer is used to represent a corresponding relationship between the facial images, the difference image, the facial contour image and the fusion feature vector; and

inputting the obtained fusion feature vector into the fully connected layer to obtain the real face prediction value of the to-be-recognized face, wherein the fully connected layer is used to represent a corresponding relationship between the fusion feature vector and the real face prediction value.

7. The method according to claim 1, wherein the real face prediction value calculation model is obtained by training by:

acquiring an initial real face prediction value calculation model;

acquiring a plurality of real face image samples and a plurality of non-real face image samples captured by the near-infrared device as a sample set, wherein the real face image samples comprise near-infrared facial image samples, brightness difference image samples and brightness difference based facial contour samples of a real face captured under different near-infrared illumination, and the non-real face image samples comprise near-infrared facial image samples, brightness difference image samples and brightness difference based facial contour samples of a real face image captured under different near-infrared illumination; and

training the initial real face prediction value calculation model to obtain the real face prediction value calculation model using each sample in the sample set as input data, and using a manually annotated numerical sample corresponding to each sample as corresponding output data.

8. An apparatus for recognizing a face, the apparatus comprising:

at least one processor; and

a memory storing instructions, the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising;

acquiring at least two facial images of a to-be-recognized face under different illuminations using a near-infrared photographing device;

generating at least one difference image based on a brightness difference between each two of the at least two facial images;

determining a facial contour image of the to-be-recognized face based on the at least one difference image;

inputting the at least two facial images, the at least one difference image, and the facial contour image into a pre-trained real face prediction value calculation model to obtain a real face prediction value of the to-be-recognized face, the real face prediction value calculation model being used to represent a corresponding relationship between the at least two facial images, the at least one difference image, the facial contour image and the real face prediction value; and

outputting prompt information for indicating successful recognition of a real face, in response to determining the obtained real face prediction value being greater than a preset threshold.

9. The apparatus according to claim 8, wherein the near-infrared photographing device comprises an illumination lamp;

and

the acquiring at least two facial images of a to-be-recognized face under different illuminations using a near-infrared photographing device, comprises;

acquiring respectively a first facial image of the to-be-recognized face under no illumination of the illumination lamp, a second facial image when the illumination lamp illuminates a first side of the to-be-recognized face, and a third facial image when the illumination lamp illuminates a second side of the to-be-recognized face.

10. The apparatus according to claim 9, wherein the generating at least one difference image based on a brightness difference between each two of the at least two facial images, comprises:

generating a first difference image based on a brightness difference between the first facial image and the second facial image, and generating a second difference image based on a difference between the first facial image and the third facial image.

11. The apparatus according to claim 10, wherein the determining a facial contour image of the to-be-recognized face based on the at least one difference image, comprises:

determining a maximum pixel value in the first difference image and the second difference image;

determining respectively a ratio of the first difference image to the maximum pixel value and a ratio of the second difference image to the maximum pixel value, to obtain a first ratio image and a second ratio image; and

marking a position of a brightness change in the first ratio image and a position of a brightness change in the second ratio image, and defining the marked first ratio image and the marked second ratio image as the facial contour image.

12. The apparatus according to claim 8, wherein the real face prediction value calculation model comprises a convolutional neural network, and the convolutional neural network comprises a convolutional layer and a fully connected layer.

13. The apparatus according to claim 12, wherein the inputting the at least two facial images, the at least one difference image, and the facial contour image into a pre-trained real face prediction value calculation model to obtain a real face prediction value of the to-be-recognized face, comprises:

inputting the at least two facial images, the at least one difference image, and the facial contour image into the convolution layer to obtain a fusion feature vector of the at least two facial images, the at least one difference image, and the facial contour image, wherein the convolution layer is used to represent a corresponding relationship between the facial images, the difference image, the facial contour image and the fusion feature vector; and

inputting the obtained fusion feature vector into the fully connected layer to obtain the real face prediction value of the to-be-recognized face, wherein the fully connected layer is used to represent a corresponding relationship between the fusion feature vector and the real face prediction value.

14. The apparatus according to claim 8, wherein the real face prediction value calculation model is obtained by training by:

acquiring an initial real face prediction value calculation model;

acquiring a plurality of real face image samples and a plurality of non-real face image samples captured by the near-infrared device as a sample set, wherein the real face image samples comprise near-infrared facial image samples, brightness difference image samples and brightness difference based facial contour samples of a real face captured under different near-infrared illumination, and the non-real face image samples comprise near-infrared facial image samples, brightness difference image samples and brightness difference based facial contour samples of a real face image captured under different near-infrared illumination; and

training the initial real face prediction value calculation model to obtain the real face prediction value calculation model using each sample in the sample set as input data, and using a manually annotated numerical sample corresponding to each sample as corresponding output data.

15. A non-transitory computer storage medium storing a computer program, the computer program when executed by one or more processors, causes the one or more processors to perform operations, the operations comprising:

acquiring at least two facial images of a to-be-recognized face under different illuminations using a near-infrared photographing device;

generating at least one difference image based on a brightness difference between each two of the at least two facial images;

determining a facial contour image of the to-be-recognized face based on the at least one difference image;

inputting the at least two facial images, the at least one difference image, and the facial contour image into a pre-trained real face prediction value calculation model to obtain a real face prediction value of the to-be-recognized face, the real face prediction value calculation model being used to represent a corresponding relationship between the at least two facial images, the at least one difference image, the facial contour image and the real face prediction value; and

outputting prompt information for indicating successful recognition of a real face, in response to determining the obtained real face prediction value being greater than a preset threshold.