Method and system for face image recognition

• US 9,530,047 B1
• Filed: 11/30/2013
• Issued: 12/27/2016
• Est. Priority Date: 11/30/2013
• Status: Active Grant

• Current Assignee: Beijing SenseTime Technology Development Co., Ltd.
• Original Assignee: Beijing SenseTime Technology Development Co., Ltd.
• Inventors: Sun, Yi, Tang, Xiaoou, Wang, Xiaogang
• Primary Examiner(s): Rahmjoo, Manuchehr
• Application Number:
  US15/039,956
• Publication Number:
  US 20160379044A1
• Time in Patent Office: 1,123 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/7747 Organisation of the process, e.g. bagging or boosting
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A method for face image recognition is disclosed. The method comprises generating one or more face region pairs of face images to be compared and recognized; forming a plurality of feature modes by exchanging the two face regions of each face region pair and horizontally flipping each face region of each face region pair; receiving, by one or more convolutional neural networks, the plurality of feature modes, each of which forms a plurality of input maps in the convolutional neural network; extracting, by the one or more convolutional neural networks, relational features from the input maps, which reflect identity similarities of the face images; and recognizing whether the compared face images belong to the same identity based on the extracted relational features of the face images. In addition, a system for face image recognition is also disclosed.

Claims

1. A method for face image recognition, comprising:

generating one or more face region pairs of face images to be compared and recognized;

forming a plurality of feature modes by exchanging two face regions of each face region pair and horizontally flipping each face region of each face region pair;

receiving, by one or more convolutional neural networks, the plurality of feature modes, each of which forms a plurality of input maps;

extracting, by the one or more convolutional neural networks, one or more identity relational features from the input maps to form a plurality of output maps which reflect identity relations of the compared face images; and

recognizing whether the face images belong to the same identity based on the identity relational features of the face images.

2. The method according to claim 1, wherein the step of generating further comprises:

detecting a plurality of facial feature points of the face images to be recognized;

aligning the face images to be recognized according to the detected one or more facial feature points; and

selecting a plurality of regions on the same position of the aligned face images to generate one or more face region pairs, respectively.

3. The method according to claim 1, wherein the convolutional neural network comprises a plurality of convolutional layers, the relational features comprises local low-level relational features and global high-level relational features, and the step of extracting comprises:

extracting local low-level relational features from the input maps in lower convolutional layers of the convolutional neural network; and

extracting global high-level relational features based on the extracted local low-level relational feature in subsequent layers of the convolutional neural network which reflect identity similarities of the compared face images.

4. The method according to claim 1, wherein after the step of extracting and before the step of recognizing, the method further comprises:

pooling the exacted relational features to obtain stable and compact relational features.

5. The method according to claim 1, wherein the step of extracting comprises:

extracting, by the same convolutional neural network, the relational features from the input maps formed by different input feature modes;

or

extracting, by different convolutional neural networks, the relational features from the same region of different face region pairs.

6. The method according to claim 1, wherein each face region of the face region pairs comprises a plurality of color channels, each color channel in each face region forms an input map in the convolutional neural network.

7. A system for face image recognition, comprising:

a memory that stores executable units; and

a processor electronically coupled to the memory to execute the executable units to perform operations of the system, wherein, the executable units comprise;

a generating unit configured to generate one or more face region pairs of face images to be compared and recognized;

a forming unit configured to form a plurality of feature modes by exchanging the two face regions of each face region pair and horizontally flipping each face region of each face region pair;

one or more convolutional neural networks configured to receive the plurality of feature modes, each of which forms a plurality of input maps, and the convolutional neural networks is further configured to extract identity relational features hierarchically from the input maps, which reflect identity similarities of the compared face images; and

a recognizing unit configured to recognize whether the face images belong to same identity based on the identity relational features of the compared face images.

8. The system according to claim 7, wherein the generating unit comprises:

a detecting module configured to detect a plurality of facial feature points of face images to be recognized;

an aligning module configured to align the face images to be recognized according to the detected facial feature points; and

a selecting module configured to select one or more regions on the same position of the aligned face images to be recognized to generate one or more face region pairs, respectively.

9. The system according to claim 7, wherein the convolutional neural network comprises a plurality of convolutional layers, the identity relational features comprise local low-level relational features and global high-level relational features, and each of the convolutional layers is further configured to:

extract local low-level relational features from the input maps in lower convolutional layers of the convolutional neural network; and

extract global high-level relational features based on the extracted local low-level relational features in the subsequent layers of the convolutional neural network, which reflect identity similarities of the face images.

10. The system according to claim 7, further comprising a pooling unit configured to pool the exacted relational features to obtain stable and compact relational features.

11. The system according to claim 7, wherein each of the convolutional layers is further configured to:

extract, by the same convolutional neural network, the relational features from the input maps formed by different input feature modes;

or

extract, by different convolutional neural networks, the relational features from the same region of different face region pairs.

12. The system according to claim 7, wherein each face region of the face region pairs comprise a plurality of color channels, each color channel in each face region forms an input map in the convolutional neural network.

13. A plurality of convolutional neural networks for extracting identity relational features for a face image recognition system, wherein each convolutional neural network comprises a plurality of convolutional layers, the identity relational features comprise local low-level relational features and global high-level relational features, each convolutional neural network is configured to:

receive one particular feature mode of one particular face region pair from the face image recognition system to form a plurality of input maps;

extract local low-level relational features from the input maps in lower convolutional layers of the convolutional neural network; and

extract global high-level relational features based on the extracted local low-level relational features in subsequent layers of the convolutional neural network, the extracted global high-level relational features reflect identity similarities of the compared face images.