Method for face recognition through facial expression normalization, recording medium and device for performing the method

• US 9,811,716 B2
• Filed: 11/16/2015
• Issued: 11/07/2017
• Est. Priority Date: 11/21/2014
• Status: Active Grant

• Current Assignee: Korea Advanced Institute of Science and Technology
• Original Assignee: Korea Advanced Institute of Science and Technology
• Inventors: Kim, Ig Jae, Choi, Hee Seung, Cho, Junghyun
• Primary Examiner(s): Patel, Kanjibhai
• Application Number:
  US14/941,812
• Publication Number:
  US 20160148041A1
• Time in Patent Office: 722 Days
• Field of Search: 382118
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/174 Facial expression recognition

Abstract

A method for face recognition through facial expression normalization includes: fitting an input two-dimensional face image into a three-dimensional face model by using a three-dimensional face database; normalizing the three-dimensional face model into a neutral-expression three-dimensional face model by using a neutral-expression parameter learned from the three-dimensional face database; converting the neutral-expression three-dimensional face model into a neutral-expression two-dimensional face image; and recognizing the neutral-expression two-dimensional face image from a two-dimensional face database. Accordingly, face recognition may be performed with high reliability without a loss of information.

Claims

1. A method for face recognition through facial expression normalization, comprising:

fitting an input two-dimensional face image into a three-dimensional face model by using a three-dimensional face database;

normalizing the three-dimensional face model into a neutral-expression three-dimensional face model by using a neutral-expression parameter learned from the three-dimensional face database;

converting the neutral-expression three-dimensional face model into a neutral-expression two-dimensional face image; and

recognizing the neutral-expression two-dimensional face image from a two-dimensional face database.

2. The method for face recognition through facial expression normalization according to claim 1,

wherein in the fitting of an input two-dimensional face image into a three-dimensional face model, an expression parameter of a three-dimensional face model of the input two-dimensional face image is predicted.

3. The method for face recognition through facial expression normalization according to claim 2,

wherein in the fitting of an input two-dimensional face image into a three-dimensional face model, at least one parameter is extracted among identity and expression of the three-dimensional face model.

4. The method for face recognition through facial expression normalization according to claim 2,

wherein in the fitting of an input two-dimensional face image into a three-dimensional face model, a multi-linear modeling technique is used.

5. The method for face recognition through facial expression normalization according to claim 2,

wherein in the normalizing of the three-dimensional face model into a neutral-expression three-dimensional face model, the predicted parameter is normalized into the neutral-expression parameter.

6. The method for face recognition through facial expression normalization according to claim 1, further comprising:

converting the neutral-expression three-dimensional face model into a two-dimensional face image of various expressions.

7. The method for face recognition through facial expression normalization according to claim 1,

wherein the three-dimensional face database is constructed in advance to include various kinds of target and expression information.

8. The method for face recognition through facial expression normalization according to claim 1,

wherein the two-dimensional face database includes at least one of an identity card, a passport and a criminal photo.

9. A computer-readable recording medium on which a computer program for performing the method for face recognition through facial expression normalization according to claim 1 is recorded.

10. A device for face recognition through facial expression normalization, comprising:

a fitting unit configured to fit an input two-dimensional face image into a three-dimensional face model by using a three-dimensional face database;

a normalizing unit configured to normalize the three-dimensional face model into a neutral-expression three-dimensional face model by using a neutral-expression parameter learned from the three-dimensional face database;

a converting unit configured to convert the neutral-expression three-dimensional face model into a neutral-expression two-dimensional face image; and

a recognizing unit configured to recognize the neutral-expression two-dimensional face image from a two-dimensional face database.

11. The device for face recognition through facial expression normalization according to claim 10,

wherein the fitting unit is configured to predict an expression parameter of a three-dimensional face model of the input two-dimensional face image.

12. The device for face recognition through facial expression normalization according to claim 11,

wherein the fitting unit is configured to extract at least one parameter among identity and expression of the three-dimensional face model.

13. The device for face recognition through facial expression normalization according to claim 11,

wherein the fitting unit is configured to use a multi-linear modeling technique.

14. The device for face recognition through facial expression normalization according to claim 11,

wherein the normalizing unit is configured to normalize the predicted parameter with the neutral-expression parameter.

15. The device for face recognition through facial expression normalization according to claim 10,

wherein the converting unit is configured to convert the neutral-expression three-dimensional face model into a two-dimensional face image of various expressions.

16. The device for face recognition through facial expression normalization according to claim 10,

wherein the three-dimensional face database is constructed in advance to include various kinds of target and expression information.

17. The device for face recognition through facial expression normalization according to claim 10,

wherein the two-dimensional face database includes at least one of an identity card, a passport and a criminal photo.