Spoof detection for facial recognition

• US 9,898,674 B2
• Filed: 12/10/2015
• Issued: 02/20/2018
• Est. Priority Date: 12/10/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Connell, II, Jonathan H., Ratha, Nalini K.
• Primary Examiner(s): YEH, EUENG NAN
• Application Number:
  US14/965,604
• Publication Number:
  US 20170169303A1
• Time in Patent Office: 803 Days
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/40 Spoof detection, e.g. liveness detection
• CPC Class Codes: G06V 40/45 Detection of the body part being alive
• CPC Class Codes: G06V 40/67 by interactive indications to the user

Abstract

An embodiment of the invention provides a method of analyzing an image of a user to determine whether the image is authentic, where a first image of a user'"'"'s face is received with a camera. Four or more two-dimensional feature points can be located that do not lie on the same two-dimensional plane. Additional images of the user'"'"'s face can be received; and, the at least four two-dimensional feature points can be located on each additional image with the image processor. The image processor can identify displacements between the two-dimensional feature points on the additional image and the two-dimensional feature points on the first image for each additional image. A processor can determine whether the displacements conform to a three-dimensional surface model. The processor can determine whether to authenticate the user based on the determination of whether the displacements conform to the three-dimensional surface model.

Claims

1. A method for determining the liveness of the face of a user, said method comprising:

acquiring a first image of a face of a user with an interface;

providing an instruction to the user to reorient the user'"'"'s head to a new position with an output device;

acquiring a second image of the face of the user with the interface;

determining if the instructed reorientation was performed with a processor connected to the interface by comparing the first image and the second image;

identifying a plurality of facial feature points on the first image and the second image;

identifying displacements of the facial feature points between the first image and the second image;

fitting the displacements of the facial feature points to a three-dimensional surface model; and

determining whether the first image and the second image are authentic with the processor based on said fitting of the displacements of the facial feature points to the three-dimensional surface model.

2. The method according to claim 1, wherein said determining of whether the first image and the second image are authentic includes identifying the first image and the second image to be authentic when the instructed head reorientation was performed and the displacements of the facial feature points fail to conform to the three-dimensional surface model.

3. The method according to claim 1, wherein said determining of whether the first image and the second image are authentic includes identifying the first image and the second image to be not authentic when at least one of the instructed head reorientation was not performed and the displacements of the facial feature points conforms to the three-dimensional surface model.

4. The method according to claim 1, wherein said determining of whether the first image and the second image are authentic includes identifying the first image and the second image to be not authentic when the instructed head reorientation was not performed and the displacements of the facial feature points conforms to the three-dimensional surface model.

5. The method according to claim 1 wherein said providing of the instruction, said acquiring of the second image, and said determining if the instructed reorientation was performed are repeated, and wherein the face is declared live only if all the reorientations were determined to have been performed.

6. The method according to claim 5, wherein said identifying of the plurality of facial feature points, said identifying of the displacements of the facial feature points, and said fitting of the displacements of the facial feature points to a three-dimensional surface model are repeated using the plurality of second images acquired, and wherein the face is declared live only if a plurality of the sets of displacements fail to conform to the three-dimensional model.

7. The method according to claim 1, wherein the facial feature points include a set of at least six points on the face that do not lie on a singly curved three-dimensional surface, and wherein the three-dimensional surface model includes a singly curved three-dimensional surface.

8. The method according to claim 7, wherein said fitting of the displacements of the facial feature points to the three-dimensional surface model produces an average residual error for the facial feature points, and

wherein said determining of whether the first image and the second image are authentic includes identifying the first image and the second image to be not authentic when the average residual error for the facial feature points is above a threshold.

9. The method according to claim 7, wherein said fitting of the displacements of the facial feature points to the three-dimensional surface model produces a residual error for the facial feature points, and

wherein said determining of whether the first image and the second image are authentic includes identifying the first image and the second image to be not authentic when the residual error for the facial feature points is above a threshold.

10. A method of analyzing an image of a user to determine whether the image is authentic, said method comprising:

receiving a first image of a user'"'"'s face with a camera;

locating at least four two-dimensional feature points with an image processing device connected to the camera, wherein the at least four two-dimensional feature points do not lie on the same two-dimensional plane;

receiving at least one additional image of the user'"'"'s face with the camera;

locating the at least four two-dimensional feature points on each additional image with an image processor;

identifying for each additional image displacements between the at least four two-dimensional feature points on the additional image and the at least four two-dimensional feature points on the first image with the image processor;

determining whether each of the identified displacements conform to a three-dimensional surface model with a processor connected to the image processing device; and

determining whether to authenticate the user with the processor, said determining of whether to authenticate the user being based on said determination of whether each of the identified displacements conform to the three-dimensional surface model.

11. The method according to claim 10, wherein the three-dimensional surface model is an affine transformation.

12. The method according to claim 10, wherein said determining whether to authenticate the user includes authenticating the user when each of the identified displacements do not conform to the three-dimensional surface model.

13. The method according to claim 10, wherein said determining whether to authenticate the user includes rejecting the user when each of the identified displacements conform to the three-dimensional surface model.

14. The method according to claim 10, further comprising:

providing an instruction to the user to reposition the user'"'"'s head; and

determining whether the user'"'"'s head was repositioned in compliance with the instruction in each additional image.

15. The method according to claim 10, wherein the at least four two-dimensional feature points includes the user'"'"'s nose, and

wherein said determining whether the displacements conform to the three-dimensional surface model includes determining that the displacements conform to the three-dimensional surface model when a displacement of the user'"'"'s nose feature point is below a threshold displacement.

16. A computer program product comprising:

a computer readable storage medium having stored thereon;

first program instructions executable by a device to cause the device to receive a first image of a user'"'"'s face;

second program instructions executable by the device to cause the device to locate at least four two-dimensional feature points, wherein the at least four two-dimensional feature points do not lie on the same two-dimensional plane;

third program instructions executable by the device to cause the device to receive at least one additional image of the user'"'"'s face;

fourth program instructions executable by the device to cause the device to locate the at least four two-dimensional feature points on each additional image;

fifth program instructions executable by the device to cause the device to identify for each additional image displacements between the at least four two-dimensional feature points on the additional image and the at least four two-dimensional feature points on the first image;

sixth program instructions executable by the device to cause the device to determine whether the identified displacements conform to a three-dimensional surface model; and

seventh program instructions executable by the device to cause the device to determine whether to authenticate the user based on the determination of whether the identified displacements conform to the three-dimensional surface model.

17. The computer program product according to claim 16, wherein the three-dimensional surface model is an affine transformation.

18. The computer program product according to claim 16, wherein said seventh program instructions authenticates the user when the identified displacements do not conform to the three-dimensional surface model.

19. The computer program product according to claim 16, wherein said seventh program instructions rejects the user when the identified displacements conform to the three-dimensional surface model.

20. The computer program product according to claim 16, further comprising:

eighth program instructions executable by the device to cause the device to provide an instruction to the user to reposition the user'"'"'s head; and

ninth program instructions executable by the device to cause the device to determine whether the user'"'"'s head was repositioned in compliance with the instruction in each additional image.