LIVENESS TEST METHOD AND APPARATUS

• US 20190197331A1
• Filed: 12/19/2018
• Published: 06/27/2019
• Est. Priority Date: 12/21/2017
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: KWAK, Youngjun, YOO, Byung In, KIM, Youngsung, CHOI, Chang Kyu, HAN, Jaejoon
• Granted Patent: US 11,157,760 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/45 Detection of the body part being alive
• CPC Class Codes: G06V 40/50 Maintenance of biometric data or enrolment thereof

Abstract

A processor-implemented liveness test method includes detecting a face region in a query image, the query image including a test object for a liveness test, determining a liveness test condition to be applied to the test object among at least one liveness test condition for at least one registered user registered in a registration database, determining at least one test region in the query image based on the detected face region and the determined liveness test condition, obtaining feature data of the test object from image data of the determined at least one test region using a neural network-based feature extractor, and determining a result of the liveness test based on the obtained feature data and registered feature data registered in the registration database and corresponding to the determined liveness test condition.

Claims

1. A processor-implemented liveness test method comprising:

detecting a face region in a query image, the query image comprising a test object for a liveness test;

determining a liveness test condition to be applied to the test object among at least one liveness test condition for at least one registered user registered in a registration database;

determining at least one test region in the query image based on the detected face region and the determined liveness test condition;

obtaining feature data of the test object from image data of the determined at least one test region using a neural network-based feature extractor; and

determining a result of the liveness test based on the obtained feature data and registered feature data registered in the registration database and corresponding to the determined liveness test condition.

2. The processor-implemented liveness test method of claim 1, wherein the liveness test condition comprises either one of both of a size of a first test region and a location of a second test region.

3. The processor-implemented liveness test method of claim 1, wherein the determining of the at least one test region comprises determining either one or both of a first test region and a second test region based on the detected face region and the determined liveness test condition.

4. The processor-implemented liveness test method of claim 3, wherein the determining of the either one or both of the first test region and the second test region comprises determining, to be the first test region, one of a first region having a size larger than a size of the detected face region, a second region having a size equal to the size of the detected face region, and a third region having a size smaller than the size of the detected face region based on the determined liveness test condition.

5. The processor-implemented liveness test method of claim 3, wherein the determining of the either one or both of the first test region and the second test region comprises determining, to be the second test region, a partial face region having a location in the detected face region determined based on the determined liveness test condition.

6. The processor-implemented liveness test method of claim 3, wherein the obtaining of the feature data of the test object comprises obtaining the feature data of the test object based on the determined either one or both of the first test region and the second test region, and a third test region corresponding to an entire region of the query image.

7. The processor-implemented liveness test method of claim 1, wherein the determining of the liveness test condition comprises, in response to a single liveness test condition for a single registered user being registered in the registration database, determining the single liveness test condition for the single registered user to be the liveness test condition to be applied to the test object.

8. The processor-implemented liveness test method of claim 1, wherein the determining of the liveness test condition comprises, in response to a plurality of liveness test conditions for a plurality of registered users being registered in the registration database:

performing user recognition on the test object to determine a registered user corresponding to the test object among the registered users; and

determining, to be the liveness test condition to be applied to the test object, a liveness test condition for the registered user corresponding to the test object among the liveness test conditions.

9. The processor-implemented liveness test method of claim 1, wherein the determining of the liveness test condition comprises, in response to a plurality of liveness test conditions for a plurality of registered users being registered in the registration database:

displaying identifiers respectively corresponding to the liveness test conditions;

receiving data for selecting one of the displayed identifiers from a user input; and

determining, to be the liveness test condition to be applied to the test object, a liveness test condition corresponding to one of the displayed identifiers selected based on the data received from the user input.

10. The processor-implemented liveness test method of claim 1, wherein the at least one test region comprises a plurality of test regions, and

the obtaining of the feature data of the test object comprises obtaining the feature data of the test object using separate feature extractors to which image data of the plurality of test regions are respectively input.

11. The processor-implemented liveness test method of claim 1, wherein the at least one test region comprises a plurality of test regions, and

the obtaining of the feature data of the test object comprises obtaining the feature data of the test object using a single feature extractor comprising separate input layers to which image data of the plurality of test regions are respectively input.

12. The processor-implemented liveness test method of claim 1, wherein the determining of the result of the liveness test comprises determining whether the test object is a live object based on a similarity between the obtained feature data and the registered feature data.

13. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform the processor-implemented liveness test method of claim 1.

14. A processor-implemented liveness data registration method comprising:

detecting a face region in a registration image, the registration image comprising a registration object to be registered;

determining at least one candidate registration region in the registration image based on the detected face region and a current liveness test condition;

obtaining feature data of the registration object from image data of the determined at least candidate registration region using a neural network-based feature extractor;

determining a liveness score based on the obtained feature data; and

determining whether to register the current liveness test condition and the obtained feature data as liveness data of the registration object based on the determined liveness score.

15. The processor-implemented liveness data registration method of claim 14, wherein the determining of the at least one candidate registration region comprises determining either one or both of a first candidate registration region and a second candidate registration region based on the detected face region and the current liveness test condition.

16. The processor-implemented liveness data registration method of claim 15, wherein the determining of the at least one candidate registration region comprises determining, to be the first candidate registration region, one of a first region having a size larger than a size of the detected face region, a second region having a size equal to the size of the detected face region, and a third region having a size smaller than the size of the detected face region based on the determined liveness test condition.

17. The processor-implemented liveness data registration method of claim 15, wherein the determining of the at least one candidate registration region comprises determining, to be the second candidate registration region, a partial face region having a location in the detected face region determined based on the current liveness test condition.

18. The processor-implemented liveness data registration method of claim 15, wherein the obtaining of the feature data of the registration object comprises obtaining the feature data of the registration object based on the determined either one or both of the first candidate registration region and the second candidate registration region, and a third candidate registration region corresponding to an entire region of the registration image.

19. The processor-implemented liveness data registration method of claim 14, wherein the at least one candidate registration comprises a plurality of candidate regions, and

the obtaining of the feature data of the registration object comprises obtaining the feature data of the registration object using separate feature extractors to which image data of the plurality of candidate registration regions are respectively input.

20. The processor-implemented liveness data registration method of claim 14, wherein the determining of whether to register the current liveness test condition and the obtained feature data as liveness data of the registration object comprises, in response to the determined liveness score being greater than at least one other liveness score determined for at least one other liveness test condition, determining to register the current liveness test condition and the obtained feature data as liveness data of the registration object.

21. A liveness test apparatus for determining liveness of a test object for a liveness test, the liveness test apparatus comprising:

a processor configured to;

detect a face region in a query image, the query image comprising the test object;

determine a liveness test condition to be applied to the test object among at least one liveness test condition for at least one registered user registered in a registration database;

determine at least one test region in the query image based on the detected face region and the determined liveness test condition;

obtain feature data of the test object from image data of the determined at least one test region using a neural network-based feature extractor; and

determine a result of the liveness test based on the obtained feature data and registered feature data registered in the registration database and corresponding to the determined liveness test condition.

22. The liveness test apparatus of claim 21, further comprising a memory configured to store instructions executable by the processor;

wherein the processor is further configured to execute the instructions to configure the processor to;

detect the face region in the query image, the query image comprising the test object;

determine the liveness test condition to be applied to the test object among the at least one liveness test condition for the at least one registered user registered in the registration database;

determine the at least one test region in the query image based on the detected face region and the determined liveness test condition;

obtain the feature data of the test object from the image data of the determined at least one test region using the neural network-based feature extractor; and

determine the result of the liveness test based on the obtained feature data and the registered feature data registered in the registration database and corresponding to the determined liveness test condition.

23. The liveness test apparatus of claim 21, wherein the liveness test condition comprises either one or both of a size of a first test region and a location of a second test region.

24. A liveness data registration apparatus for registering liveness data of a registration object to be registered, the liveness data registration apparatus comprising:

a processor configured to;

detect a face region in a registration image, the registration image comprising the registration object;

determine at least one candidate registration region in the registration image based on the detected face region and a current liveness test condition;

obtain feature data of the registration object from image data of the determined at least one candidate registration region using a neural network-based feature extractor;

determine a liveness score based on the obtained feature data; and

determine whether to register the current liveness test condition and the obtained feature data as liveness data of the registration object based on the determined liveness score.

25. The liveness data registration apparatus of claim 24, further comprising a memory configured to store instructions executable by the processor;

wherein the processor is further configured to execute the instructions to configure the processor to;

detect the face region in the registration image, the registration image comprising the registration object;

determine the at least one candidate registration region in the registration image based on the detected face region and the current liveness test condition;

obtain the feature data of the registration object from the image data of the determined at least one candidate registration region using the neural network-based feature extractor;

determine the liveness score based on the obtained feature data; and

determine whether to register the current liveness test condition and the obtained feature data as the liveness data of the registration object based on the determined liveness score.

26. A processor-implemented liveness test method comprising:

obtaining a query image comprising a test object for a liveness test;

obtaining feature data of the test object from image data of the query image based on a liveness test condition personalized for an individual user using a neural network-based feature extractor; and

performing a liveness test on the test object based on the obtained feature data and feature data previously obtained for the individual user.

27. The processor-implemented liveness test of claim 26, wherein the liveness test condition specifies either one or both of a size of a first test region and a location of a second test region;

and

the obtaining of the feature data comprises obtaining the feature data from either one or both of image data of a first test region of the query image having the size specified by the liveness test condition and image data of a second test region of the query image at the location specified by the liveness test condition using the neural network-based feature extractor.

28. The processor-implemented liveness test of claim 26, wherein the feature data previously obtained for the individual user was obtained under the liveness test condition personalized for the individual user.

29. The processor-implemented liveness method of claim 26, wherein the processor-implemented liveness test method is performed in a computing apparatus configured to determine whether to allow access to the computing apparatus based on a result of the liveness test.

30. A processor-implemented liveness data registration method comprising:

obtaining a registration image comprising a registration object to be registered;

obtaining feature data of the registration object from image data of the registration image based on a current liveness test condition using a neural network-based feature extractor;

performing a liveness test on the registration object based on the obtained feature data; and

determining whether to register the current liveness test condition and the obtained feature data as liveness data of the registration object based on a result of the liveness test.

31. The processor-implemented liveness data registration method of claim 30, wherein the current liveness test condition specifies either one or both of a size of a first candidate registration region and a location of a second candidate registration region;

and

the obtaining of the feature data of the registration object comprises obtaining the feature data from either one or both of image data of a first candidate registration region of the registration image having the size specified by the current liveness test condition and image data of a second candidate registration region of the registration image at the location specified by the liveness test condition using the neural network-based feature extractor.

32. The processor-implemented liveness data registration method of claim 30, wherein the determining of whether to register the current liveness test condition and the obtained feature data as liveness data of the registration object comprises:

determining a liveness test score based on the obtained feature data; and

determining to register the current liveness test condition and the obtained feature data as liveness data of the registration object in response to the determined liveness score being greater than at least one other liveness score determined for at least one other liveness test condition.

33. The processor-implemented liveness data registration method of claim 30, wherein the processor-implemented liveness data registration method is performed in a computing apparatus configured to determine whether to allow access to the computing apparatus based on liveness data registered for a registration object.