Authentication based on heartbeat detection and facial recognition in video data

• US 9,953,231 B1
• Filed: 11/17/2016
• Issued: 04/24/2018
• Est. Priority Date: 11/17/2015
• Status: Active Grant

• Current Assignee: UIPCO, LLC, United Services Automobile Association
• Original Assignee: United Services Automobile Association
• Inventors: Medina, III, Reynaldo, Chavez, Carlos, Sridaran, Nandhini, Colwell, Russell, Liu, Yuesheng, Briscoe, Storme Shelton
• Primary Examiner(s): Tucker, Wesley J
• Application Number:
  US15/354,529
• Time in Patent Office: 523 Days
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06V 40/15 Biometric patterns based on physiological signals, e.g. heartbeat, blood flow
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/45 Detection of the body part being alive
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques

Abstract

Techniques are described for authenticating a user to access information through an application executing on a computing device. Multiple authentication methods may be used in combination to authenticate the user with greater confidence than authentication provided by a single method, and to verify that the user is a live person and not an image or video. Facial recognition may be used in conjunction with heartbeat detection, via video analysis, to verify the user'"'"'s identity and confirm that the user is live. Facial recognition may include capturing an image of the user'"'"'s face and comparing certain points on the face with previously gathered information regarding the user. Heartbeat detection may include capturing a video segment of the user, stabilizing the captured video data, applying motion magnification techniques to the stabilized video data, and analyzing the stabilized, magnified video data to determine a presence of the user'"'"'s heartbeat.

Claims

1. A computer-implemented method performed by at least one processor, the method comprising:

receiving, by the at least one processor, video data of a user, the video data collected in response to detecting an attempt to access information through an application executing on a user device, the video data generated by a video capture component of a user device;

comparing, by the at least one processor, at least one frame of the video data to a previously collected image of the user;

analyzing, by the at least one processor, the video data to detect a heartbeat of the user;

providing, by the at least one processor, access to high-risk information through the application based at least partly on;

detecting the heartbeat of the user based on the analyzing of the video data; and

determining that the at least one frame of the video data corresponds to the previously collected image of the user; and

providing, by the at least one processor, access to low-risk information through the application based at least partly on;

failing to detect the heartbeat of the user based on the analyzing of the video data; and

determining that the at least one frame of the video data corresponds to the previously collected image of the user.

2. The method of claim 1, wherein analyzing the video data to detect the heartbeat further comprises:

performing motion stabilization on the video data; and

magnifying the video data.

3. The method of claim 1, further comprising:

presenting, by the at least one processor, at least one word through a user interface of the application;

receiving, by the at least one processor, audio data of the user speaking the at least one word, the audio data generated by an audio capture component of the user device; and

comparing, by the at least one processor, the audio data to a stored voice print associated with the user;

wherein providing access to the information through the application is further based on determining that the audio data at least partly corresponds to the stored voice print associated with the user.

4. The method of claim 1, further comprising:

analyzing, by the at least one processor, the video data to determine a current emotional state of the user;

accessing, by the at least one processor, emotional state data indicating at least one previous emotional state of the user during at least one previous attempt to access information through the application; and

comparing, by the at least one processor, the current emotional state to the at least one previous emotion state;

wherein providing access to the information through the application is further based on determining that the current emotional state at least partly corresponds to the at least one previous emotional state.

5. The method of claim 1, wherein providing access to the information through the application is further based on one or more of:

determining that an eye color of the user in at least one frame of the video data corresponds to previously determined eye color information for the user;

determining that a personal identification number (PIN) entered by the user corresponds to a valid PIN for the user;

determining that an answer provided by the user in response to a knowledge-based authentication (KBA) question corresponds to a stored answer associated with the user;

determining that an identifier of the user device corresponds to a stored device identifier associated with the user;

or

detecting that the user has entered, through a user interface of the application, a one-time passcode previously communicated to the user.

6. The method of claim 1, further comprising:

determining, by the at least one processor, a score as a sum of weights corresponding to;

detecting the heartbeat of the user; and

determining that the at least one frame of the video data corresponds to the previously collected image of the user;

wherein providing access to the information through the application is further based on determining that the score is at least a threshold score.

7. The method of claim 6, wherein the threshold score is based at least partly on a risk level of the information to be accessed through the application.

8. The method of claim 6, wherein the score is determined as the sum of weights further corresponding to one or more of:

determining that an eye color of the user in at least one frame of the video data corresponds to previously determined eye color information for the user;

determining that a personal identification number (PIN) entered by the user corresponds to a valid PIN for the user;

determining that an answer provided by the user in response to a knowledge-based authentication (KBA) question corresponds to a stored answer associated with the user;

determining that an identifier of the user device corresponds to a stored device identifier associated with the user;

detecting that the user has entered, through a user interface of the application, a one-time passcode previously communicated to the user;

or

determining that recorded audio data of the user speaking at least partly corresponds to the stored voice print associated with the user.

9. A system comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor, the memory storing instructions which, when executed, cause the at least one processor to perform operations comprising;

receiving video data of a user, the video data collected in response to detecting an attempt to access information through an application executing on a user device, the video data generated by a video capture component of a user device;

comparing at least one frame of the video data to a previously collected image of the user;

analyzing the video data to detect a heartbeat of the user;

providing access to high-risk information through the application based at least partly on;

detecting the heartbeat of the user based on the analyzing of the video data; and

determining that the at least one frame of the video data corresponds to the previously collected image of the user; and

providing access to low-risk information through the application based at least partly on;

failing to detect the heartbeat of the user based on the analyzing of the video data; and

determining that the at least one frame of the video data corresponds to the previously collected image of the user.

10. The system of claim 9, wherein analyzing the video data to detect the heartbeat further comprises:

performing motion stabilization on the video data; and

magnifying the video data.

11. The system of claim 9, the operations further comprising:

presenting at least one word through a user interface of the application;

receiving audio data of the user speaking the at least one word, the audio data generated by an audio capture component of the user device; and

comparing the audio data to a stored voice print associated with the user;

wherein providing access to the information through the application is further based on determining that the audio data at least partly corresponds to the stored voice print associated with the user.

12. The system of claim 9, the operations further comprising:

analyzing the video data to determine a current emotional state of the user;

accessing emotional state data indicating at least one previous emotional state of the user during at least one previous attempt to access information through the application; and

comparing the current emotional state to the at least one previous emotion state;

wherein providing access to the information through the application is further based on determining that the current emotional state at least partly corresponds to the at least one previous emotional state.

13. The system of claim 9, the operations further comprising:

determining a score as a sum of weights corresponding to;

detecting the heartbeat of the user; and

determining that the at least one frame of the video data corresponds to the previously collected image of the user;

wherein providing access to the information through the application is further based on determining that the score is at least a threshold score.

14. One or more non-transitory computer-readable storage media storing instructions which, when executed, cause at least one processor to perform operations comprising:

receiving video data of a user, the video data collected in response to detecting an attempt to access information through an application executing on a user device, the video data generated by a video capture component of a user device;

comparing at least one frame of the video data to a previously collected image of the user;

analyzing the video data to detect a heartbeat of the user;

providing access to high-risk information through the application based at least partly on;

detecting the heartbeat of the user based on the analyzing of the video data; and

determining that the at least one frame of the video data corresponds to the previously collected image of the user; and

providing access to low-risk information through the application based at least partly on;

failing to detect the heartbeat of the user based on the analyzing of the video data; and

determining that the at least one frame of the video data corresponds to the previously collected image of the user.

15. The one or more non-transitory computer-readable storage media of claim 14, wherein analyzing the video data to detect the heartbeat further comprises:

performing motion stabilization on the video data; and

magnifying the video data.

16. The one or more non-transitory computer-readable storage media of claim 14, the operations further comprising:

presenting at least one word through a user interface of the application;

receiving audio data of the user speaking the at least one word, the audio data generated by an audio capture component of the user device; and

comparing the audio data to a stored voice print associated with the user;

wherein providing access to the information through the application is further based on determining that the audio data at least partly corresponds to the stored voice print associated with the user.

17. The one or more non-transitory computer-readable storage media of claim 14, the operations further comprising:

analyzing the video data to determine a current emotional state of the user;

accessing emotional state data indicating at least one previous emotional state of the user during at least one previous attempt to access information through the application; and

comparing the current emotional state to the at least one previous emotion state;

wherein providing access to the information through the application is further based on determining that the current emotional state at least partly corresponds to the at least one previous emotional state.