Systems and methods for discerning eye signals and continuous biometric identification

• US 10,156,900 B2
• Filed: 01/27/2017
• Issued: 12/18/2018
• Est. Priority Date: 05/09/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Publicover, Nelson, Marggraff, Lewis
• Primary Examiner(s): SIMITOSKI, MICHAEL J
• Application Number:
  US15/418,034
• Publication Number:
  US 20170235931A1
• Time in Patent Office: 690 Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 21/316 by observing the pattern of computer usage, e.g. typical user behaviour
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/64 Protecting data integrity, e.g. using checksums, certificates or signatures
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: G06V 40/197 Matching; Classification
• CPC Class Codes: H04L 63/0861 : using biometrical features, e.g. fingerprint, retina-scan cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/44504 : Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits
• CPC Class Codes: H04W 12/065 : Continuous authentication
• CPC Class Codes: H04W 12/33 : using wearable devices, e.g. using a smartwatch or smart-glasses

Abstract

Apparatus, systems, and methods are provided for substantially continuous biometric identification (CBID) of an individual using eye signals in real time. The apparatus is included within a wearable computing device with identification of the device wearer based on iris recognition within one or more cameras directed at one or both eyes, and/or other physiological, anatomical and/or behavioral measures. Verification of device user identity can be used to enable or disable the display of secure information. Identity verification can also be included within information that is transmitted from the device in order to determine appropriate security measures by remote processing units. The apparatus may be incorporated within wearable computing that performs other functions including vision correction, head-mounted display, viewing the surrounding environment using scene camera(s), recording audio data via a microphone, and/or other sensing equipment.

Claims

1. A system comprising:

a headgear configured to be worn on a head of a user;

a first camera mounted on the headgear such that the first camera is oriented toward an eye of the user when the headgear is worn for monitoring the eye;

a processing unit operatively coupled to the first camera and configured to;

analyze an image from at least the first camera to identify a first iris of the eye;

analyze the image to identify features of the first iris;

determine a precision and number of Gabor coefficients of a biometric description of a second iris based on a desired level of security;

compare the features of the first iris to the precision and number of Gabor coefficients of the biometric description of the second iris;

confirm an identity of the user based on the features of the first iris matching the precision and number of Gabor coefficients of the biometric description of the second iris; and

authorize the user to perform one or more actions upon confirming the identity of the user;

a display mounted on the headgear such that the display is viewable by the user when the headgear is worn, the display operatively coupled to the processing unit;

a sight shield mountable to the headgear to at least partially cover the display to prevent information presented on the display from being viewable by other than the eye of the user when the headgear is worn; and

a sensor for determining a position of the sight shield, the sensor operatively coupled to the processing unit;

wherein the processing unit is configured to authorize the user to access information via the display upon confirming the identity of the user and confirming that the sight shield is mounted to the headgear to at least partially cover the display via a signal from the sensor.

2. The system of claim 1, further comprising:

at least a second camera mounted on the headgear such that the second camera is oriented toward the eye of the user from a different angle than the first camera when the headgear is worn for monitoring the eye, the at least a second camera operatively coupled to the processing unit; and

wherein the processing unit is configured to;

analyze one or more images from the first camera and the at least second camera to identify the first iris of the eye;

select, from the first and second cameras, a chosen camera providing a view of the first iris based on the one or more images; and

analyze the view to identify features of the first iris by analyzing the image from the chosen camera of the one or more images to identify the features of the first iris.

3. The system of claim 2, wherein the processing unit is configured to select the chosen camera based on a shape of the first iris identified in the one or more images from the first camera and the at least second camera.

4. The system of claim 2, wherein the processing unit is configured to select the chosen camera based on a shape of the first iris identified in a first image from the first camera being closer to circular than a shape of the first iris identified in a second image from the at least second camera.

5. The system of claim 2, wherein the processing unit is configured to select the chosen camera based on at least a portion of the first iris being identifiable from an image of the one or more images and satisfying a minimum stringency threshold needed to confirm the identity of the user.

6. The system of claim 1, further comprising:

a display mounted on the headgear such that the display is viewable by the user when the headgear is worn, the display operative coupled to the processing unit; and

wherein the processing unit is configured to

authorize the user to perform one or more actions via the display upon confirming the identity of the user; and

modify an opacity of the display to preserve security of information presented on the display to the user.

7. The system of claim 6, wherein the processing unit is configured to

identify one or more regions on the display where secure information is rendered;

and

modify the one or more regions to render the regions substantially opaque while the secure information is rendered.

8. The system of claim 1, further comprising:

a scene camera mounted on the headgear and such that the scene camera is oriented away from the eye of the user and toward a scene viewed by the eye when the headgear is worn, the scene camera operatively coupled to the processing unit; and

wherein the processing unit is configured to store to memory scene images captured by the scene camera.

9. The system of claim 1, further comprising:

a wireless transceiver mounted on the headgear and operatively coupled to the first camera, and

wherein the processing unit comprises a processor remote from the headgear and wirelessly coupled to the first camera via the wireless transceiver.

10. The system of claim 1, further comprising one or more sources of illumination mounted on the headgear such that the one or more sources of illumination are oriented toward the eye when the headgear is worn.

11. The system of claim 1, wherein the processing unit is configured to:

confirm the identity of the user based on the features of the first iris by comparing the features of the first iris to known features of at least one known user to find a match.

12. A system comprising:

a headgear configured to be worn on a head of a user;

a plurality of cameras mounted on the headgear such that the cameras are oriented toward an eye of the user from different angles when the headgear is worn for monitoring the eye;

a processing unit operatively coupled to the plurality of cameras and configured to;

analyze one or more images from the plurality of cameras to identify a first iris of the eye;

select at least one of the plurality of cameras based on the one or more images, the selected at least one of the plurality of cameras providing a desired view of the first iris;

analyze an image from the at least one of the plurality of cameras to identify features of the first iris;

determine a precision and number of Gabor coefficients of a biometric description of a second iris based on a desired level of security;

compare the features of the first iris to the precision and number of Gabor coefficients of the biometric description of the second iris;

confirm an identity of the user based on the comparison of the features of the first iris to the precision and number of Gabor coefficients of the biometric description of the second iris; and

authorize the user to perform one or more actions upon confirming the identity of the user;

a display mounted on the headgear such that the display is viewable by the user when the headgear is worn, the display operatively coupled to the processing unit;

a sight shield mountable to the headgear to at least partially cover the display to prevent information presented on the display from being viewable by other than the eye of the user when the headgear is worn; and

a sensor for determining a position of the sight shield, the sensor operatively coupled to the processing unit;

wherein the processing unit is configured to authorize the user to access information via the display upon confirming the identity of the user and confirming that the sight shield is mounted to the headgear to at least partially cover the display via a signal from the sensor.

13. The system of claim 12, wherein the processing unit is configured to:

select images from multiple cameras of the plurality of cameras, the multiple cameras of the plurality of cameras providing desired views of the first iris;

create a composite representation of the first iris from the images selected from the multiple cameras; and

confirm the identity of the user based on the composite representation of the first iris.

14. The system of claim 13, wherein the processing unit is configured to:

create the composite representation of the first iris by selecting portions of the images selected from the multiple cameras that provide desired views of the first iris and combining the portions into a composite image of the first iris.

15. The system of claim 14, wherein the processing unit is configured to:

create the composite representation of the first iris by rotating one or more of the images selected from the multiple cameras to align the images from the multiple cameras that provide desired views of the first iris along a common axis before combining the portions into the composite image of the first iris.

16. The system of claim 13, wherein the processing unit is configured to:

confirm the identity of the user based on the features of the first iris by comparing the composite representation of the first iris of the iris to Gabor coefficient-based descriptions of one or more known composite representations of irises of one or more known users to find a match.

17. The system of claim 12, wherein the processing unit is configured to:

confirm the identity of the user based on the features of the first iris by comparing the features of the first iris to Gabor coefficient-based descriptions of an iris of at least one known user to find a match.

18. The system of claim 12, wherein the processing unit is configured to at least one of substantially continuously;

periodically;

or when a predetermined action is selected by the user that requires confirmation of identity before performing the predetermined action,

analyze the one or more images;

select the at least one of the plurality of cameras;

analyze the image from the at least one of the plurality of cameras;

confirm the identity of the user; and

authorize the user.

19. A system for providing substantially continuous biometric identification of a user, the system comprising:

a headgear configured to be worn on a user'"'"'s head;

a plurality of cameras mounted on the headgear such that the plurality of cameras is oriented toward an eye of the user when the headgear is worn for monitoring the eye;

a display mounted on the headgear such that the display is viewable by the user when the headgear is worn;

a sight shield mountable to the headgear to at least partially cover the display to prevent information presented on the display from being viewable by other than the eye of the user when the headgear is worn;

a processing unit operatively coupled to the plurality of cameras and display, the processing unit configured to;

analyze one or more images from the plurality of cameras to identify features of a first iris of the eye;

determine a precision and number of Gabor coefficients of a biometric description of a second iris based on a desired level of security;

compare the features of the first iris to the precision and number of Gabor coefficients of the biometric description of the second iris;

identify the user based at least in part on the features of the first iris compared to the precision and number of Gabor coefficients of the biometric description of the second iris;

authorize the user to access information via the display upon confirming an identification of the user; and

a sensor for determining a position of the sight shield, the sensor operatively coupled to the processing unit;

wherein the processing unit is configured to authorize the user to access information via the display upon confirming the identity of the user and confirming that the sight shield is mounted to the headgear to at least partially cover the display via a signal from the sensor.