Eye image selection

• US 10,466,778 B2
• Filed: 01/17/2017
• Issued: 11/05/2019
• Est. Priority Date: 01/19/2016
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Kaehler, Adrian
• Primary Examiner(s): Potts, Ryan P
• Application Number:
  US15/408,197
• Publication Number:
  US 20170206412A1
• Time in Patent Office: 1,022 Days
• Field of Search: None
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: G06V 40/197 Matching; Classification
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/50 Maintenance of biometric data or enrolment thereof

Abstract

Systems and methods for eye image set selection, eye image collection, and eye image combination are described. Embodiments of the systems and methods for eye image set selection can include comparing a determined image quality metric with an image quality threshold to identify an eye image passing an image quality threshold, and selecting, from a plurality of eye images, a set of eye images that passes the image quality threshold.

Claims

1. A head mounted display system comprising:

an image capture device configured to capture a plurality of eye images of an eye;

a processor programmed to;

for each eye image of the plurality of eye images;

receive the eye image from the image capture device;

determine an image quality metric associated with the eye image; and

compare the determined image quality metric with an image quality threshold to determine whether the eye image passes the image quality threshold, wherein the image quality threshold corresponds to an image quality level for generating an iris code;

select, from the plurality of eye images, two eye images that pass the image quality threshold;

utilize both of the two selected eye images to generate a first iris code using a first iris code operation;

utilize both of the two selected eye images to generate a second iris code using a second iris code operation different from the first iris code operation;

determine a first confidence score based on the first iris code; and

determine a second confidence score based on the second iris code.

2. The head mounted display system of claim 1, wherein to select the two eye images that pass the image quality threshold, the processor is programmed to buffer the selected eye images into a buffer of the processor.

3. The head mounted display system of claim 1, wherein to select two eye images that pass the image quality threshold, the processor is programmed to utilize a polar coordinate representation of the two eye images.

4. The head mounted display system of claim 1, wherein the first iris code operation comprises an image fusion operation and the second iris code operation comprises an iris fusion operation.

5. The head mounted display system of claim 1, wherein the first iris code operation and the second iris code operation are performed in parallel or sequentially.

6. The head mounted display system of claim 4, wherein the processor is further programmed to:

determining an eye pose of at least one of the two selected eye images; and

determine biometric data of an eye using the eye pose of at least one of the two selected eye images.

7. The head mounted display system of claim 1, wherein to receive the eye image from the image capture device, the processor is programmed to receive the eye image from the image capture device during an eye image collection routine implemented by the processor.

8. The head mounted display system of claim 7, wherein the eye image collection routine utilizes a graphic to obtain eye images.

9. The head mounted display system of claim 1, wherein to select the two eye images that pass the image quality threshold, the processor is programmed to, for each of the two eye images:

buffer a first eye image of the plurality of eye images into a buffer;

determine that an image quality metric of a second eye image of the plurality of eye images passes an image quality metric of the first eye image; and

replace the first eye image with the second eye image in the buffer, wherein the second eye image corresponds to the selected eye image.

10. The head mounted display system of claim 1, wherein the image quality metric corresponds to a blur image quality associated with a blur of the eye image, wherein the blur of the eye image corresponds to a degree of eye movement in the eye image relative to a reference eye image.

11. The head mounted display system of claim 1, wherein the image quality metric corresponds to an amount of unoccluded pixels in the eye image.

12. The head mounted display system of claim 1, wherein the image quality metric comprises a measure of at least one of:

eye blink, glare, defocus, resolution, occluded pixels, unoccluded pixels, noise, artifacts, blur, or a combination thereof.

13. The head mounted display system of claim 1, wherein the image quality metric comprises a weighted combination of a plurality of component quality metrics.

14. The head mounted display system of claim 1, wherein the processor is further programmed to:

select the first iris code if the first confidence score exceeds the second confidence score; and

select the second iris code if the first confidence score does not exceed the second confidence score.

15. A method for processing an eye image, the method comprising:

under control of a processor;

obtaining a plurality of eye images;

for each eye image of the plurality of eye images,

determining an image quality metric associated with each eye image; and

comparing each determined image quality metric with an image quality threshold to determine whether the eye image passes the image quality threshold, wherein the image quality threshold corresponds to an image quality level for generating an iris code;

selecting, from the plurality of eye images, a non-empty set of eye images;

utilizing the set of eye images for generating a first iris code using a first iris code operation;

utilizing set of eye images for generating a second iris code using a second iris code operation different from the first iris code operation;

determining a first confidence score based on the first iris code; and

determining a second confidence score based on the second iris code.

16. The method of claim 15, wherein selecting the non-empty set of eye images comprises:

identifying a percentage of eye images of the plurality of eye images that pass the image quality threshold to be selected.

17. The method of claim 15, wherein selecting the non-empty set of eye images comprises:

identifying a total number of eye images to be selected; and

identifying the set of eye images, each having a determined image quality metric greater or equal to a determined image quality metric of an eye image not in the set.

18. The method of claim 15, wherein each eye image in the non-empty set of eye images passes the image quality threshold.

19. The method of claim 15, further comprising:

buffering the non-empty set of eye images and the corresponding determined image quality metrics in a data medium.

20. The method of claim 19, further comprising:

obtaining an additional eye image that passes the image quality threshold;

selecting an eye image of the non-empty set of eye images buffered in the data medium;

determining that an image quality metric of the additional eye image passes an image quality metric of the eye image of the non-empty set of eye images buffered in the data medium; and

replacing, in the data medium, the selected eye image of the non-empty set of eye images buffered in the data medium and the image quality metric of the selected eye image of the non-empty set of eye images buffered in the data medium with the additional eye image and the image quality metric of the additional eye image respectively.

21. The method of claim 15, wherein the image quality metric comprises a measure of at least one of:

eye blink, glare, defocus, resolution, occluded pixels, unoccluded pixels, noise, artifacts, blur, or a combination thereof.

22. The method of claim 15 further comprising:

selecting the first iris code if the first confidence score exceeds the second confidence score; and

selecting the second iris code if the first confidence score does not exceed the second confidence score.

23. The method of claim 15 wherein the first iris code operation comprises an image fusion operation and the second iris code operation comprises an iris fusion operation.