Determining eye openness with an eye tracking device

• US 10,394,019 B2
• Filed: 02/09/2018
• Issued: 08/27/2019
• Est. Priority Date: 02/27/2017
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: Ryan, Mark, Sundberg, Torbjorn, Rana, Pravin, Wang, Yimu
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US15/893,340
• Publication Number:
  US 20180247119A1
• Time in Patent Office: 564 Days
• Field of Search: 382115, 382117, 382128, 382209, 382218, 382278, 382282, 340 581, 340 553, 340 583
• CPC Class Codes: A61B 3/112 for measuring diameter of pupils
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/1079 using optical or photographic means
• CPC Class Codes: A61B 5/1103 Detecting eye twinkling
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: G02B 26/0875 by means of one or more refracting elements
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

A method for determining eye openness with an eye tracking device is disclosed. The method may include determining, for pixels of an image sensor of an eye tracking device, during a first time period when an eye of a user is open, a first sum of intensity of the pixels. The method may also include determining, during a second time period when the eye of the user is closed, a second sum of intensity of the pixels. The method may further include determining, during a third time period, a third sum of intensity of the pixels. The method may additionally include determining that upon the third sum exceeding a fourth sum of the first sum plus a threshold amount, that the eye of the user is closed, the threshold amount is equal to a product of a threshold fraction and a difference between the first sum and the second sum.

Claims

1. A method for determining eye openness with an eye tracking device, wherein the method comprises:

determining, for at least a portion of pixels of an image sensor of an eye tracking device, a baseline value based on a first sum of intensity of the pixels for a first time period when an eye of a user is open;

determining, for the at least portion of pixels of the image sensor of the eye tracking device, a blink amplitude value based on a second sum of intensity of the pixels for a second time period when the eye of the user is closed, wherein a sum of intensity of the pixels when the eye of the user is closed is higher than a sum of intensity of the pixels when the eye of the user is open;

determining, for the at least portion of pixels of the image sensor of the eye tracking device, during a third time period, a third sum of intensity of the pixels;

determining a threshold value based on the baseline value and the blink amplitude value; and

determining, with a processor, based upon the third sum exceeding the threshold value, that the eye of the user is closed.

2. The method for determining eye openness with an eye tracking device of claim 1, wherein the method further comprises:

activating an illuminator of the eye tracking device during each of the first time period, the second time period, and the third time period.

3. The method for determining eye openness with an eye tracking device of claim 1, wherein:

the first time period is dynamic and precedes the third time period.

4. The method for determining eye openness with an eye tracking device of claim 1, wherein:

the baseline value is a moving average of a sum of the intensity of the pixels of the image sensor.

5. The method for determining eye openness with an eye tracking device of claim 1, wherein the threshold value is determined by adding the baseline value with a product of a threshold fraction and a difference between the baseline value and the blink amplitude value.

6. The method for determining eye openness with an eye tracking device of claim 5, wherein:

the threshold fraction is between about ⅛ and

about ½

.

7. The method for determining eye openness with an eye tracking device of claim 1, wherein the blink amplitude value is a moving average of a sum of the intensity of the pixels of the image sensor.

8. The method for determining eye openness with an eye tracking device of claim 1, further comprising resetting the baseline value to a predetermined default value.

9. The method for determining eye openness with an eye tracking device of claim 8, wherein the baseline value is reset to the predetermined default value at the beginning of an initialization of the eye tracking device.

10. The method for determining eye openness with an eye tracking device of claim 8, wherein the baseline value is reset to the predetermined default value in response to detecting a predetermined glint pattern from an image generated from the pixels of the image sensor of the eye tracking device.

11. The method for determining eye openness with an eye tracking device of claim 10, wherein the predetermined glint pattern indicates a number of glints detected from the image meets a predetermined threshold value.

12. The method for determining eye openness with an eye tracking device of claim 11, wherein the glints are generated by a plurality of illuminators directed toward the eye of the user.

13. A method for determining eye openness with an eye tracking device, wherein the method comprises:

receiving, at one or more processors, an image of an eye of a user from an image sensor of an eye tracking device;

determining, with the one or more processors, based on the image of the eye, a radius of a pupil;

determining, with the one or more processors, based on the radius of the pupil, a total area of the pupil;

determining, with the one or more processors, based on the image of the eye and the radius of the pupil, an amount of the total area of the pupil which is not obscured by either eyelid; and

determining, with the one or more processors, based on the amount of the total area of the pupil which is not obscured by the eyelid and the total area of the pupil, whether the eye of the user is closed.

14. The method for determining eye openness with an eye tracking device of claim 13, wherein:

when the total area of the pupil which is not obscured is greater than a remainder of the pupil, the eye of the user is open.

15. The method for determining eye openness with an eye tracking device of claim 13, wherein:

when the total area of the pupil which is not obscured is less than a remainder of the pupil, the eye of the user is closed.

16. The method for determining eye openness with an eye tracking device of claim 13, wherein the method further comprises:

determining, with the one or more processors, based on the image of the eye, that a center of the pupil is obscured by either eyelid; and

determining, with the one or more processors, based on the center of the pupil being obscured by either eyelid, that the eye of the user is closed.

17. The method for determining eye openness with an eye tracking device of claim 13, wherein the method further comprises:

determining, with the one or more processors, based on the image of the eye, a maximum openness over time between either eyelids; and

determining, with the one or more processors, whether the eye of the user is closed based on the maximum openness over time.

18. The method for determining eye openness with an eye tracking device of claim 17, wherein determining whether the eye of the user is closed based on the maximum openness over time comprises:

determining, based on the image of the eye, that an openness of the eye is less than a predefined proportion of the maximum openness over time.

19. A device, comprising:

a processor; and

a non-transitory computer-readable medium communicatively coupled to the processor, wherein the processor is configured to execute program code stored in the non-transitory computer-readable medium and thereby perform operations comprising;

determining, for at least a portion of pixels of an image sensor of an eye tracking device, a baseline value based on a first sum of intensity of the pixels for a first time period when an eye of a user is open;

determining, for the at least portion of pixels of the image sensor of the eye tracking device, a blink amplitude value based on a second sum of intensity of the pixels for a second time period when the eye of the user is closed, wherein a sum of intensity of the pixels when the eye of the user is closed is higher than a sum of intensity of the pixels when the eye of the user is open;

determining a threshold value based on the baseline value and the blink amplitude value;

determining, for the at least portion of pixels of the image sensor of the eye tracking device, during a third time period, a third sum of intensity of the pixels; and

determining, based upon the third sum exceeding the threshold value, that the eye of the user is closed.

20. The device of claim 19, wherein the threshold value is determined by adding the baseline value with a product of a threshold fraction and a difference between the baseline value and the blink amplitude value.