Proximity sensing for input detection

• US 9,201,512 B1
• Filed: 07/16/2012
• Issued: 12/01/2015
• Est. Priority Date: 04/02/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Raffle, Hayes Solos, Zhao, Yong
• Primary Examiner(s): Merkoulova, Olga
• Application Number:
  US13/550,558
• Time in Patent Office: 1,233 Days
• Field of Search: 345/158, 351209-223, 340/575, 340/573.1, 340/576, 341/20, 341/21
• US Class Current: 1/1
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041

Abstract

Disclosed is a technique that can help to detect a blink of an eye and a direction along which the eye is oriented before, after, or during the blink. To this end, light data can be received from at least one light sensor. The light data indicates at least one characteristic of light reflected from an eye area. A blink event can be detected based on the light data. A gaze direction can be determined based on the blink event. At least one computing action can be performed based on the gaze direction.

Claims

1. A method comprising:

receiving light data from at least one light sensor, wherein the light data indicates at least one characteristic of light reflected from an eye area;

detecting a blink event based on the light data, wherein detecting the blink event comprises detecting, in a given portion of the light data, a first light-intensity level and a subsequent second light-intensity level, wherein the first light-intensity level is associated with an eye-open state, and wherein the second light-intensity level is associated with an eye-closed state;

determining a particular gaze direction based on the given portion of the light data, wherein determining the particular gaze direction comprises;

(i) determining a difference between the first and second light-intensity levels, and (ii) determining that the difference between the first and second light-intensity levels corresponds to the particular gaze direction; and

causing at least one computing action to be performed based on the particular gaze direction.

2. The method of claim 1, further comprising providing light to the eye area by way of one or more light sources.

3. The method of claim 1,

wherein determining that the difference between the first and second light-intensity levels corresponds to the particular gaze direction comprises (i) determining whether the difference between the first and second light-intensity levels exceeds a threshold, and (ii) determining that the particular gaze direction is an intermediate vertical direction in response to a determination that the difference between the first and second light-intensity levels exceeds the threshold.

4. The method of claim 1, direction based on the blink event comprises:

wherein determining that the difference between the first and second light-intensity levels corresponds to the particular gaze direction comprises (i) determining whether the difference between the first and second light-intensity levels exceeds a threshold, and (ii) determining that the particular gaze direction is either an upward direction or a downward direction in response to a determination that the difference between the first and second light-intensity levels is less than the threshold.

5. The method of claim 1, further comprising:

storing a plurality of light-intensity levels each corresponding to the eye-open state, wherein each of the plurality of light-intensity levels is associated with a corresponding gaze direction; and

calibrating each of the plurality of light-intensity levels for a context in which the method is used.

6. The method of claim 1, wherein causing the at least one computing action to be performed based on the particular gaze direction comprises determining the at least one computing action based on the particular gaze direction.

7. The method of claim 6, wherein determining the at least one computing action based on the particular gaze direction comprises:

determining whether the particular gaze direction corresponds to at least one of a predetermined set of directions; and

determining the at least one computing action in response to a determination that the particular gaze direction corresponds to the at least one of the predetermined set of directions.

8. The method of claim 1, wherein determining the at least one computing action further comprises:

determining whether the blink event is an intentional input blinking based on the light data; and

determining the at least one computing action in response to a determination that the blink event is the intentional input blinking.

9. A system comprising:

at least one light sensor;

a computer-readable medium;

program instructions stored to the computer-readable medium and executable by at least one processor to perform functions comprising;

(i) receiving light data from the at least one light sensor, wherein the light data indicates at least one characteristic of light reflected from an eye area;

(ii) detecting a blink event based on the light data, wherein detecting the blink event comprises detecting, in a given portion of the light data, a first light-intensity level and a subsequent second light-intensity level, wherein the first light-intensity level is associated with an eye-open state, and wherein the second light-intensity level is associated with an eye-closed state;

(iii) determining a particular gaze direction based on the given portion of the light data, wherein determining the particular gaze direction comprises;

(a) determining a difference between the first and second light-intensity levels, and (b) determining that the difference between the first and second light-intensity levels corresponds to the particular gaze direction; and

(iv) causing at least one computing action to be performed based on the particular gaze direction.

10. The system of claim 9, further comprising a light source, wherein the light source is operable to provide light to the eye area.

11. The system of claim 10, wherein the light source is operable to provide light to at least one of an upper-eyelid area of the eye area or a lower-eyelid area of the eye area, and wherein the light data indicates characteristics of at least one of light reflected from the upper-eyelid area and light reflected from the lower-eyelid area.

12. The system of claim 9, wherein the at least one light sensor is aligned at an oblique angle with respect to the eye area.

13. The system of claim 9, further comprising a head-mountable device, wherein the computer-readable medium is integrated in the head-mountable device, and wherein the at least one light sensor is affixed to the head-mountable device.

14. The system of claim 9, further comprising a motion sensor, wherein the program instructions are further executable by the at least one processor to perform functions comprising receiving motion data by way of the motion sensor, and wherein detecting the blink event is further based on the motion data.

15. The system of claim 9,

wherein determining that the difference between the first and second light-intensity levels corresponds to the particular gaze direction comprises (i) determining whether the difference between the first and second light-intensity levels exceeds a threshold, and (ii) determining that the particular gaze direction is an intermediate vertical direction in response to a determination that the difference between the first and second light-intensity levels exceeds the threshold.

16. The system of claim 9,

wherein determining that the difference between the first and second light-intensity levels corresponds to the particular gaze direction comprises (i) determining whether the difference between the first and second light-intensity levels exceeds a threshold, and (ii) determining that the particular gaze direction is either an upward direction or a downward direction in response to a determination that the difference between the first and second light-intensity levels is less than the threshold.

17. The system of claim 9, wherein causing the at least one computing action to be performed based on the particular gaze direction comprises determining the at least one computing action based on the particular gaze direction.

18. The system of claim 17, wherein the program instructions are further executable by the at least one processor to perform functions comprising:

determine whether the particular gaze direction corresponds to at least one of a predetermined set of directions; and

determine the at least one computing action in response to a determination that the particular gaze direction corresponds to the at least one of the predetermined set of directions.

19. A non-transitory computer-readable medium having stored thereon program instructions that, upon execution by at least one processor, cause a computing device to perform functions comprising:

receiving light data from at least one light sensor, wherein the light data indicates at least one characteristic of light reflected from an eye area;

detecting a blink event based on the light data, wherein detecting the blink event comprises detecting, in a given portion of the light data, a first light-intensity level and a subsequent second light-intensity level, wherein the first light-intensity level is associated with an eye-open state, and wherein the second light-intensity level is associated with an eye-closed state;

determining a particular gaze direction based on the given portion of the light data, wherein determining the gaze direction comprises;

(i) determining a difference between the first and second light-intensity levels, and (ii) determining that the difference between the first and second light-intensity levels corresponds to the particular gaze direction; and

causing at least one computing action to be performed based on the particular gaze direction.

20. The non-transitory computer-readable medium of claim 19, wherein the functions further comprise providing light to the eye area by way of one or more light sources.

21. The non-transitory computer-readable medium of claim 19, wherein causing the at least one computing action to be performed based on the particular gaze direction comprises:

determining whether the particular gaze direction corresponds to at least one of a predetermined set of directions; and

determining the at least one computing action in response to a determination that the particular gaze direction corresponds to the at least one of the predetermined set of directions.

22. The non-transitory computer-readable medium of claim 19, wherein the functions further comprise:

determining the particular gaze direction by (i) comparing the first light-intensity level to a plurality of predetermined light-intensity levels, wherein each predetermined light-intensity level corresponds to a given gaze direction, and (ii) based on the comparison, determining that the first light-intensity level corresponds to the particular gaze direction.