Proximity Sensing for Wink Detection

• US 20130257709A1
• Filed: 07/17/2012
• Published: 10/03/2013
• Est. Priority Date: 04/02/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Raffle, Hayes Solos, Johnson, Michael Patrick, Sparks, David, Wu, Bo
• Granted Patent: US 9,128,522 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

This disclosure relates to proximity sensing for wink detection. An illustrative method includes receiving data from a receiver portion of a proximity sensor. The receiver portion is disposed at a side section of a head-mountable device (HMD). When a wearer wears the HMD, the receiver portion is arranged to receive light reflected from an eye area of the wearer, the proximity sensor detects a movement of the eye area, and the data represents the movement. The method includes determining that the data corresponds to a wink gesture. The method also includes selecting a computing action to perform, based on the wink gesture. The method further includes performing the computing action.

Claims

1. A method comprising:

receiving data from a receiver portion of a proximity sensor, wherein the receiver portion is disposed at a side section of a head-mountable device (HMD), and wherein, when a wearer wears the HMD, (i) the receiver portion is arranged to receive light reflected from an eye area of the wearer, (ii) the proximity sensor detects a movement of the eye area, and (iii) the data represents the movement;

determining that the data corresponds to a wink gesture;

selecting a computing action to perform, based on the wink gesture; and

performing the computing action.

2. The method of claim 1, further comprising determining a gaze direction, based on the data.

3. The method of claim 2, wherein the determining the gaze direction comprises:

determining a first intensity level and a second intensity level, based on the data, wherein the first intensity level corresponds to an eye-open state, and wherein the second intensity level corresponds to an eye-closed state;

determining whether a difference between the first intensity level and the second intensity level exceeds a threshold; and

based on a determination that the difference exceeds the threshold, determining that the gaze direction is an intermediate vertical direction.

4. The method of claim 2, wherein the determining the gaze direction comprises:

determining a first intensity level and a second intensity level, based on the data, wherein the first intensity level corresponds to an eye-open state, and wherein the second intensity level corresponds to an eye-closed state;

determining whether a difference between the first intensity level and the second intensity level does not exceed a threshold; and

based on a determination that the difference does not exceed the threshold, determining that the gaze direction is either an upward direction or a downward direction.

5. The method of claim 2, further comprising:

storing a plurality of intensity levels for an eye-open state, wherein each intensity level in the plurality is associated with a corresponding gaze direction; and

performing a calibration of the plurality of intensity levels for a context in which the method is used.

6. The method of claim 2, wherein the selecting the computing action to perform is further based on the gaze direction.

7. The method of claim 2, further comprising determining whether the gaze direction corresponds to at least one of a predetermined set of directions, wherein the selecting the computing action to perform is further based on a determination that the gaze direction corresponds to the at least one of the predetermined set of directions.

8. The method of claim 1, further comprising causing infrared light having a predetermined characteristic to be provided to the eye area, wherein the data indicates the predetermined characteristic.

9. The method of claim 1, wherein the computing action comprises causing an image capture device to capture image data.

10. A head-mountable device (HMD) comprising:

a support structure comprising a front section and a side section, wherein the side section is adapted to receive a receiver portion of a proximity sensor such that, when a wearer wears the HMD, (i) the receiver portion is arranged to receive light reflected from an eye area of the wearer and (ii) the proximity sensor is configured to detect a movement of the eye area;

a computer-readable medium; and

program instructions stored to the computer-readable medium and executable by at least one processor to perform functions comprising;

receiving data from the receiver portion, wherein, when the HMD is worn, the data represents the movement of the eye area;

determining that the data corresponds to a wink gesture;

selecting a computing action to perform, based on the wink gesture; and

performing the computing action.

11. The HMD of claim 10, further comprising the proximity sensor.

12. The HMD of claim 10, wherein the side section is further adapted to receive the receiver portion of the proximity sensor such that, when the wearer wears the HMD, the receiver portion is aligned at an oblique angle with respect to the eye area.

13. The HMD of claim 10, further comprising a light source, wherein when the wearer wears the HMD, the light source is adapted to provide light to the eye area.

14. The HMD of claim 13, wherein when the wearer wears the HMD, the light source is adapted to provide light to one or both of an upper-eyelid area of the eye area and a lower-eyelid area of the eye area, and wherein the data represents at least one characteristic of one or both of light reflected from the upper-eyelid area and light reflected from the lower-eyelid area.

15. The HMD of claim 13, wherein the light is infrared light.

16. The HMD of claim 10, wherein the functions further comprise determining a gaze direction, based on the data.

17. A non-transitory computer-readable medium having stored thereon program instructions that, upon execution by at least one processor, cause the at least one processor to perform functions comprising:

receiving data from a receiver portion of a proximity sensor, wherein the receiver portion is disposed at a side section of a head-mountable device (HMD), and wherein, when a wearer wears the HMD, (i) the receiver portion is arranged to receive light reflected from an eye area of the wearer, (ii) the proximity sensor detects a movement of the eye area, and (iii) the data represents the movement;

determining that the data corresponds to a wink gesture;

determining a gaze direction, based on the data;

selecting a computing action to perform, based on one or both of the wink gesture and the gaze direction; and

performing the computing action.

18. The non-transitory computer-readable medium of claim 17, wherein the determining the gaze direction comprises:

determining a first intensity level and a second intensity level, based on the data, wherein the first intensity level corresponds to an eye-open state, and wherein the second intensity level corresponds to an eye-closed state;

determining whether a difference between the first intensity level and the second intensity level exceeds a threshold; and

based on a determination that the difference between the first intensity level and the second intensity level exceeds the threshold, determining that the gaze direction is an intermediate vertical direction.

19. The non-transitory computer-readable medium of claim 17, wherein the determining the gaze direction comprises:

determining a first intensity level and a second intensity level, based on the data, wherein the first intensity level corresponds to an eye-open state, and wherein the second intensity level corresponds to an eye-closed state;

determining whether a difference between the first intensity level and the second intensity level does not exceed a threshold; and

based on a determination that the difference between the first intensity level and the second intensity level does not exceed the threshold, determining that the gaze direction is either an upward direction or a downward direction.

20. The non-transitory computer-readable medium of claim 17, wherein the functions further comprise determining whether the gaze direction corresponds to at least one of a predetermined set of directions, wherein the selecting the computing action to perform is further based on a determination that the gaze direction corresponds to the at least one of the predetermined set of directions.

21. The non-transitory computer-readable medium of claim 17, wherein the functions further comprise causing infrared light having a predetermined characteristic to be provided to the eye area, wherein the data indicates the predetermined characteristic.