Information processing on a head-mountable device

• US 9,354,445 B1
• Filed: 08/31/2012
• Issued: 05/31/2016
• Est. Priority Date: 09/16/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Weaver, Joshua, Starner, Thad Eugene, Biffle, Cliff L., Keyes, Edward Allen
• Primary Examiner(s): Nghiem, Michael
• Assistant Examiner(s): Ngo, Peter
• Application Number:
  US13/601,900
• Time in Patent Office: 1,369 Days
• Field of Search: 702/104, 351/209, 351/210
• US Class Current: 1/1
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

Example methods and systems determine a position of a portion of a human eye based on electromagnetic radiation reflected from the surface of the human eye. A sensor associated with a computing device can be calibrated in response to an event. The computing device can receive data indicative of electromagnetic radiation reflected from a human eye. The computing device can determine a position of a portion of the human eye based on the received data indicative of electromagnetic radiation. The computing device can generate an indication including the position of the portion of the human eye. The computing device can transmit the indication from the computing device. In some embodiments, the data indicative of electromagnetic information can be provided by electromagnetic emitter/sensors mounted on a wearable computing device directed toward a human eye of a wearer of the wearable computing device.

Claims

1. A method comprising:

determining, by a head-mountable device (HMD), a measure of ambient light at each of one or more ambient light sensors (ALSs) arranged on the HMD, wherein each ALS is configured to measure ambient light corresponding to a respective location of one of a plurality of electromagnetic-radiation emitter/sensors (EESs) arranged on the HMD, and wherein ambient light comprises electromagnetic radiation other than electromagnetic radiation emitted by the plurality of EESs;

based on the one or more determined measures of ambient light at the one or more ALSs, determining, by the HMD, (i) a first number of EESs to use for emitting electromagnetic radiation toward an eye and (ii) a second number of EESs to use for detecting electromagnetic radiation reflected from the eye;

selecting, by the HMD from among the plurality of EESs, a first group of particular EESs based at least on the first group having the first number of EESs;

selecting, by the HMD from among the plurality of EESs, a second group of particular EESs based at least on the second group having the second number of EESs;

causing, by the HMD, each of the particular EESs of the first group to emit electromagnetic radiation toward the eye;

receiving, by the HMD from the particular EESs of the second group, data indicative of electromagnetic radiation reflected from the eye; and

determining, by the HMD, a position of a portion of the eye based on the received data.

2. The method of claim 1, further comprising:

further based on the one or more determined measures of ambient light at the one or more ALSs, determining by the HMD, for each of the particular EESs of the first group, respective electromagnetic radiation characteristics to use when emitting electromagnetic radiation toward the eye,

wherein causing each of the particular EESs of the first group to emit electromagnetic radiation toward the eye comprises causing each of the particular EESs of the first group to emit toward the eye electromagnetic radiation having the determined respective electromagnetic radiation characteristics.

3. The method of claim 2, wherein the respective electromagnetic radiation characteristics comprise one or more of:

(i) a power at which the electromagnetic radiation is emitted toward the eye, (ii) a duration for which the electromagnetic radiation is emitted toward the eye, and (iii) a frequency at which the electromagnetic radiation is emitted toward the eye.

4. The method of claim 1, wherein the portion of the eye comprises a pupil of the eye.

5. The method of claim 1, wherein selecting the first group comprises for each particular EES from among the plurality of EESs:

(i) if the determined measure of ambient light at a particular ALS associated with the particular EES exceeds a threshold measure of ambient light, including the particular EES in the first group, and

(ii) if the determined measure of ambient light at the particular ALS associated with the particular EES is below the threshold measure of ambient light, excluding the particular EES from the first group.

6. The method of claim 1, wherein selecting the second group comprises for each particular EES from among the plurality of EESs:

(i) if the determined measure of ambient light at a particular ALS associated with the particular EES exceeds a threshold measure of ambient light, including the particular EES in the second group, and

(ii) if the determined measure of ambient light at the particular ALS associated with the particular EES is below the threshold measure of ambient light, excluding the particular EES from the second group.

7. A head-mountable device (HMD), comprising:

a processor;

a plurality of electromagnetic-radiation emitter/sensors (EESs);

one or more ambient light sensors (ALSs), wherein each ALS is configured to measure ambient light corresponding to a respective location of one of the plurality of EESs, and wherein ambient light comprises electromagnetic radiation other than electromagnetic radiation emitted by the plurality of EESs;

a non-transitory computer-readable medium; and

program instructions stored on the non-transitory computer-readable medium and executable by the processor to cause the HMD to perform functions comprising;

determining a measure of ambient light at each of the one or more ALSs;

based on the one or more determined measures of ambient light at the one or more ALSs, determining (i) a first number of EESs to use for emitting electromagnetic radiation toward an eye and (ii) a second number of EESs to use for detecting electromagnetic radiation reflected from the eye;

selecting, from among the plurality EESs, a first group of particular EESs based at least on the first group having the first number of EESs;

selecting, from among the plurality of EESs, a second group of particular EESs based at least on the second group having the second number of EESs;

causing each of the particular EESs of the first group to emit electromagnetic radiation toward the eye;

receiving, from the particular EESs of the second group, data indicative of electromagnetic radiation reflected from the eye; and

determining a position of a portion of the eye based on the received data.

8. The HMD of claim 7, the functions further comprising:

further based on the one or more determined measures of ambient light at the one or more ALSs, determine for each of the particular EESs of the first group, respective electromagnetic radiation characteristics to use when emitting electromagnetic radiation toward the eye,

wherein causing each of the particular EESs of the first group to emit electromagnetic radiation toward the eye comprises causing each of the particular EESs of the first group to emit toward the eye electromagnetic radiation having the determined respective electromagnetic radiation characteristics.

9. The HMD of claim 8, wherein the respective electromagnetic radiation characteristics comprise one or more of:

(i) a power at which the electromagnetic radiation is emitted toward the eye, (ii) a duration for which the electromagnetic radiation is emitted toward the eye, and (iii) a frequency at which the electromagnetic radiation is emitted toward the eye.

10. The HMD of claim 7, wherein selecting the first group comprises for each particular EES from among the plurality of EESs:

(i) if the determined measure of ambient light at a particular ALS associated with the particular EES exceeds a threshold measure of ambient light, including the particular EES in the first group, and

(ii) if the determined measure of ambient light at the particular ALS associated with the particular EES is below the threshold measure of ambient light, excluding the particular EES from the first group.

11. The HMD of claim 7, wherein selecting the second group comprises for each particular EES from among the plurality of EESs:

(i) if the determined measure of ambient light at a particular ALS associated with the particular EES exceeds a threshold measure of ambient light, including the particular EES in the second group, and

(ii) if the determined measure of ambient light at the particular ALS associated with the particular EES is below the threshold measure of ambient light, excluding the particular EES from the second group.

12. The HMD of claim 7, wherein the portion of the eye comprises a pupil of the eye.

13. An article of manufacture including a non-transitory computer-readable medium having instructions stored thereon that, if the instructions are executed by a head-mountable device (HMD), cause the HMD to perform functions comprising:

determining a measure of ambient light at each of one or more ambient light sensors (ALSs) arranged on the HMD, wherein each ALS is configured to measure ambient light corresponding to a respective location of one of a plurality of electromagnetic-radiation emitter/sensors (EESs), and wherein ambient light comprises electromagnetic radiation other than electromagnetic radiation emitted by the plurality of EESs;

based on the one or more determined measures of ambient light at the one or more ALSs, determining (i) a first number of EESs to use for emitting electromagnetic radiation toward an eye and (ii) a second number of EESs to use for detecting electromagnetic radiation reflected from the eye;

selecting, from among the plurality EESs, a first group of one or more particular EESs based at least on the first group having the first number of EESs;

selecting, from among the plurality of EESs, a second group of particular EESs based at least on the second group having the second number of EESs;

causing each of the particular EESs of the first group to emit electromagnetic radiation toward the eye;

receiving, from the particular EESs of the second group, data indicative of electromagnetic radiation reflected from the eye; and

determining a position of a portion of the eye based on the received data.

14. The article of manufacture of claim 13, wherein the functions further comprise:

further based on the one or more determined measures of ambient light at the one or more ALSs, determining, for each of the particular EESs of the first group, respective electromagnetic radiation characteristics to use when emitting electromagnetic radiation toward the eye,

wherein causing each of the particular EESs of the first group to emit electromagnetic radiation toward the eye comprises causing each of the particular EESs of the first group to emit toward the eye electromagnetic radiation having the determined respective electromagnetic radiation characteristics.

15. The article of manufacture of claim 14, wherein the respective electromagnetic radiation characteristics comprise one or more of:

(i) a power at which the electromagnetic radiation is emitted toward the eye, (ii) a duration for which the electromagnetic radiation is emitted toward the eye, and (iii) a frequency at which the electromagnetic radiation is emitted toward the eye.

16. The article of manufacture of claim 13, wherein the portion of the eye comprises a pupil of the eye.

17. The article of manufacture of claim 13, wherein selecting the first group comprises for each particular EES from among the plurality of EESs:

(i) if the determined measure of ambient light at a particular ALS associated with the particular EES exceeds a threshold measure of ambient light, including the particular EES in the first group, and

(ii) if the determined measure of ambient light at the particular ALS associated with the particular EES is below the threshold measure of ambient light, excluding the particular EES from the first group.

18. The article of manufacture of claim 13, wherein selecting the second group comprises for each particular EES from among the plurality of EESs:

(i) if the determined measure of ambient light at a particular ALS associated with the particular EES exceeds a threshold measure of ambient light, including the particular EES in the second group, and

(ii) if the determined measure of ambient light at the particular ALS associated with the particular EES is below the threshold measure of ambient light, excluding the particular EES from the second group.