User interface method

• US 9,096,920 B1
• Filed: 03/22/2012
• Issued: 08/04/2015
• Est. Priority Date: 03/22/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Gomez, Luis Ricardo Prada, Kauffmann, Alejandro
• Primary Examiner(s): Elahi, Shan
• Application Number:
  US13/427,583
• Time in Patent Office: 1,230 Days
• Field of Search: 348/78
• US Class Current: 1/1
• CPC Class Codes: C23C 4/06 Metallic material
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 2203/04804 Transparency, e.g. transparent or translucent windows
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A computer-implemented method includes detecting, at a wearable computing device, a first direction of a first stare, wherein the wearable computing device includes a head-mountable display unit, identifying a target based on the detected first direction, and based on a determination that a first time duration of the first stare is greater than or equal to a first predetermined time threshold, identifying information relevant to the target and displaying the identified information on the display unit. Subsequent to displaying the identified information, the method includes detecting a second stare that is directed at the target or at the displayed information, and based on a determination that a second time duration of the second stare is greater than or equal to a second predetermined time threshold, identifying additional information relevant to the target, and displaying the additional information on the display unit.

Claims

1. A computer-implemented method comprising:

detecting, at a wearable computing device, a first direction of a first stare, wherein the wearable computing device includes a head-mountable display unit;

identifying a target based on the detected first direction;

based on a determination that a first time duration of the first stare is greater than or equal to a first predetermined time threshold, identifying information relevant to the target and displaying the identified information on the head-mountable display unit;

subsequent to displaying the identified information, detecting a second stare that is directed at the target or at the displayed information, wherein the first stare and the second stare are contiguous in time; and

based on a determination that a second time duration of the second stare is greater than or equal to a second predetermined time threshold, identifying additional information relevant to the target, and displaying the additional information on the head-mountable display unit.

2. The method of claim 1, wherein detecting the first direction of the first stare comprises:

obtaining at least one image of each eye of a wearer of the wearable computing device; and

determining the first stare direction from the at least one image of each eye.

3. The method of claim 2, wherein obtaining the at least one image of each eye of the wearer of the wearable computing device comprises:

illuminating each eye with an infrared light source; and

imaging each eye with a camera.

4. The method of claim 1, wherein identifying the target comprises:

capturing an image of the target; and

performing a target recognition based on the captured image.

5. The method of claim 1, wherein the head mountable display unit comprises a see-through display unit.

6. The method of claim 1, wherein the wearable computing device is a head-mounted device.

7. The method of claim 1 further comprising:

progressively displaying more additional information about the target while the stare continues.

8. A computer-implemented method comprising:

detecting, at a wearable computing device, a direction of a stare, wherein the wearable computing device includes a head-mountable display unit;

identifying a target based on the detected direction;

based on a determination that a first time duration of the stare is greater than or equal to a first predetermined time threshold, identifying information relevant to the target and displaying the identified information on the display unit;

subsequent to displaying the identified information, determining that a second time duration of the stare is greater than or equal to a second predetermined time threshold;

based on a determination that the second time duration of the stare is greater than or equal to the second predetermined time threshold, identifying additional information relevant to the target, and displaying the additional information on the display unit.

9. The method of claim 8, wherein detecting the direction of the stare comprises:

obtaining at least one image of each eye of a wearer of the wearable computing device; and

determining the stare direction from the at least one image of each eye.

10. The method of claim 9, wherein obtaining the at least one image of each eye of the wearer of the wearable computing device comprises:

illuminating each eye with an infrared light source; and

imaging each eye with a camera.

11. The method of claim 8, wherein identifying the target comprises:

capturing an image of the target; and

performing a target recognition based on the captured image.

12. The method of claim 8, wherein the head-mountable display unit comprises a see-through display unit.

13. The method of claim 8 further comprising:

progressively displaying more additional information about the target while the stare continues.

14. The method of claim 8, wherein the wearable computing device is a head-mounted device.

15. A computer-implemented method comprising:

detecting, at a wearable computing device, a direction of a stare, wherein the wearable computing device includes a head-mountable display unit;

identifying a target based on the detected direction;

based on a determination that a first time duration of the stare is greater than or equal to a first predetermined time threshold, identifying information relevant to the target and displaying the identified information on the head-mountable display unit; and

subsequent to displaying the identified information, progressively displaying additional information about the target while the stare continues, comprising;

determining that a second time duration of the stare is greater than or equal to a second predetermined time threshold; and

based on a determination that the second time duration of the stare is greater than or equal to the second predetermined time threshold, identifying additional information relevant to the target, and displaying the additional information on the head-mountable display unit.

16. The method of claim 15, wherein detecting the direction of the stare comprises:

obtaining at least one image of each eye of a wearer of the wearable computing device; and

determining the stare direction from the at least one image of each eye.

17. The method of claim 16, wherein obtaining the at least one image of each eye of the wearer of the wearable computing device comprises:

illuminating each eye with an infrared light source; and

imaging each eye with a camera.

18. The method of claim 15, wherein identifying the target comprises:

capturing an image of the target; and

performing a target recognition based on the captured image.

19. The method of claim 15, wherein the head-mountable display unit comprises a see-through display unit.

20. The method of claim 15, wherein the wearable computing device is a head-mounted device.