Eye Gaze Detection to Determine Speed of Image Movement

• US 20130106674A1
• Filed: 11/02/2011
• Published: 05/02/2013
• Est. Priority Date: 11/02/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Wheeler, Aaron Joseph, Raffle, Hayes Solos
• Granted Patent: US 8,970,452 B2
• Time in Patent Office: Days
• US Class Current: 345/8
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A wearable computing device or a head-mounted display (HMD) may be configured to track the gaze axis of an eye of the wearer. In particular, the device may be configured to observe movement of a wearer'"'"'s pupil and, based on the movement, determine inputs to a user interface. For example, using eye gaze detection, the HMD may change a tracking rate of a displayed virtual image based on where the user is looking. Gazing at the center of the HMD field of view may, for instance, allow for fine movements of the virtual display. Gazing near an edge of the HMD field of view may provide coarser movements.

Claims

1. A head-mounted display (HMD), comprising:

a head-mounted support;

an optical system attached to the head-mounted support, wherein the optical system comprises a display panel configured to generate a virtual image, wherein the virtual image is viewable from a viewing location;

an infrared light source configured to illuminate the viewing location with infrared light such that infrared light is reflected from the viewing location as reflected infrared light;

a camera configured to image the viewing location by collecting the reflected infrared light; and

a computer configured to determine a gaze axis based on one or more images of the viewing location obtained by the camera and control the display panel to move the virtual images within a field of view based on the gaze axis, a reference axis related to the HMD, and a tracking rate.

2. The HMD of claim 1, wherein the viewing location corresponds to a location of at least one eye of a wearer of the HMD.

3. The HMD of claim 2, wherein the computer is configured to determine the location of the pupil of the at least one eye of a wearer of the HMD from the received images.

4. The HMD of claim 1, wherein the optical system further comprises a visible light source.

5. The HMD of claim 2, wherein the display panel is configured to generate the virtual image by spatially modulating visible light from the visible light source.

6. The HMD of claim 1, wherein the infrared light source comprises an infrared light-emitting diode (LED).

7. The HMD of claim 1, wherein the camera comprises an infrared camera.

8. The HMD of claim 1, wherein the reference axis related to the HMD corresponds to a central axis of the field of view.

9. The HMD of claim 1, wherein the reference axis related to the HMD comprises an axis that extends from the viewing location to a target object.

10. The HMD of claim 1, wherein the reference axis related to the HMD corresponds to an orientation of the HMD.

11. A method, comprising:

determining a gaze axis within a field of view of a head-mounted display (HMD), wherein the HMD is configured to display virtual images within the field of view;

determining a reference axis related to the HMD;

adjusting a tracking rate based on the gaze axis and the reference axis; and

moving the virtual images within the field of view based on;

the gaze axis;

the reference axis; and

the tracking rate.

12. The method of claim 11, wherein determining the gaze axis comprises:

obtaining at least one image of at least one eye of a wearer of the HMD; and

determining the gaze axis from the at least one image.

13. The method of claim 12, wherein the at least one image further comprises at least one image of at least one eye pupil of a wearer of the HMD.

14. The method of claim 11, wherein the reference axis related to the HMD corresponds to a central axis of the field of view.

15. The method of claim 11, wherein the reference axis related to the HMD comprises an axis that extends from the viewing location to a target object.

16. The method of claim 11, wherein the reference axis related to the HMD corresponds to an orientation of the HMD.

17. The method of claim 11, wherein adjusting the tracking rate further comprises:

determining an angle difference between the gaze axis and the reference axis; and

adjusting the tracking rate based on the angle difference.

18. The method of claim 17, wherein adjusting the tracking rate further comprises adjusting the tracking rate proportionally based on the angle difference.

19. The method of claim 11, wherein moving the virtual images within the field of view further comprises controlling the HMD to pan the virtual images within the field of view.

20. The method of claim 11, wherein the virtual images comprise one-dimensional virtual objects.

21. The method of claim 11, wherein the virtual images comprise two-dimensional virtual objects.

22. A non-transitory computer readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions, comprising:

receiving eye-tracking images from a head-mounted display (HMD), wherein the HMD is configured to display virtual images within a field of view;

determining a gaze axis from the eye-tracking images;

determining a reference axis related to the HMD;

calculating an angle difference between the gaze axis and the reference axis;

adjusting a tracking rate based on the angle difference; and

controlling the HMD to display the virtual images based upon the gaze axis, the reference axis and the tracking rate.

23. The non-transitory computer readable medium of claim 22, wherein determining a reference axis comprises receiving information about a target object.

24. The non-transitory computer readable medium of claim 22, wherein determining a reference axis comprises receiving global positioning system (GPS) data.

25. The non-transitory computer readable medium of claim 22, wherein determining a reference axis comprises receiving accelerometer data.

26. The non-transitory computer readable medium of claim 22, wherein determining a reference axis comprises receiving gyroscope data.

27. The non-transitory computer readable medium of claim 22, wherein adjusting the tracking rate further comprises adjusting the tracking rate proportionally based on the angle difference.

28. The non-transitory computer readable medium of claim 22, wherein controlling the HMD to display the virtual images further comprises controlling the HMD to pan the virtual images within the field of view.

29. The non-transitory computer readable medium of claim 22, wherein the virtual images comprise one-dimensional virtual objects.

30. The non-transitory computer readable medium of claim 22, wherein the virtual images comprise two-dimensional virtual objects.