Target positioning with gaze tracking

• US 9,244,539 B2
• Filed: 01/07/2014
• Issued: 01/26/2016
• Est. Priority Date: 01/07/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Venable, Morgan Kolya, Kerr, Bernard James, Wilairat, Weerapan
• Primary Examiner(s): GIESY, ADAM
• Application Number:
  US14/149,756
• Publication Number:
  US 20150193018A1
• Time in Patent Office: 749 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0156 with optionally usable elements
• CPC Class Codes: G02B 2027/0174 holographic
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2209/508 Monitor
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

Embodiments that relate to positioning a target indicator via a display system are disclosed. For example, one disclosed embodiment provides a method for positioning a target indicator using gaze tracking data having a coarse accuracy from a gaze tracking system of a computing device. Head pose data having a fine accuracy greater than the coarse accuracy is received from a head tracking system. Using the gaze tracking data, an approximate user gaze region within a display region is determined, and the target indicator is displayed at an initial location within the approximate user gaze region. A reposition input from the user is received. In response, subsequently received head pose data is used to calculate an adjusted location for the target indicator. The target indicator is then displayed at the adjusted location.

Claims

1. A head-mounted computing device for positioning a target indicator displayed via a display system, the head-mounted computing device operatively connected to the display system, the head-mounted computing device comprising:

a gaze tracking system for generating gaze tracking data having a coarse accuracy from a user wearing the head-mounted computing device;

a head tracking system for generating head pose data from the user, wherein the head pose data has a fine accuracy that is greater than the coarse accuracy of the gaze tracking data; and

a positioning module executed by a processor of the head-mounted computing device, the positioning module configured to;

using the gaze tracking data, determine an approximate user gaze region within a display region generated by the display system;

control the display system to display the target indicator at an initial location within the approximate user gaze region;

receive a reposition input from the user;

in response to receiving the reposition input, use subsequently received head pose data to calculate an adjusted location for the target indicator in the display region; and

control the display system to display the target indicator at the adjusted location in the display region.

2. The head-mounted computing device of claim 1, wherein the positioning module is further configured to:

after the target indicator is displayed at the adjusted location, receive a selection user input; and

in response to receiving the selection user input, select a selectable target located at the adjusted location.

3. The head-mounted computing device of claim 1, wherein the reposition input from the user is received from an input device selected from the group consisting of a hand-held mechanically actuatable stylus, a depth camera, a ring, a microphone, and the gaze tracking system.

4. The head-mounted computing device of claim 1, wherein the target indicator is not displayed until the reposition input is received.

5. The head-mounted computing device of claim 1, wherein the positioning module is further configured to, in response to receiving the reposition input, disable the gaze tracking system.

6. The head-mounted computing device of claim 1, wherein the display system comprises a near-eye display system integrated in the head-mounted computing device.

7. The head-mounted computing device of claim 1, wherein the display system comprises a screen in a physical environment and separate from the head-mounted computing device, the head-mounted computing device further comprises an optical sensor for capturing images of the physical environment including the screen, and the positioning module is further configured to:

use the images to establish a spatial relationship between the head-mounted computing device and the screen; and

use the spatial relationship in combination with the gaze tracking data to determine the approximate user gaze region within the display region of the screen.

8. The head-mounted computing device of claim 7, wherein the positioning module is further configured to receive a screen identifier from the screen.

9. The head-mounted computing device of claim 1, wherein the positioning module is further configured to:

receive secondary tracking data; and

in response to receiving the reposition input, use subsequently received secondary tracking data to calculate the adjusted location for the target indicator.

10. A method for positioning a target indicator displayed via a display system, the method comprising:

receiving gaze tracking data having a coarse accuracy from a gaze tracking system of a computing device;

receiving head pose data having a fine accuracy that is greater than the coarse accuracy of the gaze tracking data, the head pose data received from a head tracking system of the computing device;

using the gaze tracking data, determining an approximate user gaze region within a display region generated by the display system;

controlling the display system to display the target indicator at an initial location within the approximate user gaze region;

receiving a reposition input from the user;

in response to receiving the reposition input, using subsequently received head pose data to calculate an adjusted location for the target indicator in the display region; and

controlling the display system to display the target indicator at the adjusted location in the display region.

11. The method of claim 10, further comprising:

after the target indicator is displayed at the adjusted location, receiving a selection user input; and

in response to receiving the selection user input, selecting a selectable target located at the adjusted location.

12. The method of claim 10, wherein the reposition input from the user is received from an input device selected from the group consisting of a hand-held mechanically actuatable stylus, a depth camera, a ring, a microphone, and the gaze tracking system.

13. The method of claim 10, further comprising not displaying the target indicator until the reposition input is received.

14. The method of claim 10, further comprising disabling the gaze tracking system in response to receiving the reposition input.

15. The method of claim 10, wherein the display system comprises a screen integrated in the computing device.

16. The method of claim 10, wherein the display system comprises a screen in a physical environment and separate from the computing device, the computing device further comprises an optical sensor for capturing images of the physical environment including the screen, and the method further comprising:

using the images to establish a spatial relationship between the computing device and the screen; and

using the spatial relationship in combination with the gaze tracking data to determine the approximate user gaze region within the display region of the screen.

17. The method of claim 16, further comprising receiving a screen identifier from the screen.

18. The method of claim 10, further comprising:

receiving secondary tracking data; and

in response to receiving the reposition input, using the secondary tracking data to calculate the adjusted location for the target indicator.

19. A head-mounted display device for positioning a target indicator displayed via a display system, the head-mounted display device operatively connected to a computing device, the head-mounted display device comprising:

a gaze tracking system for generating gaze tracking data having a coarse accuracy from a user wearing the head-mounted computing device;

a head tracking system for generating head pose data from the user, wherein the head pose data has a fine accuracy that is greater than the coarse accuracy of the gaze tracking data; and

a positioning module executed by a processor of the computing device, the positioning module configured to;

using the gaze tracking data, determine an approximate user gaze region within a display region generated by the display system;

receive a reposition input from the user;

in response to receiving the reposition input, control the display system to display the target indicator at an initial location within the approximate user gaze region;

in response to receiving the reposition input, use subsequently received head pose data to calculate an adjusted location for the target indicator in the display region;

control the display system to display the target indicator at the adjusted location in the display region;

after the target indicator is displayed at the adjusted location, receive a selection user input; and

in response to receiving the selection user input, select a selectable target located at the adjusted location.

20. The head-mounted display device of claim 19, wherein the reposition input from the user is received from an input device selected from the group consisting of a hand-held mechanically actuatable stylus, a depth camera, a ring, a microphone, and the gaze tracking system.