USER FOCUS CONTROLLED DIRECTIONAL USER INPUT

• US 20140372944A1
• Filed: 06/12/2013
• Published: 12/18/2014
• Est. Priority Date: 06/12/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Mulcahy, Kathleen, Cole, Jeff, Burns, Aaron, Omotani, Todd, Williams, Felicia, Psiaki, Tim
• Granted Patent: US 9,710,130 B2
• Time in Patent Office: Days
• US Class Current: 715/810
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser

Abstract

A user input for a near-eye, see-through display device is disclosed. Hands-free user input in an augmented reality environment is provided for. A user can provide input by moving the orientation of their head. For example, the user could rotate their head. In one aspect, a user can provide input by moving their eye gaze along a direction. In one aspect, when the user directs their attention at a user interface symbol, a handle extends away from the user interface symbol. The handle may serve as a type of selection device such that if the user directs their attention along the handle, away from the user interface symbol, a selection can be made. “As one example, the selection causes a spoke menu to appear which the user can select by rotating their head such that the system determines the user is looking along the spoke away from a central hub.

Claims

1. A method of providing a user interface, the method comprising:

presenting a user interface symbol in a see-through, near-eye display;

determining focus of a user that is wearing the see-through, near-eye display;

displaying a handle in the see-through, near-eye display that extends away from the user interface symbol in response to determining that the user is focused on the user interface symbol;

determining that the user'"'"'s focus has shifted along the handle away from the user interface symbol; and

activating a user interface element in response to determining that the user'"'"'s focus has shifted away from the user interface symbol along the handle to meet a selection criterion.

2. The method of claim 1, wherein the activating a user interface element comprises:

receiving a selection of an input associated with the user interface symbol.

3. The method of claim 1, wherein activating a user interface element comprises:

opening a menu associated with the user interface symbol.

4. The method of claim 3, wherein the opening a menu associated with the user interface symbol includes spawning a spokes menu that includes a plurality of menu items, each of the menu items has a spoke associated with it, and further comprising:

selecting a first menu item of the plurality of menu items in response to the user moving their focus along the spoke associated with the first menu item in the direction along the spoke towards the first menu item.

5. The method of claim 4, further comprising:

displaying at least a portion of the spoke that is associated with the first menu item in response to determining that the user'"'"'s focus moved at least partially towards the first menu item.

6. The method of claim 1, wherein the activating a user interface element in response to determining that the user'"'"'s focus has shifted away from the user interface symbol along the handle to meet a selection criterion includes:

tracking the user'"'"'s head orientation in a direction away from the user interface symbol.

7. The method of claim 1, wherein the activating a user interface element in response to determining that the user'"'"'s focus has shifted away from the user interface symbol along the handle to meet a selection criterion includes:

tracking the user'"'"'s eye movement in a direction away from the user interface symbol.

8. The method of claim 1, wherein the presenting a user interface symbol in a see-through near eye display includes:

displaying the user interface symbol in the see-through near eye display such that it appears to be on a real world object or a holographic image being displayed in the see-through near eye display.

9. The method of claim 8, wherein the displaying the user interface symbol in the see-through near eye display such that it appears to be on a real world object or a holographic image being displayed in the see-through near eye display is performed in response to the user looking at a point in the real world at which the user interface symbol is to appear to be when presented in the see-through near eye display.

10. A display system comprising:

a see-through, near-eye display device;

logic in communication with the display device, the logic is configured to;

present a user interface symbol in the see-through, near-eye display device;

track eye gaze or head orientation of a user wearing the see-through, near-eye display device;

display a handle in the see-through, near-eye display device that extends away from the user interface symbol in response to determining that either the eye gaze or the head orientation indicates that the user is looking at the user interface symbol; and

activate a user interface element in response to determining that the eye gaze or head orientation indicates that the user is looking away from the user interface symbol along the handle to meet a selection criterion.

11. The system of claim 10, wherein the logic being configured to activate a user interface element comprises the logic being configured to:

present a spokes menu that includes a plurality of menu items, each of the menu items has a spoke associated with it.

12. The system of claim 11, further comprising logic configured to:

receive a selection of a first menu item of the plurality of menu items in response to determining that the user moves their head orientation or eye gaze along a first spoke associated with the first menu item in the direction along the spoke towards the first menu item to meet a selection criterion.

13. The system of claim 12, further comprising logic configured to:

progressively present more of the first spoke as the user moves their head orientation or eye gaze further towards the first menu item.

14. The system of claim 10, wherein the logic being configured to display a handle and the logic configured to activate a user interface element includes logic being configured to:

display the handle in the see-through, near-eye display device that extends away from the user interface symbol in response to determining that the head orientation indicates that the user is looking at the user interface symbol; and

present the spokes menu in response to determining that the user'"'"'s head orientation indicates that the user is looking away from the user interface symbol along the handle to meet the selection criterion.

15. The system of claim 10, wherein the logic being configured to present a user interface symbol in the see-through, near-eye display device includes the logic being configured to:

display the user interface symbol in the see-through near eye display such that it appears to be on a real world object or a holographic image being displayed in the see-through near eye display, the user interface element is associated with the real world object or the holographic image on which the user interface symbol appears.

16. A method of providing a user interface in a see-through, near-eye display device, the method comprising:

presenting a user interface symbol in the see-through, near-eye display device;

tracking a head orientation of a user wearing the see-through, near-eye display device;

displaying a handle in the see-through, near-eye display device that extends away from the user interface symbol in response to determining, based on the head orientation, that the user is looking in the general direction of the user interface symbol; and

spawning a spoke menu in the see-through, near-eye display device in response to determining that the user'"'"'s head orientation indicates that the user has shifted from looking at the user interface symbol along the handle, the spoke menu includes a plurality of menu items around a central hub, each of the menu items has a spoke between it and the central hub.

17. The method of claim 16, further comprising:

progressively presenting more of a first spoke that is associated with a first of the menu items in response to the user changing their head orientation such that the user focus is out of a safe zone near the hub.

18. The method of claim 17, further comprising:

selecting the first menu item in response to the first spoke reaching a selection point.

19. The method of claim 17, further comprising:

selecting the first menu item in response to the user changing their head orientation in the direction along the first spoke towards the first menu item until a selection criterion is met.

20. The method of claim 16, wherein the presenting a user interface symbol in the see-through, near-eye display device includes presenting the user interface symbol on or near a real world object that is viewable through the see-through near eye display or a virtual object being presented in the see-through near eye display, wherein the spoke menu includes menu items associated with the real world object or the virtual object.