MULTI-VISOR: MANAGING APPLICATIONS IN HEAD MOUNTED DISPLAYS

• US 20130007668A1
• Filed: 07/01/2011
• Published: 01/03/2013
• Est. Priority Date: 07/01/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Andrews, Anton Oguzhan Alford, Liu, James Chia-Ming, Maitlen, Craig R., Small, Sheridan
• Granted Patent: US 9,727,132 B2
• Time in Patent Office: Days
• US Class Current: 715/841
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user

Abstract

A system and method are provided that enhances a user'"'"'s experience when using a see-through near eye display device. A user interface is provided for a user to manage single or simultaneous applications in a head mounted device. Applications for the head mounted device may be activated or deactivated by the user via the user-interface. The user'"'"'s total field of view (TFOV) which accounts for a complete range of rotation and translation of the user'"'"'s head may be determined by tracking the user'"'"'s head position and rotation relative to the user'"'"'s body and an environment associated with the user. One region of the user'"'"'s TFOV (e.g., the right-hand side) may display an “application menu” including a list of applications that can be launched, and another region of the user'"'"'s TFOV (e.g., the left-hand side) may display an “active menu” including a list of applications currently running.

Claims

1. A method for providing a user interface to manage one or more applications in a head mounted display device associated with a user, comprising:

determining a total field of view of a user through the head mounted display device;

generating a first menu including a first list of one or more applications;

displaying the first menu in a first region of the total field of view of the user;

receiving a user selection to activate an application from the first list of one or more applications provided in the first menu;

generating a second menu including a second list of one or more applications, the second list of one or more applications including the application that has been activated from the first list of one or more applications provided in the first menu; and

displaying the second menu in a second region of the total field of view of the user.

2. The method of claim 1, wherein determining the total field of view of the user includes tracking head position and rotation of the user relative to a body of the user and relative to an environment associated with the user.

3. The method of claim 1 wherein displaying the first menu in the first region of the total field of view of the user includes selecting the first region within the total field of view of the user based on one or more configured rules.

4. The method of claim 3, wherein displaying the first menu in the first region of the total field of view of the user includes displaying the first menu on a right-hand side of the total field of view of the user.

5. The method of claim 1, wherein receiving a user selection to activate an application from the first list of one or more applications provided in the first menu includes detecting a user action.

6. The method of claim 5, wherein the user action includes the user dragging the application from the first list of one or more applications provided in the first menu over a primary field-of-view of the user.

7. The method of claim 5, wherein the user action includes a user input through a secondary device.

8. The method of claim 1, wherein the second region of the total field of view is opposite to the first region.

9. The method of claim 1, wherein the one or more applications provided in the second menu run concurrently and are overlaid on top of each other in a primary field-of-view of the user.

10. The method of claim 1, further comprising receiving a second user selection to de-activate an application from the second list of one or more applications provided in the second menu.

11. The method of claim 10, wherein the second user selection to de-activate an application from the second list of one or more applications provided in the second menu includes dragging the application from the second region to the first region across a primary field-of-view of the user.

12. A head mounted display device, comprising:

a display;

one or more sensors tracking head position and rotation of the user relative to a body of the user and relative to an environment associated with the user; and

at least one processing unit, in communication with the display, the one or more sensors, the processing unit including instructions for a processing device to perform a method including;

determining a total field of view of the user through the head mounted display device;

generating a first menu including a first list of one or more applications;

displaying the first menu in a first region of the total field of view of the user;

receiving a first user selection to activate an application from the first list of one or more applications provided in the first menu;

generating a second menu including a second list of one or more applications, the second list of one or more applications including the application that has been activated from the first list of one or more applications provided in the first menu; and

displaying the second menu in a second region of the total field of view of the user.

13. The head mounted display device of claim 12, wherein the first user selection includes dragging the application from the first list of one or more applications provided in the first menu over a primary field-of-view of the user.

14. The head mounted display device of claim 12, wherein the at least one processing unit receives a second user selection to de-activate an application from the second list of one or more applications provided in the second menu.

15. One or more processor readable storage devices having processor readable code embodied on said one or more processor readable storage devices, the processor readable code for programming one or more processors to perform a method comprising:

tracking head position and rotation of a user relative to a body of the user and relative to an environment associated with the user;

determining a total field of view of the user through a head mounted display device associated with the user based on the head position and rotation tracked;

generating a first menu including a first list of one or more applications that can be activated in the head mounted display device;

displaying the first menu in a first region of the total field of view of the user;

receiving a first user selection to activate an application from the first list of one or more applications provided in the first menu;

generating a second menu including a second list of one or more applications, the one or more applications of the second list of one or more applications running concurrently and overlaid on top of each other in a third region of the total field of view of the user, the second list of one or more applications including the application that has been activated from the first list of one or more applications provided in the first menu;

displaying the second menu in a second region of the total field of view of the user, the second region being opposite to the first region;

receiving a second user selection to de-activate an application from the second list of one or more applications provided in the second menu; and

removing the application that has been de-activated from the second list of one or more applications provided in the second menu responsive to the second user selection.

16. The processor readable storage device of claim 15, wherein receiving a first user selection to activate an application from the first list of one or more applications provided in the first menu includes detecting a user action.

17. The processor readable storage device of claim 15, the user action is one of dragging the application from the first list of one or more applications provided in the first menu across a primary field-of-view of the user, eye gaze, or user input through a secondary device.

18. The processor readable storage device of claim 15, wherein the third region of the total field of view of the user is a primary field-of-view of the user.

19. The processor readable storage device of claim 15, wherein the second user selection to de-activate an application from the second list of one or more applications provided in the second menu includes dragging the application from the second region to the first region across a primary field-of-view of the user.

20. The processor readable storage device of claim 15 further comprising selecting the first region and the second region based on one or more configured rules.