MULTI-STEP VIRTUAL OBJECT SELECTION

• US 20140372957A1
• Filed: 06/18/2013
• Published: 12/18/2014
• Est. Priority Date: 06/18/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Robert L. Crocco Jr
• Inventors: Keane, Brian E., Sugden, Ben J., Crocco, Robert L. Jr., Deptford, Daniel, Salter, Tom G., Massey, Laura K., Kipman, Alex Aben-Athar, Kinnebrew, Peter Tobias, Kamuda, Nicholas Ferianc
• Granted Patent: US 9,329,682 B2
• Time in Patent Office: Days
• US Class Current: 715/852
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

A head mounted display allows user selection of a virtual object through multi-step focusing by the user. Focus on the selectable object is determined and then a validation object is displayed. When user focus moves to the validation object, a timeout determines that a selection of the validation object, and thus the selectable object has occurred. The technology can be used in see through head mounted displays to allow a user to effectively navigate an environment with a multitude of virtual objects without unintended selections.

Claims

1. A method for selecting a virtual object rendered in a see-through head mounted display, comprising:

rendering one or more virtual objects, at least one virtual object being a selectable virtual object by a user to engage a function with respect to the object;

determining gaze, head position and field of view for a user, the gaze, head position and field of view determining a user focus;

if the focus of the user intersects a selectable virtual object, determining an initial selection of the selectable virtual object;

displaying a validation object proximate to the selectable virtual object; and

detecting whether the focus of the user intersects the validation object thereby initiating a selection of the selectable object.

2. The method of claim 1 wherein the method further includes, prior to displaying a validation object, determining if the user focus remains on the selectable virtual object for a first time period.

3. The method of claim 1 wherein said detecting includes determining if the user focus remains on the validation object for a second time period.

4. The method of claim 1 further including determining the focus of the user focus leaves the selectable virtual object and initiating an unselection of the object.

5. The method of claim 1 wherein the focus of the user includes determining whether a user gaze intersects one of the selectable virtual object or the validation object.

6. The method of claim 1 wherein the focus of the user includes determining whether a user gaze and a user head direction intersect one of the selectable virtual object or the validation object within a defined difference amount.

7. The method of claim 1 wherein detecting whether the user focus intersects the validation object includes detecting a gaze motion pattern between the selectable object and the validation object.

8. The method of claim 1 wherein the method further includes, prior to displaying the validation object, determining if the user focus remains on the selectable virtual object for a first time period, and

determining if the user focus remains on the validation object for a second time period, wherein the first time period is shorter than the second time period.

9. A see though head mounted display apparatus, comprising:

a see through head mounted display; and

a processor coupled to the head mounted display, the processor programmed to

render at least one selectable virtual object in the head mounted display;

determine gaze and head position of a user, thereby providing a user focus of the user;

determining an initial selection of the selectable virtual object based on the user focus;

displaying a validation object proximate to the selectable virtual object; and

detecting a selection of the selectable object by the user when the focus of the user rests on the validation object.

10. The apparatus of claim 9 wherein the processor is programmed to determine if the user focus remains on the selectable virtual object for a first time period.

11. The apparatus of claim 10 wherein the processor is programmed to detect whether the focus if the user focus remains on the validation object for a second time period.

12. The apparatus of claim 11 wherein the first time period is shorter than the second time period.

13. The apparatus of claim 12 wherein the processor is programmed to determine the user focus of the user by determining whether a user gaze intersects one of the selectable virtual object or the validation object.

14. The apparatus of claim 13 wherein the processor is programmed to determine whether the focus of the user leaves the selectable virtual object and initiating an unselection of the object.

15. The apparatus of claim 9 wherein the processor is programmed to issue a selection indicator responsive to detecting a selection of the selectable object.

16. A computer storage medium including code instructing a processing device to rendering one or more virtual objects in a head mounted display, the code comprising:

code rendering at least a selectable virtual object, the virtual object selectable by a user to engage a function with respect to the object;

code determining a user gaze, head position and field of view for a user to establish a focus of the user;

code determining whether a user gaze and a user head direction intersect a selectable virtual object within a defined difference amount;

code determine if the user gaze and user head direction intersect said selectable virtual object, then initiating a first timer;

upon expiration of the first timer, code displaying a validation object proximate to the selectable virtual object; and

code detecting whether the user gaze and user head direction intersect the validation object for a duration of a second timer thereby initiating a selection of the selectable object.

17. The computer storage medium of claim 16 further including code for issuing a selection indicator responsive to detecting a selection of the selectable object.

18. The computer storage medium of claim 16 further including code detecting whether a gaze motion pattern between the selectable object and the validation object matches a known pattern.

19. The computer storage medium of claim 16 wherein code displaying the validation object renders the validation object to appear connected to the selectable virtual object with no other selectable virtual object therein between.

20. The computer storage medium of claim 16 further including code determining whether the gaze and head position of the user leaves the selectable virtual object and initiating an unselection of the object