GAZE SWIPE SELECTION

• US 20150205494A1
• Filed: 01/23/2014
• Published: 07/23/2015
• Est. Priority Date: 01/23/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Scott, Jason, Tomlin, Arthur C., Thomas, Mike, Kaplan, Matthew, Fajt, Nicholas Gervase, McCulloch, Daniel J., Brown, Cameron G., Plumb, Jonathan, Lee, Jeremy
• Granted Patent: US 9,201,578 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Methods for enabling hands-free selection of virtual objects are described. In some embodiments, a gaze swipe gesture may be used to select a virtual object. The gaze swipe gesture may involve an end user of a head-mounted display device (HMD) performing head movements that are tracked by the HMD to detect whether a virtual pointer controlled by the end user has swiped across two or more edges of the virtual object. In some cases, the gaze swipe gesture may comprise the end user using their head movements to move the virtual pointer through two edges of the virtual object while the end user gazes at the virtual object. In response to detecting the gaze swipe gesture, the HMD may determine a second virtual object to be displayed on the HMD based on a speed of the gaze swipe gesture and a size of the virtual object.

Claims

1. An electronic device for displaying an augmented reality environment, comprising:

a see-through display, the see-through display displays a virtual object to an end user of the electronic device, the virtual object includes a plurality of edges; and

one or more processors in communication with the see-through display, the one or more processors detect a gaze swipe activation event performed by the end user and display a virtual pointer to the end user in response to detecting the gaze swipe activation event, the one or more processors detect a gaze swipe gesture performed by the end user, the gaze swipe gesture corresponds with the end user moving the virtual pointer using one or more head movements through two or more edges of the plurality of edges, the one or more processors determine a speed of the gaze swipe gesture, the one or more processors determine a second virtual object different from the virtual object based on the speed of the gaze swipe gesture and the two or more edges of the plurality of edges traversed by the gaze swipe gesture, the one or more processors cause the second virtual object to be displayed using the see-through display.

2. The electronic device of claim 1, wherein:

the electronic device comprises an HMD; and

the one or more processors detect that the end user performed the gaze swipe gesture by tracking the end user'"'"'s one or more head movements using the HMD.

3. The electronic device of claim 1, wherein:

the electronic device comprises an HMD; and

the one or more processors detect the gaze swipe gesture performed by the end user by detecting that the end user moved the virtual pointer through the two or more edges of the plurality of edges while the end user gazed at the virtual object, the one or more processors detect that the end user gazed at the virtual object by tracking the end user'"'"'s eye movements using the HMD.

4. The electronic device of claim 1, wherein:

the one or more processors detect the gaze swipe gesture performed by the end user by detecting that the end user moved the virtual pointer through three or more edges of the plurality of edges.

5. The electronic device of claim 1, wherein:

the virtual object comprises a shape that is bounded by the plurality of edges.

6. The electronic device of claim 1, wherein:

the one or more processors determine a size of the virtual object, the one or more processors determine the second virtual object based on the two or more edges of the plurality of edges and the size of the virtual object.

7. The electronic device of claim 1, wherein:

the one or more processors cause the second virtual object to be displayed using the see-through display if the speed of the gaze swipe gesture is above a set speed threshold.

8. A method for selecting a virtual object using an HMD, comprising:

displaying the virtual object to an end user of the HMD, the virtual object includes a plurality of edges;

detecting a gaze swipe activation event performed by the end user;

displaying a virtual pointer to the end user of the HMD in response to detecting the gaze swipe activation event;

detecting a gaze swipe gesture performed by the end user of the HMD, the gaze swipe gesture corresponds with the end user moving the virtual pointer using one or more head movements through at least two edges of the plurality of edges;

determining a speed of the gaze swipe gesture;

determining a second virtual object different from the virtual object based on the speed of the gaze swipe gesture and the at least two edges of the plurality of edges traversed by the gaze swipe gesture; and

displaying the second virtual object to the end user of the HMD.

9. The method of claim 8, wherein:

the detecting a gaze swipe gesture includes detecting that the end user moved the virtual pointer through the at least two edges of the plurality of edges while the end user gazed at the virtual object.

10. The method of claim 8, wherein:

the detecting a gaze swipe gesture includes detecting that the end user moved the virtual pointer through at least three edges of the plurality of edges.

11. The method of claim 10, further comprising:

displaying an order of edge traversals to the end user in response to detecting the gaze swipe activation event, the detecting a gaze swipe gesture includes detecting that the end user moved the virtual pointer through the at least three edges based on the order of edge traversals.

12. The method of claim 8, wherein:

the virtual object comprises a shape that is bounded by the plurality of edges.

13. The method of claim 8, further comprising:

determining a size of the virtual object, the determining a second virtual object includes determining the second virtual object based on the at least two edges of the plurality of edges and the size of the virtual object.

14. The method of claim 8, further comprising:

displaying a third virtual object to the end user of the HMD prior to the detecting a gaze swipe gesture, the third virtual object includes a third plurality of edges, the detecting a gaze swipe gesture includes detecting that the end user moved the virtual pointer through the at least two edges of the plurality of edges and at least two edges of the third plurality of edges.

15. The method of claim 8, further comprising:

highlighting a first pair of edges of the plurality of edges in response to detecting the gaze swipe activation event, the first pair of edges is associated with a first selection type; and

highlighting a second pair of edges of the plurality of edges in response to detecting the gaze swipe activation event, the second pair of edges is associated with a second selection type different from the first selection type.

16. The method of claim 15, wherein:

the detecting a gaze swipe gesture includes detecting that the end user moved the virtual pointer through the second pair of edges.

17. The method of claim 8, further comprising:

determining a size of the virtual object; and

highlighting a pair of edges of the plurality of edges based on the size of the virtual object in response to detecting the gaze swipe activation event.

18. The method of claim 8, wherein:

the determining a second virtual object includes identifying the second virtual object if the speed of the gaze swipe gesture is greater than a set speed threshold.

19. One or more storage devices containing processor readable code for programming one or more processors to perform a method for selecting a virtual object using an HMD comprising the steps of:

displaying the virtual object to an end user of the HMD, the virtual object comprises a shape that is bounded by a plurality of edges;

detecting a gaze swipe activation event performed by the end user;

displaying a virtual pointer to the end user of the HMD in response to detecting the gaze swipe activation event;

detecting a gaze swipe gesture performed by the end user of the HMD, the gaze swipe gesture corresponds with the end user moving the virtual pointer using one or more head movements through two or more edges of the plurality of edges while the end user gazed at the virtual object, the detecting a gaze swipe gesture includes tracking the end user'"'"'s one or more head movements using the HMD;

determining a speed of the gaze swipe gesture;

determining a second virtual object different from the virtual object based on the speed of the gaze swipe gesture and the two or more edges of the plurality of edges traversed by the gaze swipe gesture; and

displaying the second virtual object to the end user of the HMD.

20. The one or more storage devices of claim 19, wherein:

the detecting a gaze swipe gesture includes detecting that the end user moved the virtual pointer through at least three edges of the plurality of edges; and

the determining a second virtual object includes identifying the second virtual object if the speed of the gaze swipe gesture is greater than a set speed threshold.