GESTURE BASED REGION IDENTIFICATION FOR HOLOGRAMS

• US 20130321462A1
• Filed: 06/01/2012
• Published: 12/05/2013
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Robert L. Crocco Jr
• Inventors: Kipman, Alex Aben-Athar, Sugden, Ben J., Crocco, Robert L. Jr., Keane, Brian E., Miles, Christopher E., Perez, Kathryn Stone, Massey, Laura K., Lamb, Mathew J., Salter, Tom G.
• Granted Patent: US 9,116,666 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/147 using display panels

Abstract

Techniques are provided for allowing a user to select a region within virtual imagery, such as a hologram, being presented in an HMD. The user could select the region by using their hands to form a closed loop such that from the perspective of the user, the closed loop corresponds to the region the user wishes to select. The user could select the region by using a prop, such as a picture frame. In response to the selection, the selected region could be presented using a different rendering technique than other regions of the virtual imagery. Various rendering techniques such as zooming, filtering, etc. could be applied to the selected region. The identification of the region by the user could also serve as a selection of an element in that portion of the virtual image.

Claims

1. A method comprising:

presenting virtual imagery in a see-through, near-eye display device;

accessing image data of an environment that coincides with the virtual imagery from the perspective of a user wearing the see-through, near-eye display device;

analyzing the image data to identify a region select symbol being made by the user;

identifying a region in the image data that corresponds to the region select symbol; and

determining a portion of the virtual imagery that corresponds to the region in the image data from the perspective of the user.

2. The method of claim 1, further comprising:

modifying the virtual imagery in the see-through, near-eye display device based on the portion of the virtual imagery that is determined to correspond to the region in the image data from the perspective of the user.

3. The method of claim 2, wherein the modifying the virtual imagery includes:

presenting, in the see-through, near-eye display device, the portion of the virtual imagery that corresponds to the region in the image data from the perspective of the user with a first display property and other portions of the virtual imagery with a second display property.

4. The method of claim 1, further comprising:

initiating a select event of an element within the portion of the virtual imagery; and

taking an action based on the element being selected.

5. The method of claim 1, wherein the analyzing the image data to identify a region select symbol being made by the user includes:

identifying at least one of the user'"'"'s hands; and

determining that at least one hand forms an enclosed loop.

6. The method of claim 5, wherein the identifying the user'"'"'s hands includes performing skeletal tracking.

7. The method of claim 1, wherein the analyzing the image data to identify a region select symbol being made by the user includes:

identifying a prop other than a body part that forms a closed loop.

8. A display system comprising:

a see-through, near-eye display device;

an image sensor;

logic in communication with to the display device and the image sensor, the logic is configured to;

present holographic imagery in the see-through, near-eye display device;

access image data of an environment that coincides with the holographic imagery from the perspective of a user wearing the see-through, near-eye display device;

analyze the image data to identify a region select symbol being made by the user;

identify a region in the image data that corresponds to the region select symbol;

correlate the region in the image data to a portion of the holographic imagery; and

initiate a select event in response to identifying the region select symbol being made by the user.

9. The display system of claim 8, wherein the logic is further configured to:

render the holographic imagery in the see-through display device based on the portion of the holographic imagery that was correlated to the region in the image data.

10. The display system of claim 9, wherein the logic is further configured to render the portion of the holographic imagery that was correlated to the region in the image data with a first rendering technique and other portions of the holographic imagery with a second rendering technique in response to the select event.

11. The display system of claim 9, wherein the portion of the holographic imagery includes an element, the select event is a selection of the element, the logic is further configured to take an action based on the user selecting the element.

12. The display system of claim 9, wherein the logic being configured to analyze the image data to identify a region select symbol being made by the user includes the logic being configured to identify at least one of the user'"'"'s hands when analyzing the image data to identify an enclosed region.

13. The display system of claim 12, wherein the logic being configured to identify the user'"'"'s hands includes the logic being configured to perform skeletal tracking.

14. The display system of claim 9, wherein the logic being configured to analyze the image data to identify a region select symbol being made by the user includes the logic being configured to:

identify a prop other than a body part that forms an enclosed region.

15. The display system of claim 9, wherein at least a portion of the logic communicates with the see-through, near-eye display device over a network.

16. A computer storage device having instructions stored thereon which, when executed on a processor, cause the processor to:

cause a see-through, near-eye display device to present holographic imagery;

access image data of an environment that coincides with the holographic imagery from the perspective of a user wearing the see-through, near-eye display device;

analyze the image data to identify a region select symbol being made by the user;

identify a region in the image data that corresponds to the region select symbol;

correlate the region in the image data to a portion of the holographic imagery;

initiate a select event in response to identifying the region select symbol being made by the user; and

cause the see-through, near-eye display device to present the portion of the holographic imagery with a first display property and other portions of the holographic imagery with a second display property in response to the select event.

17. The computer storage device of claim 16, wherein the first display property is selected from the group including:

zooming, presenting an x-ray, and presenting a view into a different world.

18. The computer storage device of claim 16, wherein the instructions that cause the processor to analyze the image data to identify a region select symbol being made by the user cause the processor to:

identify at least one of the user'"'"'s hands; and

determine that at least one hand forms an enclosed loop.

19. The computer storage device of claim 16, wherein the instructions that cause the processor to identify at least one of the user'"'"'s hands cause the processor to:

perform skeletal tracking.

20. The computer storage device of claim 16, wherein the instructions that cause the processor to analyze the image data to identify a region select symbol being made by the user cause the processor to:

identify a prop other than a body part that forms a closed loop.