Multi-space connected virtual data objects

• US 9,235,051 B2
• Filed: 06/18/2013
• Issued: 01/12/2016
• Est. Priority Date: 06/18/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Salter, Tom G., Sugden, Ben J., Deptford, Daniel, Crocco, Robert L. Jr., Keane, Brian E., Massey, Laura K., Kipman, Alex Aben-Athar, Kinnebrew, Peter Tobias, Kamuda, Nicholas Ferianc
• Primary Examiner(s): LEE, NICHOLAS J
• Application Number:
  US13/921,102
• Publication Number:
  US 20140368533A1
• Time in Patent Office: 938 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

A see-through head mounted display apparatus includes a display and a processor. The processor determines geo-located positions of points of interest within a field of view and generates markers indicating information regarding an associated real world object is available to the user. Markers are rendered in the display relative to the geo-located position and the field of view of the user. When a user selects a marker though a user gesture, the device displays a near-field virtual object having a visual tether to the marker simultaneously with the marker. The user may interact with the marker to view, add or delete information associated with the point of interest.

Claims

1. In a see-through head mounted display, a method of displaying information associated with real world environments, comprising:

determining a position of one or more holographic markers relative to geographic coordinates in an environment, the markers indicating information regarding a point of interest in the environment is available;

rendering the markers in the display relative to the position and a field of view of a user;

receiving a selection of a marker from a user;

rendering a near-field virtual object including a visual relationship to the marker, the near-field virtual object displaying the information associated with the marker, the visual relationship comprising a tether extending between the near field object and the marker, a first end of the visual tether associated with the marker appearing to be at a greater distance from the user than a second end of the tether and the virtual object.

2. The method of claim 1 wherein input controls are adapted to allow input using one of user movements and user utterances.

3. The method of claim 1 further including receiving at least a portion of the information from third party social media services.

4. The method of claim 1 wherein the visual relationship is a visual tether between the near-field object and the marker, a first end of the visual tether associated with the marker appearing to be at a greater distance from the user than a second end of the tether and the virtual object.

5. The method of claim 1 wherein the visual relationship is a representation of a source of the information displayed at the marker and at the near-field virtual object.

6. The method of claim 1 wherein each marker in the field of view of the user has a relative size based on a distance between the user and the point of interest associated with the marker.

7. The method of claim 1 wherein at least one marker in the field of view of the user has a relative size based on a sponsorship by an operator of the point of interest associated with the marker.

8. The method of claim 1 further including monitoring user movement relative to the environment and altering the rendering of the near-field virtual object.

9. The method of claim 8 wherein the rendering includes rendering the near-field object relative to a local coordinate system referenced to the user.

10. The method of claim 9 wherein the rendering includes rendering the near-field object relative to a global coordinate system.

11. A see-through head mounted display device including a display and a processor, the processor coupled to a storage device, comprising:

code instructing the processor to determine geo-located positions of points of interest within a field of view of a user in an environment and calculate positions of associated markers indicating information regarding an associated point of interest is available to the user;

code instructing the processor to render the markers in the display relative to the geo-located position and the field of view of the user to the geo-located position, each marker presenting a representation of a source of the information associated with the marker;

code instructing the processor to detect a selection of a marker from a user by a user gesture in the field of view which intersects a perceived position of the marker to the user;

code instructing the processor to display a near-field virtual object having a visual tether to the marker simultaneously with the marker in the field of view, the near-field virtual object displaying the information associated with the marker, the visual tether extending between the near field object and the marker, the tether displayed with a virtual length, extending into a depth of the field of view, the virtual length decreasing as a distance between the near field object and the point of interest gets smaller.

12. The display device of claim 11 wherein the representation is rendered a source-specific icon indicating of the source of the information displayed at the marker and at the near-field virtual object.

13. The display device of claim 12 further including code instructing the processor to monitor user movement relative to the environment and altering the rendering of the markers in the field of view and the near-field virtual object.

14. The display device of claim 13 wherein code instructs the processor to render the near-field object relative to a local coordinate system referenced to the user.

15. The display device of claim 13 wherein code instructs the processor to render the near-field object relative to a global coordinate system.

16. A see-through head mounted display device, comprising:

a display positionable proximate to a wearers eyes; and

a processor, the processor programmed to

determine positions of virtual object markers within a field of view of the display, the markers indicating information regarding a point of interest associated with a point of interest is accessible;

render the markers in the display, the markers representative of a geo-located position of the point of interest and a perspective of a user to the geo-located position, each marker presenting icon representing a source of the information, each marker having a variable size based on a distance from the device to the geo-located position associated with the point of interest;

detect a selection of a marker from a user by a user gesture in the field of view;

responsive to the selection, render a virtual object occupying a majority of the field of view of the display, the virtual object having a visual tether to the marker simultaneously displayed with the marker and the virtual object in the field of view, the virtual object displaying the information associated with the marker and to display on the near field virtual marker information modification input controls adapted to allow the user to add, edit and modify the information associated with and describing the real world object via at least one of gestures and speech as detected by the device.

17. The display device of claim 16 wherein the processor is programmed to render the visual tether in the field of view such that a first end of the visual tether associated with the marker appears to be at a greater distance from the user than a second end of the tether and the marker.

18. The display device of claim 16 wherein the processor is programmed to monitor user movement relative to the environment and alter the rendering of the markers the field of view and the virtual object.

19. The display device of claim 18 wherein the processor is programmed to render the tether relative to movement of the user.

20. The display device of claim 18 wherein the processor is programmed to render the tether as having a first end connected to the virtual object and a second end coupled to the marker selected.