User-based context sensitive hologram reaction

• US 10,156,721 B2
• Filed: 03/09/2015
• Issued: 12/18/2018
• Est. Priority Date: 03/09/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Scott, Jason, Keibler, Evan Michael, Tomlin, Arthur Charles, Paulovich, Jonathan, Amador-Leon, Ron, Plumb, Jonathan, Fajt, Nicholas Gervase
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US14/642,462
• Publication Number:
  US 20160266386A1
• Time in Patent Office: 1,380 Days
• Field of Search: 345633
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts

Abstract

A system and method are disclosed for displaying virtual content. When a user is settled in a given geographic area, a full version of the content may be displayed at user-defined or processor-defined location. When the user is moving, for example leaving the geographic area, a mobile version of the content may be displayed at a body-locked position peripheral to the user'"'"'s view.

Claims

1. A system for presenting a virtual environment, comprising:

a head mounted display device including a display for displaying three-dimensional virtual objects in the virtual environment; and

a processor operatively coupled to the display device, the processor configured to;

determine a geographic area in which the head mounted display device is located based on feedback from the head mounted display device, the geographic area comprising an area bounded by at least first and second walls;

determine whether the geographic area is a known geographic area by comparing image data of the feedback from the head mounted display device to stored image data for known geographic areas;

in response to a determination that the geographic area is a known geographic area, determine a rule specific to the geographic area and stored in a memory of the head mounted display, wherein the rule was stored during a previous visit to the geographic area and in response to receiving a user preference input indicating a user preference for a size at which a virtual object is to be displayed within the geographic area;

direct the display to display a virtual object at the first wall where the head mounted display device is facing the first wall, and to direct the display to display the virtual object at the second wall where the head mounted display device is facing the second wall; and

direct the display to display the virtual object at a position within the geographic area with a size that is determined based on the stored rule in response to determining that there is a stored rule.

2. The system of claim 1, wherein the position is determined based on another stored rule indicating another user preference with respect to positioning the displayed virtual object in the geographic area.

3. The system of claim 1, wherein the position is determined based on a scanning of the geographic area by sensors in the head mounted display device to determine a suitable position at which the virtual object is to appear to be displayed.

4. The system of claim 1, wherein the position is determined based on a gesture performed by a user and detected by the head mounted display device, the gesture indicating the position.

5. The system of claim 1, wherein in response to the processor determining that the geographic area is not a known geographic area, the size is determined based on a scanning of the geographic area by sensors in the head mounted display device to determine a suitable size for display of the virtual object at the position at which the virtual object is to appear to be displayed, and the given size is stored as a new stored rule for the new geographic area.

6. The system of claim 1, wherein where the processor determines that the first wall has an open space, based on feedback from the head mounted display device, the processor is configured to direct the head mounted display device to display the virtual object so as to appear to be displayed on the open space of the first wall.

7. The system of claim 1, wherein the virtual object is virtual content including at least one of static images and dynamic content.

8. The system of claim 1, wherein the virtual object is virtual content including at least one of web pages and user interfaces for software applications.

16. The method of claim 1, wherein the stored rule is specific to the geographic area such that the stored rule is only applicable when the user of the head-mounted display device is within the geographic area bounded by the first and second walls.

9. A system for presenting a virtual environment comprising:

a head mounted display device including a display for displaying three-dimensional virtual objects in the virtual environment; and

a processor operatively coupled to the display device, the processor configured to;

determine whether the head mounted display is in a settled state within a geographic area comprising an area bounded by at least first and second walls or a mobile state moving out of the geographic area and not when the user is walking around within the geographic area based on feedback from the head mounted display device;

direct the display to display one of a full, world-locked version of virtual content in response to determining that the head mounted display device is in a settled state and a mobile, body-locked version of the content where the virtual content is displayed in a periphery of a user'"'"'s view, in response to determining that the head mounted display device is in a mobile state.

10. The system of claim 9, wherein the processor is configured to direct the display to display the virtual content with a degree of translucence in the mobile version.

11. The system of claim 9, wherein the processor is configured to automatically switch between the full version of the content and the mobile version of the content based on whether the processor detects the user going from the mobile state to the settled state, and vice versa.

12. A method of presenting a virtual environment via a head mounted display device and a processor associated with the head mounted display device, the method comprising:

(a) detecting whether the head mounted display device is in a settled state within a geographic area comprising an area bounded by at least first and second walls, or a mobile state walking out of the geographic area;

(b) displaying virtual content in a determined world-locked position upon detecting that the head mounted display device is in the settled state in the geographic area; and

(c) switching to displaying a mobile version of the virtual content in a body-locked position based upon detecting that the head mounted display device is in a mobile state leaving the geographic area.

13. The method of claim 12, said step (b) comprising the step of displaying the virtual content in a position determined based on input received from a user.

14. The method of claim 12, said step (b) comprising the step of displaying the virtual content in a position determined by the processing device based on a determined layout of the geographic area and available space for displaying the virtual content.

15. The method of claim 12, said step (b) comprising the step of displaying the virtual content in one of a plurality of positions in the geographic location, based on a location and orientation of the head mounted display device in the geographic location.