Virtual reality and cross-device experiences

• US 10,332,317 B2
• Filed: 06/07/2017
• Issued: 06/25/2019
• Est. Priority Date: 10/25/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Nagy, Scott Anderson, Carter, Jason, Gavriliuc, Anatolie, Osborn, Daniel L., Mulcahy, Kathleen, Ryan, Patrick C., Chua, Michelle Tze Hiang
• Primary Examiner(s): Lett, Thomas J
• Application Number:
  US15/616,613
• Publication Number:
  US 20180114372A1
• Time in Patent Office: 748 Days
• Field of Search: 345419
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling

Abstract

The present disclosure provides approaches to facilitating virtual reality and cross-device experiences. In some implementations, an environmental snapshot is captured which includes an image of a virtual reality (VR) environment presented on a VR device and corresponding depth information of the VR environment. The image of the environmental snapshot is presented on a different device than the VR device. A user modification to content associated with the presented image is translated into the environmental snapshot based on the depth information. The environmental snapshot comprising the user modification is translated into the VR environment. The VR environment comprising the translated user modification is presented. The environmental snapshot may correspond to a personal space of a user and may be accessed by another user through a social networking interface or other user networking interface to cause the presentation of the image.

Claims

1. A computer-implemented method comprising:

capturing, using one or more depth sensing cameras of an augmented reality (AR) device and in response to a user selection of a corresponding option using the AR device while an AR environment is presented on the AR device, an environmental snapshot comprising a two-and-a-half-dimensional (2.5D) image corresponding to the AR environment presented on the AR device, the 2.5D image comprising a bitmap depicting a view of three-dimensional (3D) space of the AR environment and corresponding depth information;

presenting the 2.5D image of the environmental snapshot on a different device than the AR device;

translating a user modification to content associated with the presented 2.5D image into the environmental snapshot based on the depth information;

translating the environmental snapshot comprising the user modification into the AR environment; and

presenting the AR environment comprising the translated user modification.

2. The method of claim 1, wherein the user modification comprises a user placement of the content behind other content in the environmental snapshot based on the depth information.

3. The method of claim 1, wherein the translating of the user modification is to a three-dimensional (3D) coordinate system corresponding to the depth information.

4. The method of claim 1, wherein during the user modification, the content is depicted at a size scaled based on the depth information.

5. The method of claim 1, wherein the AR environment comprises a three-dimensional (3D) representation of a real world environment sensed by the AR device and one or more virtual objects added to the real world environment.

6. The method of claim 1, wherein a perspective depicted in the 2.5D image corresponds to a user perspective of the AR environment displayed in association with the capturing of the environmental snapshot at a time of the user selection of the corresponding option.

7. The method of claim 1, wherein the user selection is of a graphical interface element displayed in association with the presented AR environment.

8. The method of claim 1, wherein the content comprises a virtual object having at least one predefined trigger action associated with trigger criteria, and the presenting of the AR environment comprises performing the at least one predefined trigger action with respect to the virtual object based on determining the trigger criteria is satisfied.

9. The method of claim 1, wherein the presenting of the AR environment comprises capturing a user reaction to the content.

10. A computer-implemented system comprising:

one or more processors; and

one or more computer storage media storing computer-useable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

presenting a virtual reality (VR) environment in a first graphical user interface;

in response to a user selection of a corresponding option associated with the first graphical user interface, saving a two-and-a-half-dimensional (2.5D) image corresponding to the presented VR environment, the 2.5D image comprising a view of three-dimensional (3D) space of the VR environment and corresponding depth information;

presenting the saved 2.5D image in a second graphical user interface;

translating a user modification to content associated with the presented 2.5D image into the VR environment based on the depth information; and

presenting the VR environment comprising the translated user modification.

11. The computer-implemented system of claim 10, wherein a perspective captured in the 2.5D image corresponds to a position of a user device in the VR environment at a time of the user selection, the user device displaying the VR environment during the presenting.

12. The computer-implemented system of claim 10, wherein the saving of the 2.5D image is in association with a personal space of a user, and the presenting of the VR environment comprises:

automatically triggering a recording of a user reaction to the content based on a position of a user device relative to the content in the VR environment; and

transmitting the recording to the user based on the association with the personal space.

13. The computer-implemented system of claim 10, wherein the saving of the 2.5D image is in association with a personal space of a first user, and the method further comprises:

presenting to a second user a plurality of personal spaces comprising the personal space in a social network interface; and

receiving an additional user selection associated with the presented personal space, wherein the presenting of the saved 2.5D image is based on the additional user selection.

14. The computer-implemented system of claim 10, wherein during the user modification, the content is visually scaled based on the depth information.

15. The computer-implemented system of claim 10, wherein a perspective depicted in the 2.5D image corresponds to a user perspective of the VR environment displayed in association with the saving of the 2.5D image.

16. One or more non-transitory computer-readable media having executable instructions embodied thereon, which, when executed by one or more processors, cause the one or more processors to perform a method, the method comprising:

presenting, in a first graphical user interface, an environmental snapshot comprising a two-and-a-half-dimensional (2.5D) image corresponding to a virtual reality (VR) environment, the 2.5D image depicting a view of three-dimensional (3D) space of the VR environment and corresponding depth information;

translating a user modification to content associated with the 2.5D image of the presented environmental snapshot into the environmental snapshot based on the depth information;

translating the environmental snapshot comprising the user modification into the VR environment; and

presenting the VR environment comprising the translated user modification in a second graphical user interface.

17. The one or more computer-readable media of claim 16, wherein the user modification corresponds to a user positioning the content in three-dimensional space.

18. The one or more computer-readable media of claim 16, wherein the translating of the user modification is to a 3D coordinate system corresponding to the depth information.

19. The one or more computer-readable media of claim 16, wherein during the user modification, the content is depicted at a size scaled based on the depth information.

20. The one or more computer-readable media of claim 16, wherein the VR environment is an augmented reality environment.