Mixed reality objects

• US 10,467,816 B2
• Filed: 03/23/2018
• Issued: 11/05/2019
• Est. Priority Date: 03/23/2018
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ghatak, Kausik, Varadharajan, Janani, Potey, Swapnil M., Kandpal, Eatesh
• Primary Examiner(s): Yang, Ryan R
• Application Number:
  US15/934,653
• Publication Number:
  US 20190295320A1
• Time in Patent Office: 592 Days
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

Examples are disclosed herein that relate to mixed reality objects. One example provides a computing device comprising a logic machine and a storage machine holding instructions executable by the logic machine to execute a mixed reality application, the mixed reality application defining a third-party object slot, present a mixed reality environment, and upon detecting a condition associated with the third-p arty object slot, send to a remote computing system a request to fill the third-party object slot, the request based on sensing of a real-world portion of the mixed reality environment with a sensor of the computing device. The instructions are further executable to receive a third-party object from the remote computing system selected by the remote computing system based on the request, and display the third-party object in the third-party object slot such that the third-party object is constrained by a real-world portion of the mixed reality environment.

Claims

1. A computing device, comprising:

a logic machine; and

a storage machine holding instructions executable by the logic machine to;

execute a mixed reality application, the mixed reality application defining a third-party object slot;

present, on a display device, a mixed reality environment via execution of the mixed reality application;

upon detecting a condition associated with the third-party object slot, send to a remote computing system a request to fill the third-party object slot, the request based on sensing of a real-world portion of the mixed reality environment with a sensor of the computing device;

receive a third-party object from the remote computing system, the third-party object selected by the remote computing system based on the request; and

display, on the display device, the third-party object in the third-party object slot such that the third-party object is constrained by a real-world portion of the mixed reality environment.

2. The computing device of claim 1, where the request specifies a type of physical feature associated with the third-party object slot.

3. The computing device of claim 1, where the real-word portion includes a physical feature, and where the request includes data regarding one or more physical parameters of the physical feature.

4. The computing device of claim 1, where the instructions executable to sense a real-world portion are executable to determine an identity of the real-world portion.

5. The computing device of claim 1, where the condition is defined by the mixed reality application and is triggered by user interaction with the mixed reality application.

6. The computing device of claim 1, where the instructions executable to sense a real-world portion are executable to determine one or both of a location and orientation of the computing device, and where the third-party object is selected by the remote computing system based on such location/orientation.

7. The computing device of claim 1, where the instructions executable to sense a real-world portion are executable to determine a local time of the computing device, and where the third-party object is selected by the remote computing system based on the local time.

8. The computing device of claim 1, where the instructions executable to sense a real-world portion are executable to determine one or more lighting conditions local to the computing device, and where the third-party object is selected by the remote computing system based on the one or more lighting conditions.

9. The computing device of claim 1, where the instructions executable to sense a real-world portion are executable to determine one or more weather conditions local to the computing device, and where the third-party object is selected by the remote computing system based on the one or more weather conditions.

10. The computing device of claim 1, where the request is based further on sensing data corresponding to a user of the computing device, and where the third-party object is selected by the remote computing system based on the data.

11. The computing device of claim 1, where the instructions executable to display the third-party object as constrained by a real-world portion are executable to display the third-party object as physically interacting with gravity and one or more physical surfaces associated with the real-world portion.

12. The computing device of claim 1, where the instructions are further executable to receive a user input directed to the third-party object and control the display of the third-party object based on the user input.

13. The computing device of claim 12, where the instructions are further executable to identify a user interaction with the third-party object based on the user input.

14. The computing device of claim 13, where the instructions are further executable to send to the remote computing system an indication of the user interaction, and where the remote computing system is configured to select a future third-party object based on the indication.

15. The computing device of claim 12, where the user input includes one or both of a gesture input and a gaze input.

16. A method, comprising:

executing a mixed reality application, the mixed reality application defining a third-party object slot;

presenting, on a display device, a mixed reality environment via execution of the mixed reality application;

upon detecting a condition associated with the third-party object slot, sending to a remote computing system a request to fill the third-party object slot, the request based on sensing of a real-world portion of the mixed reality environment with a sensor of the computing device;

receiving a third-party object from the remote computing system, the third-party object selected by the remote computing system based on the request; and

displaying, on the display device, the third-party object in the third-party object slot such that the third-party object is constrained by a real-world portion of the mixed reality environment.

17. The method of claim 16, where the third-party object is displayed as physically interacting with gravity and one or more physical surfaces associated with the real-world portion.

18. The method of claim 16, where the real-word portion includes a physical feature, and where the request includes data regarding one or more physical parameters of the physical feature.

19. A head-mounted display device, comprising:

a display device;

one or more sensors;

a logic machine; and

a storage machine holding instructions executable by the logic machine to;

execute a mixed reality application, the mixed reality application defining a third-party object slot;

present, on the display device, a mixed reality environment via execution of the mixed reality application;

upon detecting a condition associated with the third-party object slot, send to a remote computing system a request to fill the third-party object slot, the request based on sensing of a real-world portion of the mixed reality environment with the one or more sensors;

receive a third-party object from the remote computing system, the third-party object selected by the remote computing system based on the request; and

display, on the display device, the third-party object in the third-party object slot such that the third-party object is constrained by a real-world portion of the mixed reality environment.

20. The head-mounted display device of claim 19, where the instructions executable to display the third-party object as constrained by a real-world portion are executable to display the third-party object as physically interacting with gravity and one or more physical surfaces associated with the real-world portion.