OBJECT-CENTRIC MIXED REALITY SPACE

• US 20130342570A1
• Filed: 06/25/2012
• Published: 12/26/2013
• Est. Priority Date: 06/25/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Kinnebrew, Peter Tobias, Kamuda, Nicholas Ferianc
• Granted Patent: US 9,767,720 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 3/003 to produce spatial visual effects

Abstract

A see-through, near-eye, mixed reality display apparatus providing a mixed reality environment wherein one or more virtual objects and one or more real objects exist within the view of the device. Each of the real and virtual have a commonly defined set of attributes understood by the mixed reality system allowing the system to manage relationships and interaction between virtual objects and other virtual objects, and virtual and real objects.

Claims

1. A method presenting a mixed reality environment allowing viewing of real world objects integrated with virtual objects by a user, comprising;

determining one or more real objects within a user environment;

determining one or more virtual objects within the user environment;

rendering the one or more virtual objects within a user field of view within the environment;

mapping each of the real objects and each of the virtual objects to an object instance, the object instance based on a common object definition having a set of core attributes; and

managing interaction between any of the virtual objects and real objects based on the attributes defined in the object instance of any interacting virtual and real objects.

2. The method of claim 1 wherein one attribute in the set of attributes includes a relational attribute and the relational attribute defines a relation of the object instance to a user, an environment or another object.

3. The method of claim 1 wherein the step of managing interaction comprises:

tracking each of the real objects and each of the virtual objects within the user environment;

determining a virtual-virtual object interaction when a virtual object interacts with another virtual object; and

rendering the virtual-virtual object interaction based on the set of attributes for each virtual object instance.

4. The method of claim 1 wherein the step of managing interaction comprises:

tracking each of the real objects and each of the virtual objects within the user environment;

determining a virtual-real object interaction when a virtual object interacts with a real object; and

rendering the virtual-real object interaction based on the set of attributes for a virtual object instance and the attributes of a real object instance.

5. The method of claim 1 further including:

tracking each of the real objects and each of the virtual objects within the user environment;

determining a virtual-virtual object interaction when a virtual object interacts with another virtual object;

rendering the virtual-virtual object interaction based on a user filter;

determining a virtual-real object interaction when a virtual object interacts with a real object; and

rendering the virtual-real object interaction based on the set of attributes for a virtual object instance and the attributes of a real object instance.

6. The method of claim 1 wherein the set of attributes includes one or more functions for a virtual object.

7. The method of claim 1 further including generating the object instances and sharing object instances with other users via a communication link, and receiving shared object instances from other users, and including rendering and managing the interaction between the object instances shared by other users and generated object instances.

8. A see through head mounted display apparatus, comprising:

a see-through, near-eye, augmented reality display;

one or more processing devices in wireless communication with apparatus, the one or more processing devices automatically determine an environment, one or more real objects in the environment and one or more virtual objects in the environment, the one or more processing devices assign an object instance to each of the real and virtual objects in the environment, each object instance based on an object definition provided in a data structure containing a common set of attributes for the real and virtual objects, the one or more processing devices determine input data from real world objects and virtual objects in a field of view and integrate interaction between real and virtual objects based on the object instances.

9. The apparatus of claim 8 wherein the common set of attributes comprises a data structure including includes at least one attribute of:

object type, spatial coordinates, object registration, reality rating, dynamic scaling, ownership, user permissions, content rating, physical properties, learned attributes, related objects and functions.

10. The apparatus of claim 9 wherein the physical properties include at least physics attributes defining object movement and actions and an interaction rule set defining object interaction with other objects.

11. The apparatus of claim 9 wherein the object definition includes an identifier.

12. The apparatus of claim 11 wherein each instance of an object definition is specifically identified.

13. The apparatus of claim 9 wherein the apparatus includes a memory and a data structure, the data structure including one or more object definitions modified by a user and owned by a user.

14. The apparatus of claim 13 wherein the one or more processors

track each of the real objects and each of the virtual objects within a user environment;

determine a virtual-virtual object interaction when a virtual object interacts with another virtual object;

render the virtual-virtual object interaction based on a user filter;

determine a virtual-real object interaction when a virtual object interacts with another virtual object; and

render the virtual-real object interaction based on the set of attributes for a virtual object instance and the attributes of a real object instance.

15. A method for managing interaction between virtual holographic objects and real world objects in a mixed reality environment generated by a see through head mounted display system, comprising:

determining an environment and orientation of the system, the system includes one or more sensors and a see-through display;

determining three dimensional locations of real and virtual objects within an environment of a wearer of the see-through display in the environment;

creating an object instance for each of the virtual and real object within the environment;

determining whether an interaction between at least two objects occurs, the interaction being one of an interaction between a virtual object and another virtual object, or an interaction between a virtual object and a real world object;

rendering the interaction between the at least two objects in the display based on attributes defined in the object instance of any interacting virtual and real objects and a system filter, the system filter interpreting the attributes of each of the interacting objects according to user specified filter settings relative to rendering the interaction in the display.

16. The method of claim 15 wherein each object instance contains a value for at least one attribute of:

Object type, Spatial Coordinates, Object Registration, Reality Rating, Dynamic Scaling, Ownership, User Permissions, Content Rating, Physical properties, Learned Attributes, Related Objects and Functions, and the interaction between the at least two objects is based on an the value in each said attribute.

17. The method of claim 16 wherein object instance contains a value for at least a relational attribute and the relational attribute defines a relation of the object to a user, an environment or another object.

18. The method of claim 17 wherein the spatial coordinates attribute defines the position of an object relative to the relational attribute value.

19. The method of claim 16 further including accessing generic object libraries provided by a mixed reality service, the generic object libraries containing generic object definitions accessible by an object identifier, the object definitions used to create object instances.

20. The method of claim 19 further including providing user-specific object definitions accessible by an object identifier to a mixed reality service, the user-specific object definitions used to create object instances by a creating user and a shared user.