REALISTIC OCCLUSION FOR A HEAD MOUNTED AUGMENTED REALITY DISPLAY
First Claim
1. A method for displaying realistic occlusion between a real object and a virtual object by a head mounted, augmented reality display device system comprising:
- determining a spatial occlusion relationship exists between an occluding object and an occluded object including the real object and the virtual object based on overlapping three dimensional space (3D) positions of the objects in a 3D mapping of at least a user field of view of the display device system;
responsive to identifying the spatial occlusion relationship as a partial spatial occlusion between the real object and the virtual object, retrieving object boundary data of an occluding portion of the occluding object in the partial occlusion;
determining a level of detail for a model representing a partial occlusion interface based on level of detail criteria, the partial occlusion interface is the intersection at which object boundary data of an occluding portion of the occluding object is adjacent an unoccluded portion of an occluded object;
generating a model of the partial occlusion interface based on the retrieved object boundary data in accordance with the determined level of detail;
generating a modified version of boundary data of the virtual object adjacent an unoccluded portion of the real object based on the model, the generated adjacent boundary data having a shape based on the model; and
displaying an unoccluded portion of the virtual object in accordance with the modified version of its boundary data.
2 Assignments
0 Petitions
Accused Products
Abstract
Technology is described for providing realistic occlusion between a virtual object displayed by a head mounted, augmented reality display system and a real object visible to the user'"'"'s eyes through the display. A spatial occlusion in a user field of view of the display is typically a three dimensional occlusion determined based on a three dimensional space mapping of real and virtual objects. An occlusion interface between a real object and a virtual object can be modeled at a level of detail determined based on criteria such as distance within the field of view, display size or position with respect to a point of gaze. Technology is also described for providing three dimensional audio occlusion based on an occlusion between a real object and a virtual object in the user environment.
421 Citations
20 Claims
-
1. A method for displaying realistic occlusion between a real object and a virtual object by a head mounted, augmented reality display device system comprising:
-
determining a spatial occlusion relationship exists between an occluding object and an occluded object including the real object and the virtual object based on overlapping three dimensional space (3D) positions of the objects in a 3D mapping of at least a user field of view of the display device system; responsive to identifying the spatial occlusion relationship as a partial spatial occlusion between the real object and the virtual object, retrieving object boundary data of an occluding portion of the occluding object in the partial occlusion; determining a level of detail for a model representing a partial occlusion interface based on level of detail criteria, the partial occlusion interface is the intersection at which object boundary data of an occluding portion of the occluding object is adjacent an unoccluded portion of an occluded object; generating a model of the partial occlusion interface based on the retrieved object boundary data in accordance with the determined level of detail; generating a modified version of boundary data of the virtual object adjacent an unoccluded portion of the real object based on the model, the generated adjacent boundary data having a shape based on the model; and displaying an unoccluded portion of the virtual object in accordance with the modified version of its boundary data. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A see-through, augmented reality display device system for providing realistic occlusion comprising:
-
a see-through, augmented reality display having a user field of view and being supported by a support structure of the see-through, augmented reality display device; at least one camera supported by the support structure for capturing image data and depth data for real objects in the user field of view of the see-through, augmented reality display; the one or more processors communicatively coupled to the at least one camera for receiving image and depth data including the user field of view; the one or more software controlled processors for determining a spatial occlusion relationship between an occluding object and an occluded object, the occluding object and the occluded object including a real object and a virtual object, based on the image and depth data; and the one or more software controlled processors being communicatively coupled to the see-through, augmented reality display and the one or more processors causing the see-through display to represent the spatial occlusion relationship in the display by modifying display of the virtual object. - View Dependent Claims (11, 12, 13, 14, 15, 16)
-
-
17. One or more processor readable storage devices comprising instructions which cause one or more processors to execute a method for providing realistic audiovisual occlusion between a real object and a virtual object by a head mounted, augmented reality display device system, the method comprising:
-
determining a spatial occlusion relationship between a virtual object and a real object in an environment of a head mounted, augmented reality display device based on a three dimensional mapping of the environment of the display device system; determining whether an audio occlusion relationship exists between the virtual object and the real object; responsive to an audio occlusion relationship existing, modifying audio data for the occluded object based on one or more physical properties associated with an occluding object; and causing one or more earphones of the display device to output the modified audio data. - View Dependent Claims (18, 19, 20)
-
Specification