AUGMENTED REALITY WITH REALISTIC OCCLUSION

• US 20130141419A1
• Filed: 12/01/2011
• Published: 06/06/2013
• Est. Priority Date: 12/01/2011
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Mount, Brian, Latta, Stephen, McCulloch, Daniel, Geisner, Kevin, Scott, Jason, Steed, Jonathan, Tomlin, Arthur, Mihelich, Mark
• Application Number:
  US13/309,372
• Publication Number:
  US 20130141419A1
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/847 Cooperative playing, e.g. requiring coordinated actions from several players to achieve a common goal
• CPC Class Codes: A63F 2300/6653 for altering the visibility of an object, e.g. preventing the occlusion of an object, partially hiding an object
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 3/003 to produce spatial visual effects

Abstract

A head-mounted display device is configured to visually augment an observed physical space to a user. The head-mounted display device includes a see-through display and is configured to receive augmented display information, such as a virtual object with occlusion relative to a real world object from a perspective of the see-through display.

Claims

1. A method of augmenting reality, the method comprising:

receiving first observation information of a first physical space from a first head-mounted display device, the first head-mounted display device including a first see-through display configured to visually augment an appearance of the first physical space to a user viewing the first physical space through the first see-through display;

receiving second observation information of a second physical space from a second head-mounted display device, the second head-mounted display device including a second see-through display configured to visually augment an appearance of the second physical space to a user viewing the second physical space through the second see-through display;

mapping a shared virtual reality environment to the first physical space and the second physical space based on the first observation information and the second observation information, the shared virtual reality environment including a virtual object;

sending first augmented reality display information to the first head mounted display, the first augmented reality display information configured to display the virtual object via the first see-through display with occlusion relative to a real world object from a perspective of the first see-through display.

2. The method of claim 1, where the first physical space and the second physical space are congruent, and where the first observation information is from a first perspective of the first see-through display and the second observation information is from a second perspective of the second see-through display, the first perspective being different than the second perspective.

3. The method of claim 2, where the shared virtual reality environment is mapped such that the virtual object appears to be located in a same physical space from both the first perspective and the second perspective.

4. The method of claim 1, where the first physical space and the second physical space are incongruent.

5. The method of claim 4, where the first augmented reality display information is configured to display within the shared virtual reality environment a second real world object that is physically present in the second physical space but not physically present in the first physical space.

6. The method of claim 1, where a mapped position of the real world object is between the virtual object and the first see-through display, and where the first augmented reality display information is configured to display only those portions of the virtual object that are not behind the real world object from the perspective of the first see-through display.

7. The method of claim 1, where a mapped position of the real world object is behind the virtual object from the perspective of the first see-through display, and where the first augmented reality display information is configured to display the virtual object with sufficient opacity so as to substantially block sight of the real world object through the first see-through display.

8. The method of claim 1, where mapping the shared virtual reality environment includes transforming a coordinate system of the first physical space from the perspective of the first see-through display and a coordinate system of the second physical space from a perspective of the second see-through display to a shared coordinate system.

9. The method of claim 1, where mapping the shared virtual reality environment includes transforming a coordinate system of the second physical space from a perspective of the second see-through display to a coordinate system of the first physical space from the perspective of the first see-through device.

10. The method of claim 1, further comprising sending second augmented reality display information to the second head mounted display, the second augmented reality display information configured to display the virtual object via the second see-through display with occlusion relative to the real world object from a perspective of the second see-through display.

11. The method of claim 1, where the first observation information is collected by a sensor subsystem of the first head mounted display device.

12. The method of claim 1, where the sensor subsystem includes a depth camera imaging the first physical space.

13. The method of claim 1, where the sensor subsystem includes a visible light camera imaging the first physical space.

14. The method of claim 1, where the shared virtual reality environment includes a surface reconstructed object, the surface reconstructed object originating from the first physical space or the second physical space, the surface reconstructed object having a mapped position within a shared coordinate system of the shared virtual reality environment.

15. A data-holding subsystem holding instructions executable by a logic subsystem to:

receive first observation information of a first physical space from a first head-mounted display device, the first head-mounted display device including a first see-through display configured to visually augment an appearance of the first physical space to a user viewing the first physical space through the first see-through display;

map a virtual reality environment to the first physical space based on the first observation information, the shared virtual reality environment including a virtual object;

send first augmented reality display information to the first head mounted display, the first augmented reality display information configured to display the virtual object via the first see-through display with occlusion relative to a real world object from a perspective of the first see-through display.

16. The system of claim 15, where a mapped position of the real world object is between the virtual object and the first see-through display, and where the first augmented reality display information is configured to display only those portions of the virtual object that are not behind the real world object from the perspective of the first see-through display.

17. The system of claim 15, where a mapped position of the real world object is behind the virtual object from the perspective of the first see-through display, and where the first augmented reality display information is configured to display the virtual object with sufficient opacity so as to substantially block sight of the real world object through the first see-through display.

18. The system of claim 15, where mapping the shared virtual reality environment includes transforming a coordinate system of the first physical space from the perspective of the first see-through display to a shared coordinate system.

19. The system of claim 15, where the first observation information is collected by a sensor subsystem of the first head mounted display device, the sensor subsystem including a depth camera imaging the first physical space.

20. A method of augmenting reality, the method comprising:

receiving first observation information of a first physical space from a first head-mounted display device, the first head-mounted display device including a first see-through display configured to visually augment an appearance of the first physical space to a user viewing the first physical space through the first see-through display;

receiving second observation information of a second physical space from a second head-mounted display device, the second head-mounted display device including a second see-through display configured to visually augment an appearance of the second physical space to a user viewing the second physical space through the second see-through display;

mapping a shared virtual reality environment to the first physical space and the second physical space based on the first observation information and the second observation information, the shared virtual reality environment including a virtual object, a mapped position of a first real world object in the first physical space being between the virtual object and the first see-through display from a perspective of the first see-through display and being behind the virtual object from a perspective of the second see-through display, a mapped position of a second real world object in the second physical space being between the virtual object and the second see-through display from a perspective of the second see-through display and being behind the virtual object from a perspective of the first see-through display;

sending first augmented reality display information to the first head mounted display, the first augmented reality display information configured to display only those portions of the virtual object that are not behind the first real world object from the perspective of the first see-through display and to display the virtual object blocking the second real world object; and

sending second augmented reality display information to the second head mounted display, the second augmented reality display information configured to display only those portions of the virtual object that are not behind the second real world object from the perspective of the second see-through display and to display the virtual object blocking the first real world object.