Display of shadows via see-through display

• US 9,311,751 B2
• Filed: 12/12/2011
• Issued: 04/12/2016
• Est. Priority Date: 12/12/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Lamb, Mathew
• Primary Examiner(s): Hajnik, Daniel
• Application Number:
  US13/323,403
• Publication Number:
  US 20130147826A1
• Time in Patent Office: 1,583 Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 15/60 Shadow generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2320/02 Improving the quality of display appearance
• CPC Class Codes: G09G 2320/0242 Compensation of deficiencies in the appearance of colours
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 3/003 to produce spatial visual effects

Abstract

Various embodiments are disclosed that relate to the presentation of images of shadows cast over a real-world background by images of objects displayed on a see-through display. For example, one disclosed embodiment provides a method of operating a display device having a see-through display screen. The method comprises displaying on the see-through display screen an image of an object, and while displaying the image of the object, displaying an image of a shadow cast by the object onto the background scene by acquiring an image of a background scene, determining a location of the shadow in the image of the background scene, rendering an enhanced image of the background scene by increasing a relative brightness in a region adjacent to the shadow compared to a brightness within the shadow, and displaying the enhanced image of the background scene.

Claims

1. A method of operating an optical see-through display device having a see-through display screen and a camera, the method comprising:

displaying on the see-through display screen an image of an object; and

while displaying the image of the object, displaying an image of a shadow cast by the object onto a real-world background scene by

acquiring via the camera an image of the real-world background scene viewable through the see-through display screen,

determining a location of the shadow in the image of the real-world background scene,

rendering the image of the real-world background scene to form an enhanced image of the real-world background scene by increasing a brightness of the real-world background scene in the enhanced image of the real-world background scene in a region adjacent to the shadow compared to a brightness within the shadow, and

displaying via the see-through display screen the enhanced image of the real-world background scene over and aligned with the real-world background scene.

2. The method of claim 1, further comprising reducing an amount of natural light passing through the see-through display screen via a neutral density filter.

3. The method of claim 1, wherein rendering the enhanced image of the real-world background scene comprises not increasing the brightness within the shadow.

4. The method of claim 1, wherein rendering the enhanced image of the real-world background scene comprises darkening the image of the real-world background scene globally.

5. The method of claim 1, wherein increasing the brightness in the region adjacent to the shadow comprises increasing a brightness of non-shadow regions globally.

6. The method of claim 1, wherein increasing the brightness in the region adjacent to the shadow comprises increasing the brightness locally such that a brightness differential decreases as a function of distance from the shadow.

7. The method of claim 6, further comprising detecting an edge in the image of the real-world background scene that exceeds a preselected threshold gradient, and terminating the increase in the brightness at the edge.

8. The method of claim 1, further comprising determining a color distribution of ambient light, and increasing the brightness by increasing the brightness based upon the color distribution of the ambient light.

9. The method of claim 1, wherein displaying the image of the object and the image of the shadow cast by the object comprises displaying the image of the object and the image of the shadow cast by the object on a head-mounted display system.

10. An optical see-through head-mounted display system, comprising:

a see-through display screen;

an image production system configured to display an image on the see-through display screen;

an image sensor configured to acquire an image of a real-world background scene viewable through the see-through display screen;

a logic subsystem; and

a data-holding subsystem comprising instructions executable by the logic subsystem to

display an image of an object on the see-through display screen; and

while displaying the image of the object, display an image of a shadow cast by the object onto the real-world background scene by

acquiring via the image sensor the image of the real-world background scene viewable through the see-through display screen,

determining a location of the shadow in the image of the real-world background scene,

rendering the image of the real-world background scene to form an enhanced image of the real-world background scene in which a brightness of the real-world background scene in the enhanced image of the real-world background scene in a region adjacent to the shadow is increased compared to a brightness within the shadow, and

displaying the enhanced image of the real-world background scene on the see-through display screen over and aligned with the real-world background scene.

11. The head-mounted display system of claim 10, wherein the see-through display screen further comprises a neutral density filter.

12. The head-mounted display system of claim 10, wherein the instructions are executable to render the enhanced image of the real-world background scene by increasing the brightness in the region adjacent to the shadow while not increasing the brightness within the shadow.

13. The head-mounted display system of claim 10, wherein the instructions are executable to increase the brightness in the region adjacent to the shadow by increasing a brightness of non-shadow regions globally.

14. The head-mounted display system of claim 10, wherein the instructions are executable to increase the brightness in the region adjacent to the shadow by increasing the brightness locally such that a brightness differential decreases as a function of distance from the shadow.

15. The head-mounted display system of claim 14, wherein the instructions are executable to detect an edge in the image of the real-world background scene that exceeds a preselected threshold gradient, and terminate the increase in the brightness at the edge.

16. The head-mounted display system of claim 10, wherein the instructions are further executable to determine a color distribution of ambient light, and to increase the brightness by increasing the brightness based upon the color distribution of the ambient light.

17. On an optical see-through head-mounted display system comprising a see-through display screen, a camera, and a neutral density filter, a method of displaying an image of an object and an image of a shadow of the object over a real-world background scene viewable through the see-through display screen, the method comprising:

displaying on the see-through display screen the image of the object;

reducing an amount of ambient light passing through the see-through display screen via the neutral density filter; and

while displaying the image of the object, displaying an image of a shadow of the object onto the real-world background scene by

acquiring via the camera an image of the real-world background scene viewable through the see-through display screen,

determining a location of the shadow in the image of the real-world background scene,

rendering the acquired image of the real-world background scene to form an enhanced image of the real-world background scene by

increasing a brightness of the real-world background scene in the enhanced image of the real-world background scene in non-shadow regions of the image and not increasing a brightness within the shadow, and

displaying the enhanced image of the real-world background scene on the see-through display screen over and aligned with the real-world background scene such that light is added to the non-shadow regions.

18. The method of claim 17, wherein rendering the enhanced image of the real-world background scene comprises darkening the image of the real-world background scene globally before brightening the region adjacent to the shadow.

19. The method of claim 17, further comprising determining a color distribution of ambient light, and increasing the brightness by increasing the brightness based upon the color distribution of the ambient light.