Virtual light in augmented reality

• US 10,083,540 B2
• Filed: 01/23/2017
• Issued: 09/25/2018
• Est. Priority Date: 12/01/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Sugden, Ben, Bennett, Darren, Mount, Brian, Sylvan, Sebastian, Tomlin, Arthur, Hastings, Ryan, McCulloch, Daniel, Geisner, Kevin, Crocco, Jr., Robert
• Primary Examiner(s): McDowell, Jr., Maurice L
• Application Number:
  US15/412,888
• Publication Number:
  US 20170161939A1
• Time in Patent Office: 610 Days
• CPC Class Codes: G02B 2027/0112 comprising device for genereting colour display
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 15/60 Shadow generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A head-mounted display system includes a see-through display that is configured to visually augment an appearance of a physical environment to a user viewing the physical environment through the see-through display. Graphical content presented via the see-through display is created by modeling the ambient lighting conditions of the physical environment.

Claims

1. A method for a head mounted display (HMD), comprising:

observing ambient lighting conditions of a physical environment;

assessing a perspective of the HMD;

visually augmenting an appearance of the physical environment with a virtual object; and

creating an illusion of a virtual shadow of the virtual object by virtually illuminating a non-virtual-object, non-shadow region bordering the virtual shadow, an appearance of the non-virtual-object, non-shadow region being based on the observed ambient lighting conditions.

2. The method of claim 1, wherein the virtual shadow is rendered with one or more omitted pixels.

3. The method of claim 1, wherein the non-shadow region is displayed with relatively more see-through lighting than the virtual shadow.

4. The method of claim 1, wherein the virtual shadow is positioned in accordance with an ambient lighting model describing ambient lighting conditions of the physical environment.

5. A method for a head mounted display (HMD), comprising:

observing ambient lighting conditions of a physical environment; and

visually augmenting an appearance of the physical environment with a virtual object and a graphical representation of a non-virtual-object, non-shadow region bordering a virtual shadow of the virtual object, an appearance of the graphical representation of the non-virtual-object, non-shadow region being based on the observed ambient lighting conditions.

6. The method of claim 5, wherein the virtual shadow is positioned in accordance with an ambient lighting model describing ambient lighting conditions of the physical environment.

7. The method of claim 6, wherein the ambient lighting model is created based on ambient lighting conditions observed by a plurality of optical sensor systems, and at least one of the plurality of optical sensors systems is separate from the HMD.

8. The method of claim 5, wherein the non-shadow region is visually illuminated.

9. The method of claim 5, wherein the non-shadow region is displayed with relatively more see-through lighting than the virtual shadow.

10. The method of claim 5, further comprising calculating a light vector from the observed ambient lighting conditions, and wherein the virtual shadow is positioned such that the light vector points from a perceived position of the virtual object to a perceived position of the virtual shadow.

11. The method of claim 5, further comprising calculating a light intensity from the observed ambient lighting conditions, and wherein a brightness of the graphical representation of the non-shadow region is a function of the light intensity.

12. The method of claim 5, further comprising assessing a perspective of the HMD within the physical environment.

13. The method of claim 12, wherein the graphical representation of the non-shadow region is changed as the perspective of the HMD within the physical environment changes over time.

14. A method for a head mounted display (HMD), comprising:

observing ambient lighting conditions of a physical environment; and

visually augmenting an appearance of the physical environment with a virtual object and a graphical representation of a non-virtual-object, non-shadow region bordering a virtual shadow of the virtual object, an appearance of the graphical representation of the non-virtual-object, non-shadow region being based on the observed ambient lighting conditions, and the graphical representation of the non-virtual-object, non-shadow region appearing to brighten an area exterior the virtual shadow while leaving the virtual shadow relatively darker.

15. The method of claim 14, further comprising creating an ambient lighting model describing ambient lighting conditions of the physical environment.

16. The method of claim 15, wherein the ambient lighting model is created based on ambient lighting conditions observed by a plurality of optical sensor systems, and at least one of the plurality of optical sensors systems is separate from the HMD.

17. The method of claim 14, further comprising calculating a light vector from the observed ambient lighting conditions, and wherein the virtual shadow is positioned such that the light vector points from a perceived position of the virtual object to a perceived position of the virtual shadow.

18. The method of claim 14, further comprising calculating a light intensity from the observed ambient lighting conditions, and wherein a brightness of the graphical representation of the non-shadow region is a function of the light intensity.

19. The method of claim 14, further comprising assessing a perspective of the HMD within the physical environment.

20. The method of claim 19, wherein the graphical representation of the non-shadow region is changed as the perspective of the HMD within the physical environment changes over time.