Virtual light in augmented reality

• US 9,551,871 B2
• Filed: 10/22/2014
• Issued: 01/24/2017
• Est. Priority Date: 12/01/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Sugden, Ben, Bennett, Darren, Mount, Brian, Sylvan, Sebastian, Tomlin, Arthur, Hastings, Ryan, McCulloch, Daniel, Geisner, Kevin, Crocco, Robert
• Primary Examiner(s): McDowell, Jr., Maurice L
• Application Number:
  US14/520,567
• Publication Number:
  US 20150035832A1
• Time in Patent Office: 825 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0112 comprising device for genereting colour display
• CPC Class Codes: G02B 2027/0118 comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 15/60 Shadow generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A head-mounted display system includes a see-through display that is configured to visually augment an appearance of a physical environment to a user viewing the physical environment through the see-through display. Graphical content presented via the see-through display is created by modeling the ambient lighting conditions of the physical environment.

Claims

1. A head-mounted display system, comprising:

an optical sensor system configured to observe ambient lighting conditions of a physical environment;

a position sensor system configured to assess a perspective of the optical sensor system; and

a see-through display configured to visually augment an appearance of the physical environment with a virtual object, the see-through display creating an illusion of a virtual shadow of the virtual object by virtually illuminating a non-shadow region bordering the virtual shadow, an appearance of the non-shadow region being based on the observed ambient lighting conditions.

2. The head-mounted display system of claim 1, wherein the virtual shadow is rendered with one or more omitted pixels.

3. The head-mounted display system of claim 1, wherein the non-shadow region is displayed with relatively more see-through lighting than the virtual shadow.

4. The head-mounted display system of claim 1, wherein the optical sensor system creates an ambient lighting model describing ambient lighting conditions of the physical environment, and wherein the virtual shadow is positioned in accordance with the ambient lighting model.

5. An augmented reality system, comprising:

an optical sensor system configured to observe ambient lighting conditions of a physical environment; and

a see-through display configured to visually augment an appearance of the physical environment with a virtual object and a graphical representation of a non-shadow region bordering a virtual shadow of the virtual object, an appearance of the graphical representation of the non-shadow region being based on the observed ambient lighting conditions.

6. The augmented reality system of claim 5, wherein the optical sensor system creates an ambient lighting model describing ambient lighting conditions of the physical environment, and wherein the virtual shadow is positioned in accordance with the ambient lighting model.

7. The augmented reality system of claim 5, wherein the see-through display virtually illuminates the non-shadow region.

8. The augmented reality system of claim 5, wherein the non-shadow region is displayed with relatively more see-through lighting than the virtual shadow.

9. The augmented reality system of claim 5, wherein the optical sensor system calculates a light vector from the observed ambient lighting conditions, and wherein the virtual shadow is positioned such that the light vector points from a perceived position of the virtual object to a perceived position of the virtual shadow.

10. The augmented reality system of claim 5, wherein the optical sensor system calculates a light intensity from the observed ambient lighting conditions, and wherein a brightness of the graphical representation of the non-shadow region is a function of the light intensity.

11. The augmented reality system of claim 5, further comprising a position sensor system configured to assess a perspective of the optical sensor system within the physical environment.

12. The augmented reality system of claim 11, wherein the graphical representation of the non-shadow region is changed as the perspective of the optical sensor system within the physical environment changes over time.

13. The augmented reality system of claim 5, wherein the optical sensor system is a first optical sensor system of a plurality of optical sensor systems within the physical environment, the plurality of optical sensor system including at least one off-board optical sensor system, the plurality of optical sensor system cooperating to create an ambient lighting model describing ambient lighting conditions of the physical environment.

14. An augmented reality system, comprising:

an optical sensor system configured to observe ambient lighting conditions of a physical environment; and

a see-through display configured to visually augment an appearance of the physical environment with a virtual object and a graphical representation of a non-shadow region bordering a virtual shadow of the virtual object, an appearance of the graphical representation of the non-shadow region being based on the observed ambient lighting conditions, and the graphical representation of the non-shadow region appearing to brighten an area exterior the virtual shadow while leaving the virtual shadow relatively darker.

15. The augmented reality system of claim 14, wherein the optical sensor system creates an ambient lighting model describing ambient lighting conditions of the physical environment.

16. The augmented reality system of claim 14, wherein the optical sensor system calculates a light vector from the observed ambient lighting conditions, and wherein the virtual shadow is positioned such that the light vector points from a perceived position of the virtual object to a perceived position of the virtual shadow.

17. The augmented reality system of claim 14, wherein the optical sensor system calculates a light intensity from the observed ambient lighting conditions, and wherein a brightness of the graphical representation of the non-shadow region is a function of the light intensity.

18. The augmented reality system of claim 14, further comprising a position sensor system configured to assess a perspective of the optical sensor system within the physical environment.

19. The augmented reality system of claim 18, wherein the graphical representation of the non-shadow region is changed as the perspective of the optical sensor system within the physical environment changes over time.

20. The augmented reality system of claim 14, wherein the optical sensor system is a first optical sensor system of a plurality of optical sensor systems within the physical environment, the plurality of optical sensor system including at least one off-board optical sensor system, the plurality of optical sensor system cooperating to create an ambient lighting model describing ambient lighting conditions of the physical environment.