Estimation of Light Color and Direction for Augmented Reality Applications

• US 20110063295A1
• Filed: 09/14/2009
• Published: 03/17/2011
• Est. Priority Date: 09/14/2009
• Status: Active Grant

• Current Assignee: Autodesk, Inc.
• Original Assignee: Autodesk, Inc.
• Inventors: Pene, Brian Anthony, Kuo, Eddy Yim
• Granted Patent: US 8,405,658 B2
• Time in Patent Office: Days
• US Class Current: 345/426
• CPC Class Codes: G06T 15/50 Lighting effects
• CPC Class Codes: G06T 15/60 Shadow generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

One embodiment of the invention sets forth a technique for shading a graphics object inserted into a video feed of a real-world scene based on lighting conditions in the real-world scene. The real-world scene includes a fiducial marker denotes the location in the video feed where the graphics object should be inserted. In order to shade the graphics object, the AR application computes light color values at multiple points on the fiducial marker. The color computation module also computes the direction of light cast on the fiducial marker by determining the direction of the shadow cast by a pyramid object on the fiducial marker. The AR application then shades the graphics object inserted into the video feed at the location of the fiducial marker based on the light color values and the direction of light.

Claims

1. A computer-implemented method for lighting a graphics object inserted into a video image of a real-world scene that includes a fiducial marker and a light source emitting light on the fiducial marker, the method comprising:

analyzing video data associated with the video image to determine a location of the fiducial marker within the real-world scene;

computing a color value associated with the light emitted at a first point on the fiducial marker based on a light intensity value associated with the first point and specified by the video data;

determining a location of a shadow cast by an object placed on the fiducial marker within the real-world scene by analyzing one or more light intensity values associated with a set of points surrounding the fiducial marker and specified by the video data;

computing a direction of the emitted light based on the location of the shadow cast by the object; and

modifying one or more lighting properties associated with the graphics object based on the color value of the light emitted at the first point and the direction of the emitted light.

2. The method of claim 1, further comprising the step of calibrating a video camera that captures the video image to determine a set of calibrated color values.

3. The method of claim 2, wherein the step of computing the color value is further based on the set of calibrated color values.

4. The method of claim 1, further comprising the step of inserting the graphics object into the video image at a location corresponding to the location of the fiducial marker within the real-world scene.

5. The method of claim 4, further comprising the step of transmitting the video image to a display device for display or to a storage device for storage.

6. The method of claim 1, wherein the object placed on the fiducial marker comprises a pyramid.

7. The method of claim 1, further comprising the step of computing a second color value associated with the light emitted at a second point on the fiducial marker based on a second light intensity value associated with the second point and specified by the video data.

8. The method of claim 7, wherein the step of modifying the one or more lighting properties is further based on the second color value of the light at the second point.

9. A computer-readable storage medium storing instruction that, when executed by a processor, cause a computing system to light a graphics object inserted into a video image of a real-world scene that includes a fiducial marker and a light source emitting light on the fiducial marker, by performing the steps of:

analyzing video data associated with the video image to determine a location of the fiducial marker within the real-world scene;

computing a color value associated with the light emitted at a first point on the fiducial marker based on a light intensity value associated with the first point and specified by the video data;

determining a location of a shadow cast by an object placed on the fiducial marker within the real-world scene by analyzing one or more light intensity values associated with a set of points surrounding the fiducial marker and specified by the video data;

computing a direction of the emitted light based on the location of the shadow cast by the object; and

modifying one or more lighting properties associated with the graphics object based on the color value of the light emitted at the first point and the direction of the emitted light.

10. The computer-readable storage medium of claim 9, further comprising the step of calibrating a video camera that captures the video image to determine a set of calibrated color values.

11. The computer-readable storage medium of claim 10, wherein the step of computing the color value is further based on the set of calibrated color values.

12. The computer-readable storage medium of claim 9, further comprising the step of inserting the graphics object into the video image at a location corresponding to the location of the fiducial marker within the real-world scene.

13. The computer-readable storage medium of claim 12, further comprising the step of transmitting the video image to a display device for display or to a storage device for storage.

14. The computer-readable storage medium of claim 9, wherein the object placed on the fiducial marker comprises a pyramid.

15. The computer-readable storage medium of claim 9, further comprising the step of computing a second color value associated with the light emitted at a second point on the fiducial marker based on a second light intensity value associated with the second point and specified by the video data.

16. The computer-readable storage medium of claim 15, wherein the step of modifying the one or more lighting properties is further based on the second color value of the light at the second point.

17. A computer system for lighting a graphics object inserted into a video image of a real-world scene that includes a fiducial marker and a light source emitting light on the fiducial marker, the computer system comprising:

a processor; and

a memory storing instructions that when executed by the processor are configured to;

analyze video data associated with the video image to determine a location of the fiducial marker within the real-world scene;

compute a color value associated with the light emitted at a first point on the fiducial marker based on a light intensity value associated with the first point and specified by the video data;

determine a location of a shadow cast by an object placed on the fiducial marker within the real-world scene by analyzing one or more light intensity values associated with a set of points surrounding the fiducial marker and specified by the video data;

compute a direction of the emitted light based on the location of the shadow cast by the object; and

modify one or more lighting properties associated with the graphics object based on the color value of the light emitted at the first point and the direction of the emitted light.

18. The computer system of claim 17, wherein the instructions are further configured to calibrate a video camera that captures the video image to determine a set of calibrated color values.

19. The computer system of claim 18, wherein computing the color value is further based on the set of calibrated color values.

20. The computer system of claim 17, wherein the instructions are further configured to insert the graphics object into the video image at a location corresponding to the location of the fiducial marker within the real-world scene.