Realistic scene illumination reproduction

• US 7,044,613 B2
• Filed: 06/07/2002
• Issued: 05/16/2006
• Est. Priority Date: 06/08/2001
• Status: Active Grant

• Current Assignee: University of Southern California
• Original Assignee: University of Southern California
• Inventors: Debevec, Paul E.
• Primary Examiner(s): Ometz, David
• Assistant Examiner(s): HENDERSON, ADAM
• Application Number:
  US10/165,359
• Publication Number:
  US 20020186314A1
• Time in Patent Office: 1,439 Days
• Field of Search: 362/11, 362/5
• US Class Current: 362/11
• CPC Class Codes: G03B 15/02 Illuminating scene
• CPC Class Codes: G03B 15/07 Arrangements of lamps in studios
• CPC Class Codes: H04N 5/2224 related to virtual studio applications
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/275 Generation of keying signals

Abstract

An illumination reproduction apparatus and process for illuminating a subject with the illumination at a location in a scene when the subject is not in the scene. Scene illumination data is generated that specifies the illumination at the location in the scene from a plurality of spatial directions. The subject is then illuminated with the illumination at the location in the scene by driving a plurality of light sources surrounding the subject with the scene illumination data.

Claims

1. An illumination reproduction process for illuminating a real subject with the illumination at a location in a scene when the real subject is not in the scene comprising:

a) generating scene illumination data specifying the illumination at the location in the scene from a plurality of spatial directions; and

b) illuminating the real subject with the illumination at the location in the scene by driving a plurality of real light sources surrounding the real subject with the scene illumination data.

2. The illumination reproduction process of claim 1 wherein the scene illumination data specifies the intensity of the illumination from each of the spatial directions.

3. The illumination reproduction process of claim 2 wherein each of the light sources has a spatial direction relative to the subject and is driven to an intensity that is substantially proportional to the intensity of the illumination from substantially the same spatial direction in the scene.

4. The illumination reproduction process of claim 3 wherein the scene illumination data also specifies the color of the illumination from each of the spatial directions.

5. The illumination reproduction process of claim 4 wherein the color of the light source from each spatial direction relative to the subject is substantially the same as the color of the illumination from substantially the same spatial direction in the scene.

6. The illumination reproduction process of claim 1 further comprising:

a) creating a scene image comprising an image of the scene at the location in the scene at which the subject is to be illuminated;

b) photographing the subject under the illumination of the light sources;

c) compositing the photograph of the subject with the scene image at the location in the scene.

7. The illumination reproduction process of claim 6 wherein:

a) the subject is photographed against a background with a camera;

b) the background is illuminated with a non visible light during said photographing;

c) portions of the background are blocked from the photograph by the subject; and

d) the scene image is substituted for the portions of the illuminated background that are not blocked from the camera.

8. The illumination reproduction process of claim 7 wherein the non-visible light is in the infrared range.

9. The illumination reproduction process of claim 1 wherein the subject includes a human face.

10. The illumination reproduction process of claim 1 wherein the scene is a real scene and wherein a light probe is used in said generating scene illumination data.

11. The illumination reproduction process of claim 1 wherein the illumination in the scene varies as a function of time, wherein said generating scene illumination data specifies the illumination at various times, and wherein said driving causes the light sources to illuminate the subject with the illumination in the scene at the various times.

12. The illumination reproduction process of claim 1 wherein the light sources are arranged in a spherical or dome-like configuration.

13. An illumination reproduction process for illuminating a real subject with the illumination at a location in a scene when the real subject is not in the scene comprising illuminating the real subject with the illumination at the location in the scene by a plurality of real light sources surrounding the real subject being driven with scene illumination data specifying the illumination at the location in the scene from a plurality of spatial directions.

14. An illumination apparatus for illuminating a subject with the illumination at a location in a scene when the subject is not in the scene comprising:

a) a plurality of light sources surrounding the subject;

b) a driver in communication with said light sources for driving said light sources with scene illumination data specifying the illumination that the scene provides at the location in the scene from a plurality of spatial directions.

15. The illumination apparatus of claim 14 wherein the scene illumination data specifies the intensity of the illumination from each of the spatial directions.

16. The illumination apparatus of claim 15 wherein:

a) each of said light sources has a spatial direction relative to the subject; and

b) said driver causes the intensity of each of said light sources to be substantially proportional to the intensity of the illumination in the scene from substantially the same spatial direction.

17. The illumination apparatus of claim 16 wherein the scene illumination data also specifies the color of the illumination from each of the spatial directions.

18. The illumination apparatus of claim 16 wherein said driver causes the color of the light source from each spatial direction relative to the subject to be substantially the same as the color of the illumination from substantially the same spatial direction in the scene.

19. The illumination apparatus of claim 18 wherein each of said light sources includes a plurality of lights, each with a different principal color.

20. The illumination apparatus of claim 19 wherein each of said light sources includes LEDs.

21. The illumination apparatus of claim 19 wherein each of said light sources includes a plurality of lights of each different principal color.

22. The illumination apparatus of claim 21 wherein said light sources includes LEDs.

23. The illumination apparatus of claim 14 further including:

a) a background positioned behind the subject; and

b) background illumination for illuminating the background with non-visible light.

24. The illumination apparatus of claim 23 wherein said background illumination emits illumination in the infrared range.

25. The illumination apparatus of claim 14 wherein the subject includes a human face.

26. The illumination apparatus of claim 14 wherein the illumination in the scene varies as a function of time, wherein the scene illumination data specifies the illumination at various times, and wherein said driver causes the light sources to illuminate the subject in a manner that is similar to the lighting in the scene at the various times.

27. The illumination apparatus of claim 14 wherein said light sources are arranged in a spherical or dome-like configuration.

28. The illumination apparatus of claim 14 wherein said driver includes a processor and a data storage system for storing the scene illumination data.

29. An image compositing system for producing an image of a scene containing a subject that was not photographed in the scene, but illuminated as though the subject was in the scene, comprising:

a) an illumination apparatus for illuminating the subject based on scene illumination data specifying the illumination that the scene provides at a location in the scene at which the subject is to be illuminated from a plurality of spatial directions;

b) a camera for photographing the subject while illuminated by said illumination apparatus; and

c) a compositing system for compositing the photograph of the subject from said camera in the scene.