Realistic scene lighting simulation

• US 6,685,326 B2
• Filed: 06/07/2002
• Issued: 02/03/2004
• Est. Priority Date: 06/08/2001
• Status: Active Grant

• Current Assignee: University of Southern California
• Original Assignee: University of Southern California
• Inventors: Hawkins, Timothy S., Debevec, Paul E.
• Primary Examiner(s): O'Shea, Sandra
• Assistant Examiner(s): Lee, Guiyoung
• Application Number:
  US10/165,371
• Publication Number:
  US 20020186555A1
• Time in Patent Office: 606 Days
• Field of Search: 362/233, 362/250, 362/276, 362/5, 362/11, 345/426
• US Class Current: 362/11
• CPC Class Codes: G03B 15/02 Illuminating scene
• CPC Class Codes: G03B 15/07 Arrangements of lamps in studios
• CPC Class Codes: H04N 5/2224 related to virtual studio applications
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/275 Generation of keying signals

Abstract

A lighting simulation process for simulating the lighting of a subject at a location in a scene. Scene illumination data is created that specifies the illumination that the scene provides at the location at which the subject is to be simulated from a plurality of spatial directions. Subject illumination data is created that specifies the appearance of the subject when illuminated from a plurality of different spatial directions. The subject is then illuminated as if at the location in the scene by combing the scene illumination data with the subject illumination data.

Claims

1. A lighting simulation process for simulating the lighting of a subject at a location in a scene comprising:

a) creating scene illumination data specifying the illumination that the scene provides at the location at which the subject is to be simulated from a plurality of spatial directions;

b) creating subject illumination data specifying the appearance of the subject when illuminated from a plurality of different spatial directions; and

c) rendering the subject illuminated as if at the location in the scene by combining the scene illumination data with the subject illumination data.

2. The lighting simulation process of claim 1 wherein the scene illumination data specifies the intensity and color of the illumination from each of the plurality of spatial directions.

3. The lighting simulation process of claim 1 wherein the subject illumination data is multiplied by the scene illumination data during said rendering.

4. The lighting simulation process of claim 3 wherein:

a) the scene illumination data includes pixel values;

b) the subject illumination data includes pixel values; and

c) pixel values of the subject illumination data from each spatial direction are multiplied by the pixel values of the scene illumination data from substantially the same spatial direction and summed over all spatial directions.

5. The lighting simulation process of claim 4 wherein each pixel contains values specifying color and intensity.

6. The lighting simulation process of claim 1 wherein said creating subject illumination data includes:

a) illuminating the subject with a small light directed towards the subject from a first spatial direction;

b) photographing the appearance of the subject under the illumination of the light from the first spatial direction;

c) illuminating the subject by a small light directed towards the subject from a second spatial direction different from the first spatial direction; and

d) photographing the appearance of the subject under the illumination of the light from the second spatial direction.

7. The lighting simulation process of claim 6 wherein steps (c) and (d) are repeated for additional spatial directions, each different from all of the others, until the totality of spatial directions from which an illuminating small light is directed approximates evenly-spaced locations on a sphere or dome.

8. The lighting simulation process of claim 7 wherein the spots of light are generated by an array of sequentially-fired light sources mounted on an arm that rotates around the subject.

9. The lighting simulation process of claim 6 wherein the small light is substantially white in color.

10. The lighting simulation process of claim 6 wherein:

a) a single light source is used to illuminate the subject from each spatial direction; and

b) the light source is successively moved to illuminate the subject from each spatial direction after each photographing.

11. The lighting simulation process of claim 10 wherein the light source is moved in a spiral path.

12. The lighting simulation process of claim 1 wherein the subject illumination data specifies the appearance of the subject from a single viewpoint.

13. The lighting simulation process of claim 1 wherein the subject illumination data specifies the appearance of the subject from a plurality of different viewpoints.

14. The lighting simulation process of claim 13 wherein said rendering the subject renders the subject from a viewpoint different from any of the viewpoints specified by the subject illumination data by extrapolating from the viewpoints that are specified by the subject illumination data.

15. The lighting simulation process of claim 1 wherein the subject includes a human face.

16. The lighting simulation process of claim 1 wherein the subject includes a cultural artifact.

17. A lighting simulation process for simulating the lighting of a subject at a location in a scene comprising:

a) creating subject illumination data specifying the appearance of the subject when illuminated from a plurality of different spatial directions; and

b) rendering the subject illuminated as if at the location in the scene by combining the subject illumination data with scene illumination data specifying the illumination that the scene provides at the location at which the subject is to be simulated from a plurality of spatial directions.

18. A lighting simulation process for simulating the lighting of a subject at a location in a scene comprising rendering the subject illuminated as if at the location in the scene by combining scene illumination data specifying the illumination that the scene provides at the location at which the subject is to be simulated from a plurality of spatial directions with subject illumination data specifying the appearance of the subject when illuminated from a plurality of different spatial directions.

19. Subject illumination data gathering apparatus for creating subject illumination data used in a lighting simulation process for simulating the lighting of the subject at a location in a scene comprising:

a) lighting apparatus for sequentially illuminating the subject with a small light from a sequence of different spatial directions;

b) a camera for taking a photograph of the subject when illuminated from each spatial direction; and

c) a recorder for recording each photograph and information reflecting the spatial direction from which the subject was illuminated at the time of each photograph.

20. The subject illumination data gathering apparatus of claim 19 wherein said lighting apparatus includes:

a) a light; and

b) movement apparatus that moves said light to each spatial direction.

21. The subject illumination data gathering apparatus of claim 20 wherein said moving apparatus moves said light in a spiral path.

22. The subject illumination data gathering apparatus of claim 19 wherein said lighting apparatus includes:

a) an arm;

b) a plurality of lights mounted on said arm;

c) rotation apparatus connected to said arm that rotates said arm; and

d) a strobing circuit that sequentially-fires each light on said arm as it rotates.

23. The subject illumination data gathering apparatus of claim 22 wherein said arm is semicircular.

24. The subject illumination data gathering apparatus of claim 19 wherein the spatial directions approximate evenly-spaced locations on a sphere or dome.

25. The subject illumination data gathering apparatus of claim 19 wherein the small light is substantially white.

26. The subject illumination data gathering apparatus of claim 19 including a plurality of cameras, each for taking a photograph of the subject when illuminated from each spatial direction from a viewpoint different from the viewpoint photographed by the other cameras.

27. The subject illumination data gathering apparatus of claim 26 wherein said recorder also records information indicative of the identity of the camera that took each photograph.

28. The subject illumination data gathering apparatus of claim 19 wherein said lighting apparatus includes a plurality of strobe lights and a D.C power supply for supplying pulses of D.C. power to each of said strobe lights.

29. The subject illumination data gathering apparatus of claim 19 further including subject-stabilizing apparatus for helping the subject remain still during the photographing.

30. The subject illumination data gathering apparatus of claim 29 wherein said subject-stabilizing apparatus includes a headrest.

31. Computer-readable media containing computer programming instructions which, when loaded in a computer system, cause the computer system to render a subject illuminated as if at a location in a scene by combining subject illumination data specifying the appearance of the subject when illuminated from a plurality of different spatial directions with scene illumination data specifying the illumination that the scene provides at the location at which the subject is to be illuminated from a plurality of spatial directions.