Reality-based light environment for digital imaging in motion pictures

• US 20040150641A1
• Filed: 11/17/2003
• Published: 08/05/2004
• Est. Priority Date: 11/15/2002
• Status: Active Grant

• Current Assignee: Warner Bros Entertainment Incorporated (Warner Bros. Discovery, Inc.)
• Original Assignee: ESC Entertainment Co.
• Inventors: Duiker, Haarm-Pieter
• Granted Patent: US 6,983,082 B2
• Time in Patent Office: Days
• US Class Current: 345/426
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2210/61 Scene description
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects

Abstract

Image-based lighting for use in rendering of digital objects is derived from image data collected from a real motion-picture set. The image data includes sets of correlated images at different exposure levels, each corresponding to a selected location. The image data is processed to define an extended dynamic range panoramic image for each selected set location. The image data is color and intensity corrected based on a scaled reference object. At least key lights are modeled for the set by processing a plurality of such panoramic images. Other lighting may be defined by interpolating from the panoramic images to mathematically define a location-specific set of fill lights. In the alternative, other set lighting may be modeled by projecting panoramic image data onto a dynamically sub-dividable light primitive derived from the set geometry. Any desired object may then rendered using the defined lights, and inserted into a corrected real image of the set at the desired location.

Claims

1. A method for defining lighting for rendering a digital object, the method comprising:

reading image datasets each corresponding to a unique coordinate within a limited physical space, each image dataset comprising a plurality of correlated images captured at different exposures;

defining a plurality of panoramic maps, each panoramic map comprising a map of color and intensity information derived from images in a corresponding one of the image sets; and

modeling a dynamic lighting model from the plurality of panoramic maps, the dynamic lighting model defining lighting for digital rendering as a function of virtual coordinates of a modeled space, wherein the modeled space corresponds to the limited physical space, and the virtual coordinates are selected from the group consisting of space coordinates, time coordinates, and any combination of space and time coordinates.

2. The method of claim 1, wherein the reading step further comprises reading the unique coordinate defining a location within the defined physical space, and wherein the modeling step further comprises defining the dynamic lighting model as a function of the virtual coordinates that define different locations within the modeled space.

3. The method of claim 1, wherein the reading step further comprises reading the unique coordinate defining a time at a selected location, and wherein the modeling step further comprises defining the dynamic lighting model as a function of the virtual coordinates that define different times.

4. The method of claim 1, wherein the modeling step further comprises modeling the dynamic lighting model as a time-independent function.

5. The method of claim 1, wherein the modeling step further comprises modeling the dynamic lighting model as a time-dependent function.

6. The method of claim 1, further comprising reading an image calibration dataset comprising images of a defined reference object linked to an identifier of a camera by which each image was captured.

7. The method of claim 1, wherein the second reading step further comprises reading the image calibration dataset comprising images of the defined reference object selected from a 20-step gray scale and a Macbeth color chart.

8. The method of claim 1, further comprising color-correcting a plurality of images of the limited physical space based on an analysis of images of the defined reference object.

9. The method of claim 1, wherein the defining step further comprises defining the plurality of panoramic maps using the images of the defined reference object to determine characteristics of at least one camera.

10. The method of claim 1, wherein the reading step further comprises reading the image datasets comprising images of a convex specularly-reflecting surface.

11. The method of claim 1, wherein the modeling step further comprises modeling the dynamic lighting model comprising at least one modeled key light, and at least one modeled fill light.

12. The method of claim 1, wherein the modeling step further comprises modeling the dynamic lighting model comprising an modeled light-emitting surface surrounding a digital object to be rendered.

13. The method of claim 12, wherein the modeling step further comprises subdividing the modeled light-emitting surface into surface regions based on a geometric relationship between a digital object to be rendered and the modeled light-emitting surface.

14. The method of claim 1, wherein the modeling step further comprises interpolating an object-specific panorama from the plurality of panoramic maps.

15. The method of claim 1, wherein the modeling step further comprises determining at least one modeled key light and at least one modeled fill light from the object-specific panorama.

16. A method for defining lighting for rendering a digital object, the method comprising:

reading at least one image dataset corresponding to a unique coordinate within a limited physical space, the image dataset comprising a plurality of correlated images captured at different exposures;

defining at least one panoramic map comprising a map of color and intensity information derived from images in a corresponding one of the image sets; and

modeling a lighting model from the panoramic map, the lighting model defining lighting for digital rendering as at least one modeled key light and at least one modeled fill light.

17. A computer-readable media containing instructions for defining lighting for rendering a digital object, the instructions comprising:

reading image datasets each corresponding to a unique coordinate within a limited physical space, each image dataset comprising a plurality of correlated images captured at different exposures;

defining a plurality of panoramic maps, each panoramic map comprising a map of color and intensity information derived from images in a corresponding one of the image sets; and

modeling a dynamic lighting model from the plurality of panoramic maps, the dynamic lighting model defining lighting for digital rendering as a function of virtual coordinates of a modeled space, wherein the modeled space corresponds to the limited physical space, and the virtual coordinates are selected from the group consisting of space coordinates, time coordinates, and any combination of space and time coordinates.

18. The computer-readable media of claim 17, wherein the modeling instruction further comprises modeling the dynamic lighting model as a time-independent function.

19. The computer-readable media of claim 17, wherein the modeling instruction further comprises modeling the dynamic lighting model as a time-dependent function.

20. The computer-readable media of claim 17, further comprising an instruction for reading an image calibration dataset comprising images of a defined reference object linked to an identifier of a camera by which each image was captured.

21. The computer-readable media of claim 20, further comprising an instruction for color-correcting a plurality of images of the limited physical space based on an analysis of images of the defined reference object.

22. The computer-readable media of claim 20, wherein the defining instruction further comprises defining the plurality of panoramic maps using the images of the defined reference object to determine characteristics of at least one camera.

23. The computer-readable media of claim 17, wherein the reading instruction further comprises reading the image datasets comprising images of a convex specularly-reflecting surface.

24. The computer-readable media of claim 17, wherein the modeling instruction further comprises modeling the dynamic lighting model comprising at least one modeled key light, and at least one modeled fill light.

25. The computer-readable media of claim 17, wherein the modeling instruction further comprises modeling the dynamic lighting model comprising a modeled light-emitting surface surrounding a digital object to be rendered.

26. The computer-readable media of claim 25, wherein the modeling instruction further comprises subdividing the modeled light-emitting surface into surface regions based on a geometric relationship between a digital object to be rendered and the modeled light-emitting surface.

27. The computer-readable media of claim 17, wherein the modeling step further comprises interpolating an object-specific panorama from the plurality of panoramic maps.

28. The computer-readable media of claim 27, wherein the modeling instruction further comprises determining at least one modeled key light and at least one modeled fill light from the object-specific panorama.