Lighting and shadowing methods and arrangements for use in computer graphic simulations

• US 6,362,822 B1
• Filed: 03/12/1999
• Issued: 03/26/2002
• Est. Priority Date: 03/12/1999
• Status: Expired due to Term

• Current Assignee: Terminal Reality Inc.
• Original Assignee: Terminal Reality Inc.
• Inventors: Randel, Mark R.
• Primary Examiner(s): Vo, Cliff N.
• Application Number:
  US09/268,078
• Time in Patent Office: 1,110 Days
• Field of Search: 345/418, 345/419, 345/425, 345/426, 345/427, 345/114, 345/117, 345/118
• US Class Current: 345/426
• CPC Class Codes: G06T 15/405 using Z-buffer
• CPC Class Codes: G06T 15/506 Illumination models
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects

Abstract

The effects of lighting and resulting shadows within a computer simulated three-dimensional scene are modeled by rendering a light depth image and a light color image for each of the light sources. The light depth images are compared to a camera depth image to determine if a point within the scene is lighted by the various light sources. An accumulated light image is produced by combining those portions of the light color images determined to be lighting the scene. The resulting accumulated light image is then combined with a camera color image to produce a lighted camera image that can be further processed and eventually displayed on a computer display screen. The light color image can be static or dynamic. Transformations between different perspective and/or coordinate systems can be precalculated for fixed cameras or light sources. The various images and manipulations can include individual pixel data values, multiple-pixel values, polygon values, texture maps, and the like.

Claims

1. A shadow rendering method for use in a computer system, the method comprising the steps of:

providing observer data of a simulated multi-dimensional scene;

providing lighting data associated with a plurality of simulated light sources arranged to illuminate said scene, said lighting data including light image data;

for each of said plurality of light sources, comparing at least a portion of said observer data with at least a portion of said lighting data to determine if a modeled point within said scene is illuminated by said light source and storing at least a portion of said light image data associated with said point and said light source in a light accumulation buffer; and

then

combining at least a portion of said light accumulation buffer with said observer data; and

displaying resulting image data to a computer screen.

2. The method as recited in claim 1, wherein said observer data includes observed color data and observed depth data associated with a plurality of modeled polygons within said scene as rendered from an observer'"'"'s perspective.

3. The method as recited in claim 2, wherein said plurality of modeled polygons within said scene are associated with at least one pixel on said computer screen, such that said observed color data includes an observed red-green-blue value for said pixel and said observed depth data includes an observed z-buffer value for said pixel.

4. The method as recited in claim 2, wherein said lighting data includes source color data associated with at least one of said light sources and source depth data associated with said plurality of modeled polygons within said scene as rendered from a plurality of different light source'"'"'s perspectives.

5. The method as recited in claim 4, wherein said plurality of modeled polygons within said scene are associated with at least one pixel on said computer screen, such that said source color data includes a source red-green-blue value for said pixel and said source depth data includes a source z-buffer value for said pixel.

6. The method as recited in claim 4, wherein the step of comparing at least a portion of said observer data with at least a portion of said lighting data to determine if a modeled point within said scene is illuminated by said light source further includes comparing at least a portion of said observed depth data with at least a portion of said source depth data to determine if said modeled point is illuminated by said light source.

7. The method as recited in claim 6, wherein the step of comparing at least a portion of said observed depth data with at least a portion of said source depth data to determine if said modeled point is illuminated by said light source further includes converting at least a portion of said observed depth data from said observer'"'"'s perspective to at least one of said plurality of different light source'"'"'s perspectives, before comparing said observed depth data with said source depth data.

8. The method as recited in claim 7, wherein the step of converting at least a portion of said observed depth data from said observer'"'"'s perspective to at least one of said plurality of different light source'"'"'s perspectives further includes using a precalculated matrix transformation look-up table for at least one of said plurality of light sources, when said light source has a fixed perspective of said scene.

9. The method as recited in claim 4, wherein at least a portion of said source color data is selectively controlled source color data that can be changed over a period of time during which at least the step of displaying resulting image data to said computer screen is repeated a plurality of times.

10. The method as recited in claim 9, wherein said controlled source color data includes data selected from a set comprising motion picture data, video data, animation data, and computer graphics data.

11. An arrangement configured to render shadows in a simulated multi-dimensional scene, the arrangement comprising:

a display screen configured to display image data;

memory for storing data including observer data associated with a simulated multi-dimensional scene, and lighting data associated with a plurality of simulated light sources arranged to illuminate said scene, said lighting data including light image data, said memory further including a light accumulation buffer portion and a frame buffer portion;

at least one processor coupled to said memory and said display screen and operatively configured to, for each of said plurality of light sources, compare at least a portion of said observer data with at least a portion of said lighting data to determine if a modeled point within said scene is illuminated by said light source and storing at least a portion of said light image data associated with said point and said light source in said light accumulation buffer, then combining at least a portion of said light accumulation buffer with said observer data, and displaying resulting image data in said frame buffer, and outputting at least a portion of said image data in said frame buffer to said display screen.

12. The arrangement as recited in claim 11, wherein said observer data includes observed color data and observed depth data associated with a plurality of modeled polygons within said scene as rendered from an observer'"'"'s perspective.

13. The arrangement as recited in claim 12, wherein said plurality of modeled polygons within said scene are associated with at least one pixel on said display screen, such that said observed color data includes an observed red-green-blue value for said pixel and said observed depth data includes a observed z-buffer value for said pixel.

14. The arrangement as recited in claim 12, wherein said lighting data includes source color data associated with at least one of said light sources and source depth data associated with said plurality of modeled polygons within said scene as rendered from a plurality of different light source'"'"'s perspectives.

15. The arrangement as recited in claim 14, wherein said plurality of modeled polygons within said scene are associated with at least one pixel on said display screen, such that said source color data includes a source red-green-blue value for said pixel and said source depth data includes a source zbuffer value for said pixel.

16. The arrangement as recited in claim 14, wherein said processor is further configured to compare at least a portion of said observed depth data with at least a portion of said source depth data to determine if said modeled point is illuminated by said light source.

17. The arrangement as recited in claim 16, wherein said processor is further configured to convert at least a portion of said observed depth data from said observer'"'"'s perspective to at least one of said plurality of different light source'"'"'s perspectives, before comparing said observed depth data with said source depth data.

18. The arrangement as recited in claim 17, wherein said memory further includes at least one precalculated matrix transformation table associated with at least one of said plurality of light sources, and said processor is further configured to use said precalculated matrix transformation look-up table when said light source is simulated as having a fixed perspective of said scene.

19. The arrangement as recited in claim 14, wherein said processor is further configured to selectively control at least a portion of said source color data over a period of time.

20. The arrangement as recited in claim 19, wherein said controlled source color data includes data selected from a set comprising motion picture data, video data, animation data, and computer graphics data.

21. A method for simulating light falling on a modeled object in a computer generated multi-dimensional graphics simulation, the method comprising the steps of:

for a simulated camera, rendering a camera view of at least one modeled object that is at least partially optically opaque, to produce a camera depth array comprising camera depth data values and a corresponding camera image array comprising camera image data values;

for a first simulated light, rendering a first light view of said modeled object to produce a first light depth array comprising first light depth data values and a corresponding first light image array comprising first light image data values;

transforming at least a portion of said camera depth data values to said first light view, thereby generating a first transformed camera array comprising first transformed camera depth data values;

for each data value therein, comparing said first light depth array to said first transformed camera array to determine if said data value in said first light depth array is closer to said first simulated light, and if so, adding a corresponding data value from said first light image array to a light accumulation array comprising light accumulation data values; and

for each data value therein, multiplying said camera image array by a corresponding data value from said light accumulation array to produce a lighted camera image array comprising lighted camera image values.

22. The method as recited in claim 21, wherein, prior to the step of multiplying said camera image array by a corresponding data value from said light accumulation array to produce a lighted camera image array, the method further includes the steps of:

for a second simulated light, rendering a second light view of said modeled object to produce a second light depth array comprising second light depth data values and a corresponding second light image array comprising second light image data values;

transforming at least a portion of said camera depth data values to said second light view, thereby generating a second transformed camera array comprising second transformed camera depth data values; and

for each data value therein, comparing said second light depth array to said second transformed camera array to determine if said data value in said second light depth array is closer to said second simulated light, and if so, adding a corresponding data value from said second light image array to said light accumulation array.

23. The method as recited in claim 21, wherein each of said camera depth values includes z-buffer data associated with a different pixel selected from a plurality of pixels on a computer display screen.

24. The method as recited in claim 23, wherein each of said first light depth values includes z-buffer data associated with a different pixel selected from a plurality of pixels on a computer display screen.

25. The method as recited in claim 21, wherein each of said camera depth values includes z-buffer data associated with a different set of pixels selected from a plurality of pixels on a computer display screen.

26. The method as recited in claim 25, wherein each of said first light depth values includes z-buffer data associated with a different set of pixels selected from a plurality of pixels on a computer display screen.

27. The method as recited in claim 21, wherein said camera image data and said first light image data each include color data associated with at least one pixel on a computer screen.

28. The method as recited in claim 21, further comprising the steps of:

repeating the steps recited in claim 21 at a frame rate; and

sequentially displaying a plurality of frames of data on a computer screen at said frame rate, wherein subsequent frames of data include subsequently processed lighted camera image data, and wherein said step of rendering said first light view further comprises dynamically changing at least one of said first light image data values between said subsequent frames of data.

29. The method as recited in claim 28 wherein at least a portion of said first light image data values represent dynamically changing color data selected from a set comprising motion picture data, video data, animation data, and computer graphics data.

30. The method as recited in claim 28, wherein said frame rate is at least about 25 frames per second.

31. The method as recited in claim 21, wherein the step of transforming at least a portion of said camera depth data values to said first light view further includes the step of transforming said camera depth array from a camera coordinate system to a corresponding first light coordinate system.

32. The method as recited in claim 31, wherein the step of transforming said camera depth array from a camera coordinate system to a corresponding first light coordinate system further includes the step of using a precalculated transformation table to transform directly from said camera coordinate system to said corresponding first light coordinate system.

33. A computer-readable medium carrying at least one set of computer instructions configured to cause a computer to operatively simulate light falling on a modeled object in a computer generated multi-dimensional graphics simulation by performing operations comprising:

a) rendering an observer view of at least a portion of a spatially modeled object as a plurality of observed depth values and observed image values;

b) rendering a source view of at least a portion of said modeled object as a plurality of source depth values and a plurality of source image values;

c) transforming at least a portion of said observed depth values to said source view;

d) modifying at least one image accumulation value with one of said observed image values if said corresponding transformed observer value is equal to a comparable one of said source depth values;

e) multiplying said one of said observed image values by said at least one image accumulation value to produce at least one pixel value; and

f) displaying said pixel value on a computer screen.

34. The computer-readable medium as recited in claim 33, further configured to cause the computer to perform the further step of:

g) following step d), repeating steps b) through d) for at least one additional source view.

35. The computer-readable medium as recited in claim 34, further configured to cause the computer to perform the further steps of:

h) repeating steps a) through g) a frame rate; and

wherein step f) further includes sequentially displaying a plurality of pixels as frames of data on said computer screen at said frame rate, and said step of rendering said source view further includes changing at least one of said source image values between said subsequent frames of data.

36. The computer-readable medium as recited in claim 35 wherein at least a portion of said source image values represent color data selected from a set comprising motion picture data, video data, animation data, and computer graphics data.

37. The computer-readable medium as recited in claim 35, wherein step c) further includes transforming at least a portion of said observed depth values from an observer coordinate system to a corresponding source coordinate system.

38. The computer-readable medium as recited in claim 37, wherein the step of transforming at least a portion of said observed depth values from an observer coordinate system to a corresponding source coordinate system further includes using a precalculated transformation table to transform directly from said observer coordinate system to said corresponding source coordinate system.

39. A computer-readable medium carrying at least one set of computer instructions configured to cause at least one processor within a computer system to operatively render simulated shadows in a multi-dimensional simulated scene by performing the steps of:

providing observer data of a simulated multi-dimensional scene;

providing lighting data associated with a plurality of simulated light sources arranged to illuminate said scene, said lighting data including light image data;

for each of said plurality of light sources, comparing at least a portion of said observer data with at least a portion of said lighting data to determine if a modeled point within said scene is illuminated by said light source and storing at least a portion of said light image data associated with said point and said light source in a light accumulation buffer; and

then

combining at least a portion of said light accumulation buffer with said observer data; and

displaying resulting image data to a computer screen.

40. The computer-readable medium as recited in claim 39, wherein said observer data includes observed color data and observed depth data associated with a plurality of modeled polygons within said scene as rendered from an observer'"'"'s perspective.

41. The computer-readable medium as recited in claim 40, wherein said plurality of modeled polygons within said scene are associated with at least one pixel on said computer screen, such that said observed color data includes an observed red-green-blue value for said pixel and said observed depth data includes a observed z-buffer value for said pixel.

42. The computer-readable medium as recited in claim 40, wherein said lighting data includes source color data associated with at least one of said light sources and source depth data associated with said plurality of modeled polygons within said scene as rendered from a plurality of different light source'"'"'s perspectives.

43. The computer-readable medium as recited in claim 42, wherein said plurality of modeled polygons within said scene are associated with at least one pixel on said computer screen, such that said source color data includes a source red-green-blue value for said pixel and said source depth data includes a source z-buffer value for said pixel.

44. The computer-readable medium as recited in claim 42, wherein the step of comparing at least a portion of said observer data with at least a portion of said lighting data to determine if a modeled point within said scene is illuminated by said light source further includes comparing at least a portion of said observed depth data with at least a portion of said source depth data to determine if said modeled point is illuminated by said light source.

45. The computer-readable medium as recited in claim 44, wherein the step of comparing at least a portion of said observed depth data with at least a portion of said source depth data to determine if said modeled point is illuminated by said light source further includes converting at least a portion of said observed depth data from said observer'"'"'s perspective to at least one of said plurality of different light source'"'"'s perspectives, before comparing said observed depth data with said source depth data.

46. The computer-readable medium as recited in claim 45, wherein the step of converting at least a portion of said observed depth data from said observer'"'"'s perspective to at least one of said plurality of different light source'"'"'s perspectives further includes using a precalculated matrix transformation look-up table for at least one of said plurality of light sources, when said light source has a fixed perspective of said scene.

47. The computer-readable medium as recited in claim 42, wherein at least a portion of said source color data is selectively controlled source color data that can be changed over a period of time during which at least the step of displaying resulting image data to said computer screen is repeated a plurality of times.

48. The computer-readable medium as recited in claim 47, wherein said controlled source color data includes data selected from a set comprising motion picture data, video data, animation data, and computer graphics data.