System and method for generating and playback of three-dimensional movies

• US 6,429,867 B1
• Filed: 03/15/1999
• Issued: 08/06/2002
• Est. Priority Date: 03/15/1999
• Status: Expired due to Term

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Deering, Michael F.
• Primary Examiner(s): Vo, Cliff N.
• Application Number:
  US09/268,397
• Time in Patent Office: 1,240 Days
• Field of Search: 345/418, 345/44, 345/420, 345/421, 345/422, 345/423, 345/424, 345/425
• US Class Current: 345/423
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 13/289 Switching between monoscopic and stereoscopic modes
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/334 using spectral multiplexing
• CPC Class Codes: H04N 13/337 using polarisation multiplexing
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding

Abstract

A system and method for generating and playing back three-dimensional (3D) movies are disclosed. The system is capable of partially rendering frames without relying upon exact viewpoint information. The partially rendered frames may be rendered to the extent possible without performing viewpoint-dependent processes, and then compressed and stored to a carrier medium. To reduce the amount of data to be stored, the viewer'"'"'s possible viewpoints may be restricted (e.g., by defining a viewpoint-limiting volume or region). The resulting partially-rendered geometry data may be compressed using geometry compression. During playback, the compressed frames are read as a stream, and decompressed. Any final viewpoint-dependent rendering operations may then be performed (e.g., some lighting calculations and atmospheric effects, some fogging, specular highlighting, and reflections). A sensor such as a head-tracker may provide real-time viewpoint information that may be used by the playback system. After rendering, the frames are rasterized and then displayed in stereo.

Claims

1. A method for generating three-dimensional movies comprising:

receiving three-dimensional graphics data representing a scene;

specifying viewpoint limitations for said scene;

partially rendering, in a primarily viewpoint-independent manner, one or more frames representing said scene, wherein said partially rendering further comprises tessellating said three-dimensional graphics data into micropolygons to meet one or more predetermined criteria;

compressing said one or more partially rendered frames; and

outputting said compressed frames to a carrier medium.

2. The method as recited in claim 1, wherein said one or more predetermined criteria comprise smoothness and predetermined size function in micropolygon size in world space.

3. The method as recited in claim 1, wherein said one or more predetermined criteria comprise an estimated worst-case micropolygon size in screen space.

4. The method as recited in claim 3, wherein said estimated worst-case micropolygon size in screen spaceis estimated using the distance from a closet permitted veiwpoint to a given micopolygon.

5. The method as recited in claim 3, said estimated worst-case micropolygon size in screen space is estimated using the combination of a worst-case distance from a closest permitted viewpoint to a given micropolygon and a worst-case viewing angle from said viewpoint to a given micropolygon.

6. The method as recited in claim 1, wherein said partially rendering comprises selecting micropolygons visible from viewpoints complying with said viewpoint limitations.

7. The method as recited in claim 1, wherein said partially rendering comprises performing a subset of lighting calculations for said micropolygons, wherein said subset of lighting calculations are mostly viewpoint-independent.

8. The method as recited in claim 7, wherein said subset of lighting calculations includes viewpoint-independent shadowing.

9. The method as recited in claim 7, wherein said subset of lighting calculations is selected from the group comprising:

viewpoint-independent reflections, viewpoint-independent depth of field effects, viewpoint-independent bump mapping, and viewpoint-independent displacement mapping.

10. The method as recited in claim 1, wherein said specifying viewpoint limitations comprises specifying a viewpoint-limiting volume.

11. The method as recited in claim 10, wherein said viewpoint-limiting volume has a depth of zero and is a viewpoint-limiting surface.

12. The method as recited in claim 1, wherein said specifying viewpoint limitations comprises specifying a set of valid viewpoints and a set of valid viewpoint orientations.

13. The method as recited in claim 1, wherein said graphics data represents one or more three-dimensional graphics objects, and wherein said method further comprises tessellating said graphics data into a plurality of micropolygons in world space.

14. The method as recited in claim 1, wherein said partially rendering further comprises applying one or more lighting effects to said micropolygons, wherein said one or more lighting effects are viewpoint-independent.

15. The method as recited in claim 1, wherein said carrier medium is selected from the group comprising:

transmission media and computer-readable media.

16. The method as recited in claim 1, wherein said compressed frames are configured to be read in a stream and rasterized in real-time using user-controlled viewpoint information.

17. The method as recited in claim 1, wherein said compressed frames are configured to be rasterized in real-time by applying viewpoint-dependent lighting effects.

18. The method as recited in claim 1, wherein each compressed frame comprises separate data.

19. The method as recited in claim 1, further comprising outputting said viewpoint limitations to said carrier medium.

20. The method as recited in claim 1, further comprising indicating when one or more portions of a particular frame may be reused in a subsequent frame.

21. The method as recited in claim 1, wherein only key frames are stored and intermediate frames are interpolated therefrom.

22. A method for playing back three-dimensional movies comprising:

reading a plurality of compressed frames from a carrier medium;

decompressing said compressed frames into a plurality of world-space micropolygons;

inputting viewer-controlled viewpoint information;

translating said micropolygons to screen-space;

applying one or more viewpoint-dependent lighting effects to said micropolygons;

rasterizing said micropolygons to generate a frame; and

displaying said frame on a display device.

23. The method as recited in claim 22, further comprising applying viewpoint-dependent atmospheric effects to said micropolygons.

24. The method as recited in claim 22, further comprising applying viewpoint-dependent texture effects to said micropolygons.

25. The method as recited in claim 22, further comprising applying viewpoint-dependent reflection effects to said micropolygons.

26. The method as recited in claim 22, further comprising comparing said viewer-controlled viewpoint information with viewpoint limitations read from said carrier medium.

27. The method as recited in claim 22, further comprising forcing said viewer-controlled viewpoint information to comply with viewpoint limitations read from said carrier medium.

28. The method as recited in claim 22, wherein said translating, rasterizing, and displaying are performed in real-time.

29. The method as recited in claim 22, wherein each decompressed frame is translated to screen space, rasterized, and displayed in visual stereo.

30. The method as recited in claim 22, further comprising including:

reading a stream of compressed audio from said carrier medium;

decompressing said stream of compressed audio;

processing said decompressed audio to reflect said real-time viewer-controlled ear location information; and

playing back said decompressed and processed audio.

31. A software program embodied on a first carrier medium, wherein said software program is configured to generate three-dimensional movies, wherein said software program comprises a plurality of instructions, wherein said plurality of instructions are configured to:

receive three-dimensional graphics data representing a scene;

specify viewpoint limitations for said scene based on viewer input;

partially render one or more frames representing said scene;

compress said one or more partially-rendered frames; and

output said compressed frames to a second carrier medium, wherein said first carrier medium and said second carrier medium are selected from the group consisting of computer-readable media and transmission media.

32. A software program embodied on a first carrier medium, wherein said software program is configured to playback three-dimensional movies, wherein said software program comprises a plurality of instructions, wherein said plurality of instructions are configured to:

read a plurality of compressed frames from a second carrier medium, wherein said first carrier medium and said second carrier medium are selected from the group consisting of computer-readable media and transmission media;

decompress said compressed frames into a plurality of world-space micropolygons;

input viewer-controlled viewpoint information;

translate said micropolygons to screen-space;

apply one or more viewpoint-dependent lighting effects to said micropolygons;

rasterize said micropolygons to generate a frame; and

display said frame on a display device.

33. A graphics system configured to playback previously generated three-dimensional movies, said graphics system comprising:

an input means for reading a stream of compressed frames from a carrier medium, wherein said input means is further configured to receive viewpoint limitation information with said compressed frames;

a decompressor coupled to receive and decompress said compressed frames into a plurality of partially-rendered world-space micropolygons;

a graphics processor coupled to receive said partially-rendered world-space micropolygons; and

a sensor coupled to provide said graphics processor with viewer viewpoint information, wherein said graphics processor is configured to translate said world-space micropolygons to screen-space micropolygons using said viewer viewpoint information, and wherein said graphics processor is configured to rasterize said screen-space micropolygons to generate one or more 3D frames in stereo.

34. The system as recited in claim 33, wherein said graphics processor is configured to force said viewer viewpoint information to comply with said viewpoint limitation information.

35. The system as recited in claim 33, further comprising a display device and a sample buffer, wherein said sample buffer is coupled between said graphics processor and said display device, wherein said sample buffer is super-sampled relative to the number of pixels displayable by said display device.

36. The system as recited in claim 33, wherein said graphics processor is further configured to perform a subset of lighting calculations on said partially-rendered world-space micropolygons.

37. The system as recited in claim 36, wherein said subset of lighting calculations includes all viewpoint-dependent lighting calculations.

38. The system as recited in claim 36, wherein said viewpoint-dependent calculations are selected from the group comprising:

viewpoint-dependent reflections and viewpoint-dependent depth of field effects.

39. A method for generating three-dimensional movies comprising:

receiving three-dimensional graphics data;

specifying viewpoint limitations for said three-dimensional graphics data;

tessellating said three-dimensional graphics data into micropolygons;

partially rendering selected micropolygons, wherein said selected micropolygons are visible from viewpoints complying with said viewpoint limitations;

compressing the partially rendered selected micropolygons; and

outputting the compressed partially rendered selected micropolygons to a carrier medium.

40. The method as recited in claim 39, wherein said partially rendering comprises performing a subset of lighting calculations for said selected micropolygons, wherein said subset of lighting calculations are mostly viewpoint-independent.

41. The method as recited in claim 40, wherein said subset of lighting calculations is selected from the group comprising:

viewpoint-independent reflections, viewpoint-independent depth of field effects, viewpoint-independent bump mapping, viewpoint-independent displacement mapping, and viewpoint-independent shadowing.

42. A method for generating three-dimensional movies comprising:

receiving three-dimensional graphics data;

specifying viewpoint limitations for said three-dimensional graphics data;

tessellating said three-dimensional graphics data into micropolygons;

partially rendering said micropolygons, wherein said partially rendering comprises performing a subset of lighting calculations for said micropolygons, wherein said subset of lighting calculations are mostly independent of said viewpoint limitations;

compressing said partially-rendered micropolygons; and

outputting said compressed partially-rendered micropolygons to a carrier medium.

43. The method as recited in claim 42, wherein said subset of lighting calculations is selected from the group comprising:

viewpoint-independent reflections, viewpoint-independent depth of field effects, viewpoint-independent bump mapping, viewpoint-independent displacement mapping, and viewpoint-independent shadowing.

44. The method as recited in claim 42, wherein said outputting further comprises outputting said viewpoint limitations to a carrier medium.

45. A method for generating three-dimensional movies comprising:

receiving three-dimensional graphics data representing a scene;

specifying viewpoint limitations for said scene;

partially rendering, in a manner independent of said viewpoint limitations, one or more frames representing said scene, wherein said partially rendering further comprises tessellating said three-dimensional graphics data into micropolygons, wherein said partially rendering comprises performing a subset of lighting calculations for said micropolygons, and wherein said subset of lighting calculations are mostly independent of said viewpoint limitations;

compressing said one or more partially-rendered frames; and

outputting said compressed frames to a carrier medium.

46. A graphics system configured to playback previously generated three-dimensional movies, said graphics system comprising:

an input means for reading a stream of compressed frames from a carrier medium, wherein said input means is further configured to receive viewpoint limitation information with said compressed frames;

a decompressor coupled to receive and decompress said compressed frames into a plurality of partially rendered world-space micropolygons;

a graphics processor coupled to receive said partially-rendered world-space micropolygons; and

a sensor coupled to provide said graphics processor with viewer viewpoint information, wherein said graphics processor is configured to translate said world-space micropolygons to screen-space micropolygons using said viewer viewpoint information and to force said viewer viewpoint information to comply with said viewpoint limitation information, and wherein said graphics processor is configured to rasterize said screen-space micropolygons to generate one or more 3D frames in stereo.

47. A software program embodied on a first carrier medium, wherein said software program is configured to generate three-dimensional movies, wherein said software program comprises a plurality of instructions, wherein said plurality of instructions are configured to:

receive three-dimensional graphics data;

specify viewpoint limitations for said three-dimensional graphics data based on viewer input;

tessellate said three-dimensional graphics data into micropolygons;

partially render said micropolygons, wherein said partially render comprises performing a subset of lighting calculations for said micropolygons, wherein said subset of lighting calculations are mostly independent of said viewpoint limitations;

compress said partially-rendered micropolygons; and

output said compressed partially-rendered micropolygons to a second carrier medium, wherein said first carrier medium and said second carrier medium are selected from the group comprising computer-readable media and transmission media.

48. A software program embodied on a first carrier medium, wherein said software program is configured to playback three-dimensional movies, wherein said software program comprises a plurality of instructions, wherein said plurality of instructions are configured to:

read compressed partially-rendered three-dimensional graphics data from a second carrier medium, wherein said first carrier medium and said second carrier medium are selected from the group comprising computer-readable media and transmission media;

decompress said compressed partially-rendered three-dimensional graphics data into a plurality of world-space micropolygons;

receive viewer-controlled viewpoint information;

partially render said micropolygons, wherein said partially render comprises performing a subset of lighting calculations for said micropolygons, wherein said subset of lighting calculations are mostly dependent on said viewer-controlled viewpoint information;

rasterize said micropolygons to generate a frame; and

display said frame on a display device.

49. A method for playing back three-dimensional movies comprising:

reading compressed partially-rendered three-dimensional graphics data and viewpoint limitation information from a carrier medium;

decompressing said compressed partially-tendered three-dimensional graphics into a plurality of world-space micropolygons;

inputting viewer-controlled viewpoint information, wherein said viewer-controlled viewpoint information is forced to comply with said viewpoint limitation information;

translating said micropolygons to screen-space;

applying one or more viewpoint-dependent lighting effects to said micropolygons;

rasterizing said micropolygons to generate a frame; and

displaying said frame on a display device.