GENERATING PHOTOREALISTIC SKY IN COMPUTER GENERATED ANIMATION

• US 20170287196A1
• Filed: 04/01/2016
• Published: 10/05/2017
• Est. Priority Date: 04/01/2016
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Raeburn, Gavin, Wood, James Alexander, Stephen, Scott Crawford, Janson, Kelvin Neil
• Application Number:
  US15/088,470
• Publication Number:
  US 20170287196A1
• Time in Patent Office: Days
• CPC Class Codes: A63F 13/60 Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/506 Illumination models
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

Realistic sky simulations are created in a computer generated graphics environment by incorporating captured image data of real sky over a time period, and converting these images into a streams of textures over time which can be sampled as a function of space and time within a game engine. The captured image data include data captured from a light probe indicating intensity and direction of light and a presumed direction of the sun. To capture such image data, an image capture rig comprising multiple cameras and a light probe is used. The image data captured from such cameras over a time period are processed to generate data used by an animation engine to produce photorealistic images of the sky.

Claims

1. A computer configured to generate computer animation in real time in response to user input, the computer comprising:

memory comprising a plurality of buffers;

a processing unit configured to access the plurality of buffers;

the processing unit further configured by a computer program to;

load the plurality of buffers with a sky texture and a diffuse cubemap, wherein the diffuse cubemap comprises a map of information representing ambient lighting for illuminating a scene, the sky texture and diffuse cubemap originating from samples of actual sky taken over a period of time;

receive scene data, a current time and a viewpoint;

render the scene data according to the viewpoint and the diffuse cubemap, such that objects in the scene data are illuminated based on at least the diffuse cubemap;

sample and interpolate the sky texture according to the current time; and

apply the rendered scene data as a foreground image to the interpolated sky texture as a background image.

2. The computer of claim 1, wherein the processing unit is further configured to:

apply a fog cubemap derived from the samples of actual sky to the rendered scene data and interpolated sky texture.

3. The computer of claim 1, wherein the processing unit is further configured to:

apply a cloud shadow map derived from the sample of actual sky to the scene data when rendering the scene data.

4. The computer of claim 3, wherein the processing unit is further configured to apply the cloud shadow map to the interpolated sky texture as a mask.

5. The computer of claim 1, wherein the sky texture comprises a sequence of high dynamic range images, each derived from a plurality of simultaneous exposures from a plurality of cameras sampled at a frame rate of a plurality of images.

6. The computer of claim 5, wherein the period of time is at least twenty-four hours.

7. The computer of claim 5, wherein the plurality of cameras comprises three cameras, each configured to capture a plurality of images for each frame at the frame rate.

8. An article of manufacture, comprising:

a computer storage medium comprising at least a memory or a storage device

computer program instructions stored on the computer storage medium that, when processed by a computer, configure the computer to;

load a plurality of buffers with a sky texture and a diffuse cubemap, wherein the diffuse cubemap comprises a map of information representing ambient lighting for illuminating a scene, the sky texture and diffuse cubemap originating from samples of actual sky taken over a period of time;

receive scene data, a current time and a viewpoint;

render the scene data according to the viewpoint and the diffuse cubemap;

sample and interpolate the sky texture according to the current time; and

apply the rendered scene data as a foreground image to the interpolated sky texture as a background image.

9. The article of manufacture of claim 8, wherein the computer is further configured to apply a fog cubemap derived from the samples of actual sky to the rendered scene data and interpolated sky texture.

10. The article of manufacture of claim 8, wherein the computer is further configured to apply a cloud shadow map derived from the sample of actual sky to the scene data when rendering the scene data.

11. The article of manufacture of claim 10, wherein the computer is further configured to apply the cloud shadow map to the interpolated sky texture as a mask.

12. The article of manufacture of claim 8, wherein the computer program instructions form a game engine, wherein the game engine further configures the computer to:

receive user inputs;

in response to user inputs, continually update game state including updated scene data according to game logic; and

the game engine providing the current time associated with the game state.

13. The article of manufacture of claim 8, wherein the sky texture comprises a sequence of high dynamic range images, each derived from a plurality of simultaneous exposures from a plurality of cameras sampled at a frame rate of a plurality of images.

14. The article of manufacture of claim 13, wherein the plurality of cameras comprises three cameras, each configured to capture a plurality of images for each frame at the frame rate.

15. A computer-implemented process, comprising:

loading a plurality of buffers with a sky texture and a diffuse cubemap, wherein the diffuse cubemap comprises a map of information representing ambient lighting for illuminating a scene, the sky texture and diffuse cubemap originating from samples of actual sky taken over a period of time;

receiving scene data, a current time and a viewpoint;

rendering the scene data according to the viewpoint and the diffuse cubemap, such that objects in the scene data are illuminated based on at least the diffuse cubemap;

sampling and interpolate the sky texture according to the current time; and

applying the rendered scene data as a foreground image to the interpolated sky texture as a background image.

16. The computer-implemented process of claim 15, further comprising applying a fog cubemap derived from the samples of actual sky to the rendered scene data and interpolated sky texture.

17. The computer-implemented process of claim 15, further comprising applying a cloud shadow map derived from the sample of actual sky to the scene data when rendering the scene data.

18. The computer-implemented process of claim 17, further comprising applying the cloud shadow map to the interpolated sky texture as a mask.

19. The computer-implemented process of claim 16, wherein the sky texture comprises a sequence of high dynamic range images, each derived from a plurality of simultaneous exposures from a plurality of cameras sampled at a frame rate of a plurality of images.

20. The computer-implemented process of claim 16, wherein the period of time is at least twenty-four hours.