Object Illumination in Hybrid Rasterization and Ray Traced 3-D Rendering

• US 20150262407A1
• Filed: 03/11/2015
• Published: 09/17/2015
• Est. Priority Date: 03/13/2014
• Status: Active Grant

• Current Assignee: Imagination Technologies Limited (Imagination Technologies Group Ltd.)
• Original Assignee: Imagination Technologies Limited (Imagination Technologies Group Ltd.)
• Inventors: Peterson, Luke T., Fursund, Jens
• Granted Patent: US 10,083,541 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 15/80 Shading

Abstract

Rendering systems that can use combinations of rasterization rendering processes and ray tracing rendering processes are disclosed. In some implementations, these systems perform a rasterization pass to identify visible surfaces of pixels in an image. Some implementations may begin shading processes for visible surfaces, before the geometry is entirely processed, in which rays are emitted. Rays can be culled at various points during processing, based on determining whether the surface from which the ray was emitted is still visible. Rendering systems may implement rendering effects as disclosed.

Claims

1. A 3-D rendering process, comprising:

determining, for each frame of a sequence of frames, visible surfaces for pixels in the frame, from a respective viewpoint;

shading the pixels, the shading for at least some of the pixels comprising sampling a texture, the texture mappable to a surface visible at one or more of the pixels; and

maintaining the texture during the determining of the visible surfaces and the shading by performing sampling operations for a selection of elements in the texture and performing a combining of existing data and results of the sampling.

2. The 3-D rendering process of claim 1, wherein the determining comprises rasterizing a set of geometry based on the respective viewpoint for the frame, and the maintaining comprises, concurrent with the rasterizing, performing a surface-specific lighting process comprising

emitting rays from distributed points on a surface of an object in the 3-D scene,

traversing the emitting rays to identify respective intersections between the rays and objects in the 3-D scene,

shading the intersections between the rays and respective objects that were intersected, the shading comprising determining an effect on one or more of the distributed points on the surface of the object and updating a non-transitory memory storing current lighting condition data for the distributed points on the surface of the object, wherein the rasterizing comprises accessing the current lighting condition data for one or more of the distributed points, in response to determining, during the rasterizing, that those one or more distributed points are visible in a frame of the sequence of frames.

3. The 3-D rendering process of claim 1, wherein the texture describes ambient lighting conditions on one or more objects to which that texture maps, and the maintaining comprises emitting rays for only a portion of the elements in the texture, during rasterizing of each frame in the sequence, and performing a weighted combination of results of the emitted rays with existing values for the samples to which the rays pertain, where rasterizing comprises sampling data from the texture for an object to which the texture maps at points on the object determined to be visible in a frame of the sequence of frames and using the sampled data to produced pixel shading results for pixels of that frame.

4. The 3-D rendering process of claim 1, wherein the maintaining of the texture comprises controlling usage of a processing engine to maintain a pre-determined set of light maps in a non-transitory memory, wherein different light maps of the set are associated with different objects in a 3-D scene, data in the light maps persisting in the non-transitory memory across multiple frames in the sequence of frames.

5. The 3-D rendering process of claim 1, wherein each element in the texture is identified by a parametric coordinate, and the updating comprises mapping the respective parametric coordinate for a selected element in the texture to a location on the object, and the sampling operations comprise tracing one or more rays from a location on the object, to identify respective intersections between the rays and other geometry located in the 3-D scene, shading identified intersections, and combining results of shading the identified intersections with an existing value of the selected data element.

6. The 3-D rendering process of claim 1, further comprising storing the texture in memory and providing read access to the texture in the memory, by a rasterization process, while the maintaining is being performed.

7. The 3-D rendering process of claim 1, further comprising allocating a budget for how much sampling can be performed for the texture during each of the frames, in order to meet a target frame rate at which the sequence of frames is to be displayed.

8. The 3-D rendering process of claim 1, wherein the weighted combining comprises either weighting the results more heavily, relative to the existing data, or invalidating the existing data, in response to a change in geometry in the 3-D scene.

9. The 3-D rendering process of claim 1, wherein the selection of the elements is made by prioritizing elements of the texture based on a likelihood that those elements map to locations on the surface that will be visible at one or more pixels in a frame of the sequence of frames.

10. The 3-D rendering process of claim 9, wherein the likelihood that elements map to locations on the surface that are visible is determined by comparing a respective normal of the surface at a location to which an element of the texture maps, with a normal of a visible surface stored in a buffer produced by the determining, for each frame of a sequence of frames, visible surfaces for each pixel of the frame, from the respective viewpoint of that frame.

11. The 3-D rendering process of claim 9, wherein the likelihood that elements map to locations on the surface that are visible is determined by tracing a ray from a location to which an element of the texture maps towards the respective viewpoint of that frame, and determining whether the ray hits any geometry in the 3-D scene.

12. The 3-D rendering process of claim 9, wherein the likelihood that elements map to locations on the surface that are visible is determined by calculating a depth from a location to which an element of the texture maps to the respective viewpoint of that frame, and comparing the calculated depth with a depth stored in a buffer produced by the determining, for each frame of a sequence of frames, visible surfaces for each pixel of the frame, from the respective viewpoint of that frame.

13. The 3-D rendering process of claim 1, further comprising making the selection of elements in multiple increments.

14. The 3-D rendering process of claim 13, wherein at least one of the increments, for a frame, precedes determining surfaces visible at pixels in that frame.

15. The 3-D rendering process of claim 1, wherein making the selection of elements comprises setting a sampling density for the texture, based on a level of detail indicator, generated during rasterization applicable to that surface.

16. The 3-D rendering process of claim 1, wherein making the selection of elements comprises selecting elements of the texture based on respective measured amount of variation in results of sampling conducted for elements of the texture.

17. The 3-D rendering process of claim 1, further comprising, during rasterization of a current frame in the sequence of frames, making a selection of elements for which sampling is to be conducted, based on a setup of the 3-D scene, for a frame subsequent to the current frame.

18. The 3-D rendering process of claim 17, wherein making the selection of elements comprises selecting elements of the texture based on inputs indicative of a change in viewpoint from an application for which the sequence of frames is being produced.

19. The 3-D rendering process of claim 1, wherein making the selection of elements comprises selecting a subset of the elements, at a baseline sampling rate, without regard to variation of sampling results for the elements of the texture, and selecting another subset of the elements based on characterizing variation of sampling results for the elements of the texture.

20. The 3-D rendering process of claim 1, wherein making the selection of elements comprises selecting elements of the texture based on inputs from a physics process conducted prior to geometry finalization for the 3-D scene to be used in rendering a particular frame in the sequence of frames.

21. The 3-D rendering process of claim 1, wherein the texture is a portion of a texture atlas, and other textures in the texture atlas are mappable to other objects in the 3-D scene.

22. The 3-D rendering process of claim 1, further comprising, during rasterization of a current frame in the sequence of frames, making a selection of elements for which sampling is to be conducted, based on a rate of change in variation of results from sampling for elements within different regions of the texture.

23. An apparatus that implements a 3-D graphics rendering pipeline, comprising:

a set of programmable computation units;

a rasterization subsystem configured for receiving a stream of geometry elements, for each frame in a sequence of frames, and determining a respective visible geometry element for each pixel in each frame in the sequence of frames, wherein the set of programmable computation units is configurable to execute rasterization shading code for the visible geometry elements, the executing rasterization shading code capable of outputting texture sample requests;

a texture sampling subsystem coupled to receive texture sample requests and to return texture data for each texture sample request based on current texture data stored in a memory, wherein a texture stored in the texture memory is mappable to a surface of a geometry element;

a ray tracing subsystem configured to traverse rays through an acceleration structure to identify respective portions of geometry elements to be tested for intersection with the rays, wherein the set of programmable computation units are further configurable to execute ray shading code for intersections determined between rays and geometry elements; and

a texture maintenance subsystem configured to maintain the texture by selecting elements of the texture, determining locations that map to those selected elements of the texture, emitting rays from those locations, receiving results pertaining to shading of these rays and updating elements in the texture concurrently with the execution of the rasterization subsystem.

24. The apparatus of claim 23, wherein the texture describes ambient lighting conditions on one or more objects to which that texture maps, and the texture maintenance subsystem is configured to emit rays for only a portion of the elements in the texture, during rasterization of each frame in the sequence of frames, and performing a weighted combination of results of the emitted rays with existing values for the samples to which the rays pertain, wherein rasterizing comprises sampling data from the texture for an object to which the texture maps at points on the object determined to be visible in a frame of the sequence of frames and using the sampled data to produced pixel shading results for pixels of that frame.

25. The apparatus of claim 23, wherein the texture maintenance subsystem comprises a controller configured to determine portions of the texture to be updated while each of the plurality of frames in the sequence are produced.

26. The apparatus of claim 24, wherein the controller is further configured to provide a weighted blending of the results with existing data in the texture, wherein the relative weights assigned to each of the results and the existing data is based on an estimate of a rate of change in values of the results.

27. The apparatus of claim 23, wherein the texture maintenance subsystem is implemented by executing a process on one or more processing elements, the texture pertains to ambient lighting conditions on an object located in the 3-D scene, and the emitted rays are from respective locations on the object determined by mapping a parametric coordinate of an element of the texture selected for sampling to the location on the object, shading an intersection identified for the ray to produce the results used to update the elements of the texture.