Memory management for systems for generating 3-dimensional computer images

• US 8,669,987 B2
• Filed: 02/15/2007
• Issued: 03/11/2014
• Est. Priority Date: 09/29/2006
• Status: Active Grant

• Current Assignee: Imagination Technologies Limited (Imagination Technologies Group Ltd.)
• Original Assignee: Imagination Technologies Limited (Imagination Technologies Group Ltd.)
• Inventors: Redshaw, Jonathan, Morphet, Steve
• Primary Examiner(s): Perromat, Carlos
• Application Number:
  US11/707,209
• Publication Number:
  US 20080186318A1
• Time in Patent Office: 2,581 Days
• Field of Search: 345/553
• US Class Current: 345/443
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation

Abstract

Memory management system and method for use with systems for generating 3-dimensional computer generated images are provided. The system includes a) a device for subdividing the image into rectangular areas, b) a memory for storing object data for each rectangular area and depth data derived from the object data, c) a device for storing the object data in the memory, d) a device for deriving the depth data for each rectangular area from the object data, e) a device for storing the depth data for each rectangular area in the memory, f) a device for loading further object data into the memory to replace part of the existing contents, g) a device for retrieving the stored depth data, h) a device for deriving updated depth data for each picture element of each rectangular area from the new object and stored depth data, and storing the updated depth data to replace the previously stored depth data, i) a device for causing features e), f), g) and h) to repeatedly perform the functions until there is no further object data to load to the 1 memory, and j) a device for deriving image data and shading data from the depth data for display.

Claims

1. A system for use in rendering an image from 3-D graphics data comprising:

a heap memory;

a manager configured for reserving z buffer space, in the heap memory, to store Z buffer data for at least one macrotile of an image to be rendered, wherein each macrotile is defined to include a respective subset of pixels in the image, and for allocating blocks from a remaining portion of the heap memory to display lists for respective macrotiles; and

a tiling accelerator comprising a macrotiling engine configured for inputting a stream of geometry elements, determining overlap between the inputted geometry elements and the macrotiles, and writing identifying information for the geometry elements to display lists according to the determined overlap, wherein data for each display list is written to a respective current memory block, allocated from the heap memory, to that display list,

wherein the system is configured to determine, before the stream of geometry elements has been completely processed by the macrotiling engine, that a selected display list, associated with a macrotile, is to begin image synthesis processing, resulting in depth information that is written into blocks of memory from the reserved z buffer space, and to reallocate the memory used for storing the selected display list first to reserved z buffer space, at least until there is reserved z-buffer space for at least one macrotile, and if any blocks of memory remain from the memory used for storing the selected display list, then to make those blocks available for on demand allocation by the manager for storing further outputs produced by the tiling accelerator in processing additional geometry elements from the stream of geometry elements.

2. The system for use in rendering an image from 3-D graphics data of claim 1, wherein the manager is configured to divide a portion of the heap memory, remaining after the allocation of reserved z buffer memory, among the macrotiles, so that the entire heap memory is assigned either to reserved z buffer memory or to a macrotile.

3. The system for use in rendering an image from 3-D graphics data of claim 2, wherein the manager is configured to divide the portion of the heap memory, remaining after the allocation of reserved z buffer memory, unequally among the macrotiles, and the system is configured to select the macrotile with the largest allocation of memory to begin image synthesis processing first.

4. The system for use in rendering an image from 3-D graphics data of claim 1, wherein the pixels corresponding to each macrotile are further divided into a respective set of tiles, and the tiling accelerator further is configured to allocate, responsive to demand, portions of a block to store control stream data produced by a tiling engine in the tiling accelerator, the control stream data referencing geometry object data written by the macrotiling engine.

5. The system for use in rendering an image from 3-D graphics data of claim 4, wherein the portions of the one or more blocks allocated to store control stream data allocated on demand to store tile-specific control stream data produced by the tiling engine each are 16 words, of a determined memory word size.

6. The system for use in rendering an image from 3-D graphics data of claim 1, wherein the manager is further configured to allocate a portion of the heap memory to a global display list used to store object data for objects overlapping multiple macrotiles.

7. The system for use in rendering an image from 3-D graphics data of claim 6, wherein the tiling accelerator further is configured to write object identifying information to the global display list for objects determined to overlap more than one macrotile.

8. The system for use in rendering an image from 3-D graphics data of claim 6, further comprising writing object identifying information to the global display list for objects determined to be within a single macrotle, but within a pre-determined distance from an edge of that macrotile.

9. The system for use in rendering an image from 3-D graphics data of claim 1, wherein the system is further configured to double buffer address information for a start of a display list for a macrotile and a current end of the display list for that macrotile, and to provide access to one set of the double buffered address information to each of the tiling accelerator and an image synthesis engine.

10. The system for use in rendering an image from 3-D graphics data of claim 1, wherein the system is further configured to begin image synthesis processing of the global display list by creating another context and continuing to process the stream of geometry into macrotile-specific display lists in the new context, while rendering from the global display list and existing macrotile display lists.

11. The system for use in rendering an image from 3-D graphics data of claim 1, wherein the system further is configured to support concurrent processing of geometry produced from a plurality of applications by allowing tiled geometry elements produced by any of the plurality of applications to be rendered into the reserved z buffer memory.

12. The system for use in rendering an image from 3-D graphics data of claim 1, wherein a total size of the heap memory is at least as large as a size required to store z data for all pixels of (n+1) macrotiles, wherein n is the number of macrotiles in the image.

13. The system for use in rendering an image from 3-D graphics data of claim 1, wherein the system is configured to determine to begin processing the selected display list in response to a measure of usage of the heap memory.

14. A method for rendering an image from 3-D graphics data comprising:

determining a pixel resolution of an image to be rendered;

defining a set of macrotiles, each macrotile encompassing a respective region of pixels in the image to be rendered;

allocating reserved z buffer memory, from heap memory, wherein an amount of memory allocated to the reserved z buffer memory is sized to store depth data for pixels in at least one macrotile;

allocating blocks from the heap memory, responsive to demand, to store geometry object-identifying information and control stream information generated by a tiling accelerator operating on an inputted stream of geometry objects, to create a respective display list for each macrotile overlapped by inputted geometry objects;

selecting a macrotile, responsive to determining that an amount of memory allocated from the heap memory exceeds a threshold, to begin image synthesis processing, which includes writing depth data to the reserved z buffer memory; and

reallocating blocks of memory that stored one or more of object identifying information and control stream information for the display list for the selected macrotile first to replace the reserved z buffer memory for use during image synthesis processing of another macrotile at least until there is reserved z-buffer memory for at least one macrotile, and if any blocks of memory remain from the memory used for storing the selected display list, thereafter to provide those blocks for on demand allocation for storing object-identifying information and control stream information produced by the tiling accelerator for additional geometry elements.

15. The method for rendering an image from 3-D graphics data of claim 14, wherein a portion of the heap memory, remaining after the allocation of reserved z buffer memory, is divided among the macrotiles, so that the entire heap memory is assigned either to reserved z buffer memory or to a macrotile.

16. The method for rendering an image from 3-D graphics data of claim 15, wherein the portion of the heap memory, remaining after the allocation of reserved z buffer memory, is divided unequally among the macrotiles, and the macrotile with the largest allocation of memory is selected to begin image synthesis processing first.

17. The method for rendering an image from 3-D graphics data of claim 14, wherein the pixels corresponding to each macrotile are further divided into a respective set of tiles, and further comprising allocating, responsive to demand, portions of a block to store control stream data produced by a tiling engine, the control stream data referencing geometry object data written by a macrotiling engine.

18. The method for rendering an image from 3-D graphics data of claim 14, further comprising allocating portions of the one or more blocks allocated to a given macrotile to store control stream data to store tile-specific control stream data produced for tiles within that macrotile.

19. The method for rendering an image from 3-D graphics data of claim 18, wherein each block is 4KB in size, and the portions of the one or more blocks allocated to store control stream data allocated on demand to store tile-specific control stream data produced by the tiling engine each is 16 words, of a determined memory word size.

20. The method for rendering an image from 3-D graphics data of claim 14, further comprising allocating a portion of the heap memory to a global display list used to store object data for objects overlapping multiple macrotiles.

21. The method for rendering an image from 3-D graphics data of claim 14, further comprising writing object identifying information to the global display list for objects determined to overlap more than one macrotile.

22. The method for rendering an image from 3-D graphics data of claim 14, further comprising writing object identifying information to the global display list for objects determined to be within a macrotle, but within a pre-determined distance from an edge of that macrotile.

23. The method for rendering an image from 3-D graphics data of claim 14, further comprising double buffering address information for a start of a display list for a macrotile and a current end of the display list for that macrotile, and providing access to one set of the double buffered address information to each of a tiling accelerator and an image synthesis engine.

24. The method for rendering an image from 3-D graphics data of claim 14, further comprising, before the stream of geometry objects has been completely inputted, to begin image synthesis processing of the global display list by creating another context and continuing to process the stream of geometry into macrotile-specific display lists in the new context, while rendering from the global display list and existing macrotile display lists.

25. The method for rendering an image from 3-D graphics data of claim 14, further comprising supporting concurrent processing of geometry produced from a plurality of applications by allowing tiled geometry elements produced by any of the plurality of applications to be rendered into the reserved z buffer memory.

26. The method for rendering an image from 3-D graphics data of claim 14, wherein a total size of the heap memory is at least as large as a size required to store z data for all pixels of (n+1) macrotiles, wherein n is the number of macrotiles in the image.