METHOD, APPARATUS AND SYSTEM FOR RENDERING A GRAPHICAL REPRESENTATION WITHIN LIMITED MEMORY

• US 20180181846A1
• Filed: 12/15/2017
• Published: 06/28/2018
• Est. Priority Date: 12/22/2016
• Status: Active Grant

• Current Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Original Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Inventors: KOZIARZ, KRZYSZTOF ADAM, WADE, CHRISTOPHER KEITH DOWELL, DUDINA, TATIANA
• Granted Patent: US 10,198,674 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/1206 resulting in increased flexibility in input data format or job format or job type
• CPC Class Codes: G06F 3/122 with regard to computing resources, e.g. memory, CPU
• CPC Class Codes: G06F 3/1224 Client or server resources management
• CPC Class Codes: G06F 3/1248 by printer language recognition, e.g. PDL, PCL, PDF
• CPC Class Codes: G06K 15/1813 Page description language recognition
• CPC Class Codes: G06K 15/1817 Buffers
• CPC Class Codes: G06K 15/1849 using an intermediate representation, e.g. a list of graphical primitives
• CPC Class Codes: G06K 15/1886 Storage of the print image data or of parts thereof
• CPC Class Codes: G06K 15/401 Sharing memory space

Abstract

A method of controlling memory usage in a graphics rendering system. The method comprises converting a plurality of layers of graphical objects to an intermediate format representation, the layers being formed by grouping the graphical objects into a plurality of layers based on a first complexity threshold; and in response to detecting a memory shortage condition in execution of the graphics rendering system, determining a second complexity threshold based on the detected memory shortage condition, the second complexity threshold being lower than the first complexity threshold. The method also comprises identifying a layer of the plurality of layers based on the second complexity threshold, the identified layer being different to a layer which triggered the memory shortage condition, and converting the identified layer of graphical objects to the intermediate format representation to release memory occupied by graphical objects of the identified layer.

Claims

1. A method of controlling memory usage in a graphics rendering system, the method comprising:

converting a plurality of layers of graphical objects to an intermediate format representation, the layers being formed by grouping the graphical objects into a plurality of layers based on a first complexity threshold;

in response to detecting a memory shortage condition in execution of the graphics rendering system, determining a second complexity threshold based on the detected memory shortage condition, the second complexity threshold being lower than the first complexity threshold;

identifying a layer of the plurality of layers based on the second complexity threshold, the identified layer being different to a layer which triggered the memory shortage condition; and

converting the identified layer of graphical objects to the intermediate format representation to release memory occupied by graphical objects of the identified layer.

2. The method according to claim 1, wherein the detected memory shortage condition relates to memory for interpreting a page display language of the graphical objects.

3. The method according to claim 1, wherein the memory shortage condition relates to memory for converting the plurality of layers of graphical objects to the intermediate format representation.

4. The method according to claim 1, wherein identifying the layer comprises selecting a y-band of a page containing the graphical objects based on the second complexity threshold.

5. The method according to claim 1, wherein identifying the layer comprises splitting a z-layer associated with the plurality of layers based on the second complexity threshold.

6. The method according to claim 1, further comprising selecting another layer of the plurality of layers based on the second complexity threshold and converting the selected layer to the intermediate format representation.

7. The method according to claim 1, wherein the memory shortage condition is detected by comparing currently reserved memory with a predetermined threshold.

8. The method according to claim 1, wherein the memory shortage condition is detected when a memory allocation request fails.

9. The method according to claim 1, further comprising determining a state of execution of the graphics rendering system by selecting a state from:

a first state corresponding to the memory shortage; and

a second state corresponding to a state when the system recovered from the memory shortage condition.

10. The method according to claim 1, further comprising:

if converting the identified layer fails to complete within a fixed memory, reclaiming memory from at least one of a print description language interpreter module and a display list generator module.

11. The method according to claim 1, wherein control of the memory is completed in stages according to a severity level of the graphics rendering system.

12. The method according to claim 1, further comprising rendering and compressing pixel data associated with the identified layer if conversion of the identified layer to the intermediate format representation fails and a predetermined severity level is detected.

13. The method according to claim 12, further comprising reducing quality of a bitmap of the graphical objects if a predetermined severity level is detected.

14. A method of controlling memory usage in a graphics rendering system processing graphical objects in independent spatial portions, the method comprising:

converting a plurality of layers of graphical objects to an intermediate format representation, the layers being formed by grouping the graphical objects for each of a plurality of spatial portions into layers based on a first complexity threshold chosen to facilitate throughput of the graphics rendering system;

in response to detecting a memory shortage condition in execution of the graphics rendering system processing graphical objects of a first of the plurality of spatial portions, determining a second layer complexity threshold to address the memory shortage condition, the second layer complexity threshold being lower than the first complexity threshold;

identifying a layer of the graphical objects associated with a second of the plurality of spatial portions using the plurality of layers based on the second layer complexity threshold, the second spatial portion being different to the first spatial portion which triggered the memory shortage condition; and

converting the identified layer of graphical objects to the intermediate format representation to release memory occupied by graphical objects of the identified layer.

15. The method according to claim 14, further comprising:

in response to successful conversion of the identified layer, determining if the released memory satisfies a memory allocation request, and

converting a layer associated with the first spatial portion to the intermediate format if the released memory satisfies the memory allocation request.

16. A non-transitory computer readable medium having a program stored thereon for controlling memory usage in a graphics rendering system, the program comprising:

code for converting a plurality of layers of graphical objects to an intermediate format representation, the layers being formed by grouping the graphical objects into a plurality of layers based on a first complexity threshold;

code for, in response to detecting a memory shortage condition in execution of the graphics rendering system, determining a second complexity threshold based on the detected memory shortage condition, the second complexity threshold being lower than the first complexity threshold;

code for identifying a layer of the plurality of layers based on the second complexity threshold, the identified layer being different to a layer which triggered the memory shortage condition; and

code for converting the identified layer of graphical objects to the intermediate format representation to release memory occupied by graphical objects of the identified layer.

17. Apparatus for controlling memory usage in a graphics rendering system, the apparatus configured to:

convert a plurality of layers of graphical objects to an intermediate format representation, the layers being formed by grouping the graphical objects into a plurality of layers based on a first complexity threshold;

in response to detecting a memory shortage condition in execution of the graphics rendering system, determine a second complexity threshold based on the detected memory shortage condition, the second complexity threshold being lower than the first complexity threshold;

identify a layer of the plurality of layers based on the second complexity threshold, the identified layer being different to a layer which triggered the memory shortage condition; and

convert the identified layer of graphical objects to the intermediate format representation to release memory occupied by graphical objects of the identified layer.

18. A graphic rendering system comprising:

a memory for storing data and a computer readable medium; and

a processor coupled to the memory for executing a computer program, the program having instructions for;

converting a plurality of layers of graphical objects to an intermediate format representation, the layers being formed by grouping the graphical objects into a plurality of layers based on a first complexity threshold;

in response to detecting a memory shortage condition in execution of the graphics rendering system, determining a second complexity threshold based on the detected memory shortage condition, the second complexity threshold being lower than the first complexity threshold;

identifying a layer of the plurality of layers based on the second complexity threshold, the identified layer being different to a layer which triggered the memory shortage condition; and

converting the identified layer of graphical objects to the intermediate format representation to release memory occupied by graphical objects of the identified layer.

19. A method of rendering a graphical representation, the method comprising:

receiving a plurality of objects of the graphical representation grouped into layers in accordance with z-order, wherein partial rasterization of each layer is estimated to succeed within an allocated memory;

determining, while generating an intermediate format representation for a selected layer by partially rasterizing objects of the selected layer, that more than the allocated memory is required to complete the generating;

in response to determining that more than the allocated memory is required, splitting, in z-order, the selected layer into at least two sub-layers; and

rendering the graphical representation by generating an intermediate format representation for each of the sub-layers to facilitate rasterization of the selected layer within the allocated memory.

20. The method according to claim 19, wherein the intermediate format representations for the sub-layers are generated serially to facilitate the generating of the intermediate format representations within the allocated memory.

21. The method according to claim 19, wherein an intermediate format representation for a second sub-layer is generated in response to completing generation of an intermediate format representation for a first sub-layer.

22. The method according to claim 19, further comprising determining a complexity measure for the selected layer.

23. The method according to claim 22, wherein the selected layer is split by determining a pivot point based on the determined complexity measure.

24. The method according to claim 23, wherein the pivot point is further determined based on z-order of objects in a transparency group falling within the selected layer.

25. The method according to claim 19, wherein generating the intermediate format representation for each of the sub-layers starts from a K+1 strip, wherein at least K strips of the intermediate format representation are generated before determining that more than the allocated memory is required, wherein K is not less than 1.

26. The method according to claim 19, wherein the allocated memory is at least one of a working memory and a working memory together with a memory for storing the plurality of objects.

27. The method according to claim 19, further comprising merging, in z-order, at least the generated intermediate format representations.

28. The method according to claim 19, wherein the selected layer is associated with a reference count indicating a number of corresponding sub-layers to be rasterized.

29. The method according to claim 28, wherein the reference count is increased in response to splitting, in z-order, the selected layer, and the reference count is decreased in response to rasterizing a sub-layer associated with the selected layer.

30. An apparatus for rendering a graphical representation, the apparatus comprising:

a processor; and

a memory device storing a software program for directing the processor to perform a method comprising the steps of;

receiving a plurality of objects of the graphical representation grouped into layers in accordance with z-order, wherein partial rasterization of each layer is estimated to succeed within an allocated memory;

determining, while generating an intermediate format representation for a selected layer by partially rasterizing objects of the selected layer, that more than the allocated memory is required to complete the generating;

splitting, in z-order, the selected layer into at least two sub-layers in response to determining that more than the allocated memory is required; and

rendering the graphical representation by generating an intermediate format representation for each of the sub-layers to facilitate rasterization of the selected layer within the allocated memory.

31. A non-transitory computer readable storage medium storing program instructions for rendering a graphical representation by directing at least one processor to perform a method comprising the steps of:

receiving a plurality of objects of the graphical representation grouped into layers in accordance with z-order, wherein partial rasterization of each layer is estimated to succeed within an allocated memory;

determining, while generating an intermediate format representation for a selected layer by partially rasterizing objects of the selected layer, that more than the allocated memory is required to complete the generating;

splitting, in z-order, the selected layer into at least two sub-layers in response to determining that more than the allocated memory is required; and

rendering the graphical representation by generating an intermediate format representation for each of the sub-layers to facilitate rasterization of the selected layer within the allocated memory.

32. A graphics processing system comprising a hardware processor, a memory coupled to the hardware processor, and computer program instructions stored in the memory that, when executed on the processor to render a graphical representation, cause the graphics processing system to:

receive a plurality of objects of the graphical representation grouped into layers in accordance with z-order, wherein partial rasterization of each layer is estimated to succeed within an allocated memory;

determine, while generating an intermediate format representation for a selected layer by partially rasterizing objects of the selected layer, that more than the allocated memory is required to complete the generating;

split, in z-order, the selected layer into at least two sub-layers in response to determining that more than the allocated memory is required; and

render the graphical representation by generating an intermediate format representation for each of the sub-layers to facilitate rasterization of the selected layer within the allocated memory.