Graphics processing enhancement by tracking object and/or primitive identifiers

• US 9,710,957 B2
• Filed: 04/05/2014
• Issued: 07/18/2017
• Est. Priority Date: 04/05/2014
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Europe Limited (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Berghoff, Tobias
• Primary Examiner(s): Wu, Sing-Wai
• Application Number:
  US14/246,067
• Publication Number:
  US 20150287239A1
• Time in Patent Office: 1,200 Days
• CPC Class Codes: G06F 9/48 Program initiating; Program switching, e.g. by interrupt
• CPC Class Codes: G06T 11/40 Filling a planar surface by adding surface attributes, e.g. colour or texture
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/405 using Z-buffer
• CPC Class Codes: G06T 15/80 Shading
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation

Abstract

Graphics processing includes setting up a plurality of objects in a scene in virtual space, each object being defined by a set of vertices. A unique object identifier is assigned to each object and written to an ID buffer. Draw calls are issued to draw the objects associated with the object identifiers. Parameter values of the vertices are manipulated to output vertex parameter values. Primitives are set up from the vertices, each primitive being defined by one or more of the vertices. Each primitive belongs to one or more of the objects. Each primitive is rasterized at a plurality of pixels. Processing the pixels includes spatial or temporal anti-aliasing that utilizes the one or more object identifiers of the plurality of object identifiers. The pixels are processed for each rasterized primitive to generate an output frame.

Claims

1. A method of rendering graphics with a processor, the method comprising:

setting up a plurality of objects in a scene in virtual space, each said object being defined by a set of vertices;

assigning a unique object identifier to each of the objects;

issuing one or more draw calls to draw the objects;

manipulating parameter values of the vertices to generate output vertex parameter values;

setting up a plurality of primitives from the vertices, each said primitive being defined by a set of one or more of the vertices and each said primitive belonging to one or more of the objects;

performing geometry shading in which one or more output primitives are generated from one or more input primitives, wherein performing the geometry shading includes generating a primitive identifier for one or more primitives generated by the geometry shading by incrementing a primitive identifier for each input primitive by a maximum amount of geometric amplification of output primitives supported by an object with which the input primitives are associated;

rasterizing each of the primitives for a plurality of pixels;

writing a depth at each said pixel to a Z buffer, the Z buffer having a first resolution;

writing one or more of the object identifiers to at least one ID buffer, such that each pixel in the Z buffer is associated with an object identifier in the ID buffer;

processing the pixels for each of the rasterized primitives to determine a set of color values in a color buffer, the color buffer having a second resolution that is less than the first resolution; and

determining a set of display pixels for an output frame at the resolution of the Z buffer using the color values in the color buffer and the identifiers in the ID buffer.

2. The method of claim 1, wherein the determining the set of display pixels includes applying spatial and/or temporal antialiasing using the one or more object identifiers in the ID buffer, in combination with the color buffer and/or Z buffer .

3. The method of claim 1, wherein the ID buffer is a dedicated ID buffer dedicated to storing identifiers.

4. The method of claim 1, wherein the ID buffer is shared buffer.

5. The method of claim 1, wherein the ID buffer is a plurality of dedicated ID buffers.

6. The method of claim 1, wherein assigning the unique object identifier includes adding an object identifier and a primitive identifier.

7. The method of claim 1, wherein the assigning the unique object identifier includes performing a logical OR between an object identifier and a primitive identifier.

8. The method of claim 1, further comprising:

assigning a unique primitive identifier to each of the primitives and writing the primitive identifier to the ID buffer.

9. The method of claim 8, wherein the determining the set of display pixels includes applying spatial and/or temporal antialiasing using the object identifiers and the primitive identifiers in the ID buffer, in combination with the color buffer and/or Z buffer.

10. The method of claim 8, wherein the at least one ID buffer includes a dedicated object ID buffer dedicated to storing object identifiers and a dedicated primitive ID buffer dedicated to storing primitive identifiers.

11. The method of claim 8, wherein the at least one ID buffer is a dedicated ID buffer dedicated to storing object identifiers and primitive identifiers.

12. The method of claim 8, wherein said manipulating parameter values of the vertices comprises performing geometry shader computations by a geometry shader, wherein said assigning the unique primitive identifier comprises:

generating a primitive ID with the geometry shader during the geometry shader computations.

13. The method of claim 8, wherein said assigning the unique primitive identifier to each of the primitives comprises, before said manipulating parameter values of the vertices:

performing primitive processing operations to generate a primitive ID, and

incrementing the primitive ID each time the primitive processing operations output a patch.

14. The method of claim 8, wherein said manipulating parameter values of the vertices comprises performing tessellation computations with a tessellation unit, wherein said assigning the unique primitive identifier comprises:

before said tessellation computations, generating a primitive ID;

receiving the primitive ID with the tessellation unit; and

incrementing the primitive ID with each output primitive generated by the tessellation unit during the tessellation computations.

15. The method of claim 1, wherein said manipulating the parameter values of the vertices includes manipulating the parameter values with a vertex shader and generating a primitive ID for eventual write to the ID buffer.

16. The method of claim 1, wherein the assigning the unique object identifier includes assigning an object identifier for an instanced object with a vertex shader.

17. The method of claim 1, wherein the assigning the unique object identifier includes assigning an object identifier for an instanced object with a vertex shader by backtracking from an object identifier and a primitive identifier to determine an instance identifier for the instanced object.

18. The method of claim 1, further comprising performing geometry shading in which one or more output primitives are generated from one or more input primitives.

19. The method of claim 1, further comprising performing geometry shading in which one or more output primitives are generated from one or more input primitives, wherein the performing geometry shading includes generating a primitive identifier for one or more primitives generated by the geometry shading.

20. The method of claim 1,

wherein said processing the pixels for each of the rasterized primitives includes manipulating pixel parameter values with a pixel shader,

wherein said manipulating the pixel parameter values with the pixel shader includes reading one or more of the primitive identifiers with the pixel shader.

21. The method of claim 1, further comprising tracking the plurality of object identifiers to at least one subsequent frame subsequent to said output frame, such that object identifiers are consistently assigned to objects from frame to frame.

22. The method of claim 1, wherein the setting up a plurality of objects includes assigning the unique object identifier to each of the objects and writing the object identifier to an ID buffer.

23. The method of claim 1, wherein the issuing the draw call includes assigning the unique object identifier to each of the objects and writing the object identifier to an ID buffer, whereby there is a plurality of object identifiers.

24. The method of claim 1 wherein the assigning a unique object identifier to each of the objects includes assigning the same object identifier to two or more instances of the same object in the same frame or in two or more different frames.

25. The method of claim 1, wherein said determining the set of display pixels includes using the one or more object identifiers to keep track of which primitives or pixels belong to which objects.

26. The method of claim 1, wherein said determining the set of display pixels includes using one or more of the object identifiers to provide pixel values for pixels in one frame by repeating a pixel value for the same pixel in the same object in a previous frame.

27. The method of claim 1, wherein said determining the set of display pixels includes utilizing one or more object identifiers where the same object is re-drawn multiple times in a given frame.

28. The method of claim 1, further comprising using object identifiers to keep track of the flow of objects from frame to frame.

88. A non-transitory computer readable medium having processor-executable instructions embodied therein, wherein execution of the instructions by a processor causes the processor to implement a method of rendering graphics with a processor as set forth in claim 1.

29. A system comprising:

a processor, and

a memory coupled to the processor,

wherein the processor is configured to perform a method of rendering graphics, the method comprising;

setting up a plurality of objects in a scene in virtual space, each said object being defined by a set of vertices;

assigning a unique object identifier to each of the objects;

issuing one or more draw calls to draw the objects;

manipulating parameter values of the vertices to generate output vertex parameter values;

setting up a plurality of primitives from the vertices, each said primitive being defined by a set of one or more of the vertices and each said primitive belonging to one or more of the objects;

performing geometry shading in which one or more output primitives are generated from one or more input primitives, wherein the performing geometry shading includes generating a primitive identifier for one or more primitives generated by the geometry shading by incrementing a primitive identifier for each input primitive by a maximum amount of geometric amplification of output primitives supported by an object with which the input primitives are associated;

rasterizing each of the primitives for a plurality of pixels;

writing a depth at each said pixel to a Z buffer, the Z buffer having a first resolution;

writing one or more of the object identifiers to at least one ID buffer, such that each pixel in the Z buffer is associated with an object identifier in the ID buffer;

processing the pixels for each of the rasterized primitives to determine a set of color values in a color buffer, the color buffer having a second resolution that is less than the first resolution; and

determining a set of display pixels for an output frame at the resolution of the Z buffer using the color values in the color buffer and the identifiers in the ID buffer.

30. The system of claim 29,

wherein the processor includes a central processing unit (CPU) and a graphics processing unit (GPU).

31. The system of claim 29, further comprising a display device,

wherein the method further comprises presenting the graphics on the display device.

32. The system of claim 29, wherein the determining the set of display pixels includes applying spatial and/or temporal antialiasing using the one or more object identifiers in the ID buffer, in combination with the color buffer and/or Z buffer.

33. The system of claim 29, wherein the ID buffer is a dedicated ID buffer dedicated to storing identifiers.

34. The system of claim 29, wherein the ID buffer is shared buffer.

35. The system of claim 29, wherein the ID buffer is a plurality of dedicated ID buffers.

36. The system of claim 29, wherein the assigning the unique object identifier includes adding an object identifier and a primitive identifier.

37. The system of claim 29, wherein the assigning the unique object identifier includes performing a logical OR between an object identifier and a primitive identifier.

38. The system of claim 29, wherein the method further comprises:

assigning a unique primitive identifier to each of the primitives and writing the primitive identifier to the ID buffer.

39. The system of claim 38, wherein the determining the set of display pixels includes applying spatial and/or temporal antialiasing using the object identifiers and the primitive identifiers in the ID buffer, in combination with the color buffer and/or Z buffer.

40. The system of claim 38, wherein said manipulating parameter values of the vertices comprises performing geometry shader computations by a geometry shader, wherein said assigning the unique primitive identifier comprises:

generating a primitive ID with the geometry shader during the geometry shader computations.

41. The system of claim 38, wherein said assigning the unique primitive identifier to each of the primitives comprises, before said manipulating parameter values of the vertices:

performing primitive processing operations to generate a primitive ID, and

incrementing the primitive ID each time the primitive processing operations output a patch.

42. The system of claim 38, wherein said manipulating parameter values of the vertices comprises performing tessellation computations with a tessellation unit, wherein said assigning the unique primitive identifier comprises:

before said tessellation computations, generating a primitive ID;

receiving the primitive ID with the tessellation unit; and

incrementing the primitive ID with each output primitive generated by the tessellation unit during the tessellation computations.

43. The system of claim 29, wherein the at least one ID buffer includes a dedicated object ID buffer dedicated to storing object identifiers and a dedicated primitive ID buffer dedicated to storing primitive identifiers.

44. The system of claim 29, wherein the at least one ID buffer is a dedicated ID buffer dedicated to storing object identifiers and primitive identifiers.

45. The system of claim 29, wherein said manipulating the parameter values of the vertices includes manipulating the parameter values with a vertex shader and generating a primitive ID for eventual write to the ID buffer.

46. The system of claim 29, wherein the assigning the unique object identifier includes assigning an object identifier for an instanced object with a vertex shader.

47. The system of claim 29, wherein said assigning the unique object identifier includes assigning an object identifier for an instanced object with a vertex shader by backtracking from an object identifier and a primitive identifier to determine an instance identifier for the instanced object.

48. The system of claim 29, wherein the method further comprises performing geometry shading in which one or more output primitives are generated from one or more input primitives.

49. The system of claim 29, wherein the method further comprises performing geometry shading in which one or more output primitives are generated from one or more input primitives, wherein the performing geometry shading includes generating a primitive identifier for one or more primitives generated by the geometry shading.

50. The system of claim 29,

wherein said processing the pixels for each of the rasterized primitives includes manipulating pixel parameter values with a pixel shader,

wherein said manipulating the pixel parameter values with the pixel shader includes reading one or more of the primitive identifiers with the pixel shader.

51. The system of claim 29, further comprising tracking the plurality of object identifiers to at least one subsequent frame subsequent to said output frame, such that object identifiers are consistently assigned to objects from frame to frame.

52. The system of claim 29, wherein the setting up a plurality of objects includes assigning the unique object identifier to each of the objects and writing the object identifier to an ID buffer.

53. The system of claim 29, wherein the issuing the draw call includes assigning the unique object identifier to each of the objects and writing the object identifier to an ID buffer, whereby there is a plurality of object identifiers.

54. The system of claim 29, wherein the assigning a unique object identifier to each of the objects includes assigning the same object identifier to two or more instances of the same object in the same frame or in two or more different frames.

55. The system of claim 29, wherein said determining the set of display pixels includes using the one or more object identifiers to keep track of which primitives or pixels belong to which objects.

56. The system of claim 29, wherein said determining the set of display pixels includes using one or more of the object identifiers to provide pixel values for pixels in one frame by repeating a pixel value for the same pixel in the same object in a previous frame.

57. The system of claim 29, wherein said determining the set of display pixels includes utilizing one or more object identifiers where the same object is re-drawn multiple times in a given frame.

58. The system of claim 29, wherein the method further comprises using object identifiers to keep track of the flow of objects from frame to frame.

59. A graphics processing unit (GPU) comprising:

an ID module configured to;

receive information corresponding to a plurality of objects in a scene in virtual space, each said object being defined by a set of vertices;

receive one or more object identifiers in a command buffer for each said object in the plurality, and write the one or more object identifiers to an ID buffer;

wherein the GPU is further configured to perform geometry shading in which one or more output primitives are generated from one or more input primitives, wherein performing geometry shading includes generating a primitive identifier for one or more primitives generated by the geometry shading by incrementing a primitive identifier for each input primitive by a maximum amount of geometric amplification of output primitives supported by an object with which the input primitives are associated; and

wherein the GPU is further configured to render the two or more instanced objects by;

manipulating parameter values of the vertices to generate output vertex parameter values;

setting up a plurality of primitives from the vertices, each said primitive being defined by a set of one or more of the vertices and each said primitive belonging to one or more of the objects;

rasterizing each of the primitives for a plurality of pixels;

writing a depth at each said pixel to a Z buffer, the Z buffer having a first resolution;

writing one or more of the object identifiers to at least one ID buffer, such that each pixel in the Z buffer is associated with an object identifier in the ID buffer;

processing the pixels for each of the rasterized primitives to determine a set of color values in a color buffer, the color buffer having a second resolution that is less than the first resolution; and

determining a set of display pixels for an output frame at the resolution of the Z buffer using the color values in the color buffer and the identifiers in the ID buffer.

60. The GPU of claim 59, wherein the ID module is a hardware module.

61. The GPU of claim 59, wherein the determine the set of display pixels includes apply spatial and/or temporal antialiasing using the one or more object identifiers in the ID buffer, in combination with the color buffer and/or Z buffer .

62. The GPU of claim 59, wherein the ID buffer is a dedicated ID buffer dedicated to storing identifiers.

63. The GPU of claim 59, wherein the ID buffer is shared buffer.

64. The GPU of claim 59, wherein the ID buffer is a plurality of dedicated ID buffers.

65. The GPU of claim 59, wherein the assigning the unique object identifier includes adding an object identifier and a primitive identifier.

66. The GPU of claim 59, wherein the assigning the unique object identifier includes performing a logical OR between an object identifier and a primitive identifier.

67. The GPU of claim 59, wherein the GPU is further configured to assign a unique primitive identifier to each of the primitives and writing the primitive identifier to the ID buffer.

68. The GPU of claim 67, wherein said determine the set of display pixels includes apply spatial and/or temporal antialiasing using the object identifiers and the primitive identifiers in the ID buffer, in combination with the color buffer and/or Z buffer.

69. The GPU of claim 67, wherein said manipulate parameter values of the vertices comprises performing geometry shader computations by a geometry shader, wherein said assign the unique primitive identifier comprises:

generate a primitive ID with the geometry shader during the geometry shader computations.

70. The GPU of claim 67, wherein said assign the unique primitive identifier to each of the primitives comprises, before said manipulate parameter values of the vertices:

perform primitive processing operations to generate a primitive ID; and

increment the primitive ID each time the primitive processing operations output a patch.

71. The GPU of claim 67, wherein said manipulate parameter values of the vertices comprises performing tessellation computations with a tessellation unit, wherein said assigning the unique primitive identifier comprises:

before said tessellation computations, generating a primitive ID;

receiving the primitive ID with the tessellation unit; and

incrementing the primitive ID with each output primitive generated by the tessellation unit during the tessellation computations.

72. The GPU of claim 67,

wherein said processing the pixels for each of the rasterized primitives includes manipulating pixel parameter values with a pixel shader,

wherein said manipulating the pixel parameter values with the pixel shader includes reading one or more of the primitive identifiers with the pixel shader.

73. The GPU of claim 59, wherein the at least one ID buffer includes a dedicated object ID buffer dedicated to storing object identifiers and a dedicated primitive ID buffer dedicated to storing primitive identifiers.

74. The GPU of claim 59, wherein the at least one ID buffer is a dedicated ID buffer dedicated to storing object identifiers and primitive identifiers.

75. The GPU of claim 59, wherein said manipulate the parameter values of the vertices includes manipulating the parameter values with a vertex shader and generating a primitive ID for eventual write to the ID buffer.

76. The GPU of claim 59, wherein said assign the unique object identifier includes assigning an object identifier for an instanced object with a vertex shader.

77. The GPU of claim 59, wherein said assign the unique object identifier includes assigning an object identifier for an instanced object with a vertex shader by backtracking from an object identifier and a primitive identifier to determine an instance identifier for the instanced object.

78. The GPU of claim 59, wherein the GPU is further configured to perform geometry shading in which one or more output primitives are generated from one or more input primitives.

79. The GPU of claim 59, wherein the GPU is further configured to perform geometry shading in which one or more output primitives are generated from one or more input primitives, wherein performing geometry shading includes generating a primitive identifier for one or more primitives generated by the geometry shading.

80. The GPU of claim 59, wherein the GPU is further configured track the plurality of object identifiers to at least one subsequent frame subsequent to said output frame, such that object identifiers are consistently assigned to objects from frame to frame.

81. The GPU of claim 59, wherein said set up a plurality of objects includes assigning the unique object identifier to each of the objects and writing the object identifier to an ID buffer.

82. The GPU of claim 59, wherein said issue the draw call includes assigning the unique object identifier to each of the objects and writing the object identifier to an ID buffer, whereby there is a plurality of object identifiers.

83. The GPU of claim 59, wherein said assign a unique object identifier to each of the objects includes assigning the same object identifier to two or more instances of the same object in the same frame or in two or more different frames.

84. The GPU of claim 59, wherein said determine the set of display pixels includes use the one or more object identifier them to keep track of which primitives or pixels belong to which objects.

85. The GPU of claim 59, wherein said determine the set of display pixels includes using one or more of the object identifiers to provide pixel values for pixels in one frame by repeating a pixel value for the same pixel in the same object in a previous frame.

86. The GPU of claim 59, wherein said determine the set of display pixels includes utilizing one or more object identifiers where the same object is re-drawn multiple times in a given frame.

87. The GPU of claim 59, wherein the GPU is further configured to use object identifiers to keep track of the flow of objects from frame to frame.

89. A method of rendering one or more graphics frames with a processor, the one or more graphics frames depicting a plurality of objects, the method comprising:

a) assigning a unique object identifier to each of the objects;

b) for each said frame, manipulating vertices of each of the objects in virtual space, each said object including one or more primitives;

b′

) for each said frame, performing geometry shading in which one or more output primitives are generated from one or more input primitives, wherein performing geometry shading includes generating a primitive identifier for one or more primitives generated by the geometry shading by incrementing a primitive identifier for each input primitive by a maximum amount of geometric amplification of output primitives supported by an object with which the input primitives are associated

c) for each said frame, converting the primitives of the objects, from the virtual space to screen space, wherein the screen space comprises a plurality of depth samples and a plurality of color samples;

d) for each said frame, determining a final color value of each of the color samples in screen space and a final depth value of each of the depth samples in screen space, wherein the final depth values are stored in a depth buffer and the final color values are stored in a color buffer, wherein the depth buffer has a higher resolution than the color buffer;

e) for each said frame, storing at least one of the object identifiers in an ID buffer, wherein each of the final depth samples in the depth buffer are associated with one of the object identifiers in the ID buffer;

f) for each said frame, determining a final set of display pixels at the higher resolution of the depth buffer using the color values in the color buffer and the object identifiers in the ID buffer.