Varying effective resolution by screen location in graphics processing by approximating projection of vertices onto curved viewport

• US 10,438,319 B2
• Filed: 12/01/2017
• Issued: 10/08/2019
• Est. Priority Date: 04/05/2014
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Inventors: Cerny, Mark Evan
• Primary Examiner(s): Wu, Sing-Wai
• Application Number:
  US15/829,579
• Publication Number:
  US 20180089798A1
• Time in Patent Office: 676 Days
• CPC Class Codes: G02B 2027/0123 comprising devices increasing the field of view
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 15/80 Shading
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 2210/36 Level of detail
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

In graphics processing data is received representing one or more vertices for a scene in a virtual space. A projection of the vertices onto a screen space of a display device is performed. The projection approximates a projection of the vertices onto a curved viewport. Primitive assembly is performed on the vertices to generate a one or more primitives in screen space from the projection of the vertices onto the screen space. Scan conversion is performed on the one or more primitives to determine which pixel or pixels of the plurality of pixels are part of corresponding primitives of the one or more primitives. A finished frame is then generated by performing pixel processing to assign pixel values to the pixel or pixels that are part of the corresponding primitives. In some implementations, the finished frame can be stored in the memory or displayed on the display device.

Claims

1. A method for graphics processing with a graphics processing system having a graphics processing unit and memory, comprising:

performing a mathematical projection of one or more vertices for a scene in a virtual space onto a screen space before rendering the scene on a display device having a plurality of pixels, wherein the mathematical projection approximates projection of the vertices onto a curved viewport by mathematically projecting the vertices onto a flat screen space followed by transformation of subsections of the flat screen space representing approximately equal solid angle to corresponding subsections of approximately equal area thereby reducing an effective area of the screen space that is to be rendered, wherein the transformation of subsections occurs before rendering, wherein performing the mathematical projection before performing primitive assembly on the vertices to generate one or more primitives reduces a computational load associated with rendering; and

storing the finished frame in the memory or displaying the finished frame on the display device.

2. The method of claim 1, performing the projection of the vertices includes:

computing initial projections of the vertices from virtual space onto a planar viewport of the scene, the planar viewport being subdivided into a plurality of subsections, wherein each subsection of the plurality corresponds to an approximately equal solid angle of the screen space;

performing coarse rasterization for each primitive converted to screen space coordinates to determine which subsection or subsections the primitive overlaps; and

for each primitive of the one or more primitives performing a screenspace correction for each subsection that the primitive overlaps to map the coordinates of each vertex of the primitive to a corresponding corrected vertex coordinates associated with the subsection, wherein the screen space correction is equivalent to mapping each subsection that the primitive overlaps to a corresponding planar corrected subsection of a plurality of planar corrected subsections of the curved viewport, wherein the corrected subsections of the curved viewport have equal area.

3. The method of claim 2, wherein performing the screen space correction includes performing a computation involving applying a piecewise linear approximation of an arctangent formula to screen space coordinates of one or more vertices of the primitive to generate corresponding corrected vertices.

4. The method of claim 2, wherein performing the screen space correction includes performing a computation involving applying a piecewise linear approximation of the image distortion produced by physical optics in the display device to screen space coordinates of one or more vertices of the primitive to generate corresponding corrected vertices.

5. The method of claim 2, wherein the scan conversion is performed per corrected subsection, wherein for a given primitive of the one or more primitives in a given subsection of the plurality of subsections scan conversion is performed on that portion of the given primitive located in the given subsection using the corrected vertex coordinates for the given primitive that are associated with the given subsection.

6. The method of claim 2, wherein the subsections of the plurality of subsections of the planar viewport are rectangular.

7. The method of claim 1, wherein the projection of the vertices, primitive assembly, and scan conversion are implemented in software by a compute shader.

8. The method of claim 1, wherein the projection of the vertices, primitive assembly, and scan conversion are implemented in hardware.

9. The method of claim 1, further comprising determining a portion of a screen of the display device that a user is looking at and wherein the projection is configured to vary an effective pixel resolution such that the effective pixel resolution is highest for one or more subsections of the screen containing the portion the user is looking at.

10. The method of claim 1, wherein the projection is static for given optics and field of view of the display device.

11. The method of claim 1, wherein the projection is static for given optics and field of view of the display device and wherein the projection is configured such that, one or more central subsections of the screen have a full effective pixel resolution, and subsections further from the center have progressively lower effective pixel resolution.

12. The method of claim 1, wherein the display device is characterized by a field of view of 90 degrees or more.

13. The method of claim 1, wherein the display device is a head-mounted display device.

14. A non-transitory computer-readable medium having computer executable instructions embodied therein that, when executed, implement the method of claim 1.

15. A system for graphics processing, comprising:

a graphics processing unit (GPU); and

a memory,

wherein the GPU is configured to perform a mathematical projection of one or more vertices for a scene in a virtual space onto a screen space before rendering the scene on a display device having a plurality of pixels, wherein the mathematical projection approximates a projection of the vertices onto a curved viewport by projecting the vertices onto a flat screen space followed by transformation of subsections of the flat screen space representing approximately equal solid angle to corresponding subsections of approximately equal area thereby reducing an effective area of the screen space that is to be rendered, wherein the transformation of subsections occurs before rendering, wherein performing the mathematical projection before performing primitive assembly on the vertices and scan conversion

reduces a computational load associated with rendering; and

wherein the GPU is configured to store the finished frame in the memory or display the finished frame on the display device.

16. The system of claim 15, wherein the GPU is configured to perform the projection of the vertices by:

computing initial projections of the vertices from virtual space onto a planar viewport of the scene, the planar viewport being subdivided into a plurality of subsections, wherein each subsection of the plurality corresponds to an approximately equal solid angle of the screen space;

performing coarse rasterization for each primitive converted to screen space coordinates to determine which subsection or subsections the primitive overlaps; and

for each primitive of the one or more primitives performing a screenspace correction for each subsection that the primitive overlaps to map the coordinates of each vertex of the primitive to a corresponding corrected vertex coordinates associated with the subsection, wherein the screen space correction is equivalent to mapping each subsection that the primitive overlaps to a corresponding planar corrected subsection of a plurality of planar corrected subsections of the curved viewport.

17. The system of claim 16, wherein the GPU is configured to perform the screen space correction by performing a computation involving applying a piecewise linear approximation of an arctangent formula to screen space coordinates of one or more vertices of the primitive to generate corresponding corrected vertices.

18. The system of claim 16, wherein the GPU is configured to perform the screen space correction by performing a computation involving applying a piecewise linear approximation of image distortion produced by physical optics in the display device to screen space coordinates of one or more vertices of the primitive to generate corresponding corrected vertices.

19. The system of claim 16, wherein the GPU is configured to perform the scan conversion per corrected subsection, wherein for a given primitive of the one or more primitives in a given subsection of the plurality of subsections the GPU performs the scan conversion on that portion of the given primitive located in the given subsection using the corrected vertex coordinates for the given primitive that are associated with the given subsection.

20. The system of claim 16, wherein the subsections of the plurality of subsections of the planar viewport are rectangular.

21. The system of claim 16, wherein the corrected subsections of the curved viewport have equal area.

22. The system of claim 21, wherein the subsections of the plurality of subsections of the planar viewport are rectangular.

23. The system of claim 15, wherein the GPU is configured to implement projection of the vertices, primitive assembly, and scan conversion in software with a compute shader.

24. The system of claim 15, wherein the GPU is configured to implement the projection of the vertices, primitive assembly, and scan conversion in hardware.

25. The system of claim 15, wherein the GPU is configured to implement coarse division of primitives between subsections in software, and the projection of the vertices, primitive assembly, and scan conversion in hardware.

26. The system of claim 25, wherein the GPU is configured to associate subsection indices to primitive vertices in software, with each subsection index selecting a screen space projection and viewport from a palette implemented in hardware.

27. The system of claim 25, wherein the GPU is configured to associate subsection indices to primitive vertex indices in software, with each subsection index selecting a screen space projection and viewport from a palette implemented in hardware.

28. The system of claim 15, further comprising the display device.

29. The system of claim 15, wherein the display device is characterized by a field of view of 90 degrees or more.

30. The system of claim 15, wherein the display device is a head-mounted display device.

31. The system of claim 15, wherein the system is configured to determine a portion of a screen of the display device that a user is looking at and wherein the projection is configured to vary an effective pixel resolution such that the effective pixel resolution is highest for one or more subsections of the screen containing the portion the user is looking at.

32. The system of claim 15, wherein the projection is static for given optics and field of view of the display device.

33. The system of claim 15, wherein the projection is static for given optics and field of view of the display device and wherein the projection is configured such that, one or more central subsections of the screen have a full effective pixel resolution, and subsections further from the center have progressively lower effective pixel resolution.