Varying effective resolution by screen location by altering rasterization parameters

• US 10,685,425 B2
• Filed: 09/13/2019
• Issued: 06/16/2020
• Est. Priority Date: 04/05/2014
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Inventors: Cerny, Mark Evan
• Primary Examiner(s): Wu, Sing-Wai
• Application Number:
  US16/570,258
• Publication Number:
  US 20200005431A1
• Time in Patent Office: 277 Days
• CPC Class Codes: G02B 2027/0123 comprising devices increasing the field of view
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 11/40 Filling a planar surface by adding surface attributes, e.g. colour or texture
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/80 Shading
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 2210/36 Level of detail
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

In graphics processing data is received representing one or more vertices for a scene in a virtual space. Primitive assembly is performed on the vertices to compute projections of the vertices from virtual space onto a viewport of the scene in a screen space of a display device containing a plurality of pixels, the plurality of pixels being subdivided into a plurality of subsections. Scan conversion determines which pixels of the plurality of pixels are part of each primitive that has been converted to screen space coordinates. Coarse rasterization for each primitive determines which subsection or subsections the primitive overlaps. Metadata associated with the subsection a primitive overlaps determines a pixel resolution for the subsection. The metadata is used in processing pixels for the subsection to generate final pixel values for the viewport of the scene that is displayed on the display device in such a way that parts of the scene in two different subsections have different pixel resolution.

Claims

1. A method for graphics processing, comprising:

performing primitive assembly on vertices of a scene in a virtual space to compute projections of the vertices from the virtual space onto a viewport of the scene in a screen space of a display device containing a plurality of pixels, the plurality of pixels of the display device being subdivided into a plurality of subsections;

determining which pixels of the plurality of pixels are part of each primitive that has been converted to screen space coordinates; and

using metadata associated with a subsection a primitive overlaps in processing pixels for the subsection to generate final pixel values for the viewport of the scene that is displayed on the display device, wherein the metadata determines a pixel resolution for the subsection, wherein the pixel resolution is a number of active pixels per unit area of the screen for the subsection, and wherein the pixel values are generated in such a way that parts of the scene in two different subsections have different pixel resolutions and pixel data stored in a memory does not change wherein output screen areas are packed into a smaller memory footprint.

2. The method of claim 1, wherein the metadata specifies a number of active pixels in the subsection and wherein using the metadata in processing the pixels for the subsection to generate the final pixel values includes performing pixel processing only on the specified number of active pixels for the subsection.

3. The method of claim 2, wherein the metadata includes scaling parameters and wherein using the metadata in processing pixels for the subsection includes adjusting vertex locations in screen space and texture space gradients for the primitive to account for the pixel resolution for the subsection.

4. The method of claim 2, wherein the metadata specifies that all the pixels for the subsection are active pixels when the pixel resolution for the subsection is a standard resolution and wherein the metadata specifies that less than all the pixels for the subsection are active pixels when the pixel resolution subsection is less than the standard resolution.

5. The method of claim 2, wherein the metadata specifies a standard resolution when the subsection is located at or near a center of the screen space and wherein the metadata specifies less than the standard resolution when the subsection is located at or near an edge or corner of the screen space.

6. The method of claim 2, wherein the metadata includes one or more scaling parameters and wherein using the metadata in processing pixels for the subsection includes adjusting vertex locations in screen space, and texture space gradients and parameter barycentric interpolant values for the primitive to account for the pixel resolution for the subsection and adjusting texture space gradients includes performing per-pixel gradient scale correction in applying textures to account for different pixel resolutions for neighboring subsections of the plurality of subsections.

7. The method of claim 1, wherein the display device is characterized by a field of view of 90 degrees or more.

8. The method of claim 1, wherein the display device is a head-mounted display device.

9. The method of claim 1, wherein the screen space corresponds to a curved screen.

10. The method of claim 1, further comprising determining a portion of a screen of the display device that a user is looking at and wherein the metadata is configured to vary the pixel resolution such that pixel resolution is highest for one or more subsections of the screen containing the portion the user is looking at.

11. The method of claim 1, wherein the metadata is static for given optics and a given field of view of the display device.

12. The method of claim 1, wherein the metadata specifies a number of active pixels in the subsection and wherein using the metadata in processing the pixels for the subsection to generate the final pixel values includes performing pixel processing only on the specified number of active pixels for the subsection, and wherein the metadata is configured to specify different numbers of active pixels for different subsections of the plurality of subsections.

13. The method of claim 1, wherein, each subsection of the plurality of subsections corresponds to a fixed size portion of the screen space.

14. The method of claim 1, wherein each subsection of the plurality of subsections corresponds to a variable size portion of the screen space.

15. The method of claim 1, wherein the metadata defines each subsection of the plurality of subsections by ranges of pixels in vertical and horizontal directions.

16. The method of claim 1, wherein the metadata defines each subsection of the plurality of subsections by coarse rasterization tiles of some size.

17. The method of claim 1, wherein output screen areas are packed into a smaller memory footprint by embedding additional information into the metadata to encode output memory offsets.

18. The method of claim 1, wherein a portion of the metadata associated with a particular subsection of the plurality of subsections includes information specifying a number of active pixels for the particular subsection.

19. The method of claim 1, wherein the metadata is stored in the form of a table in a memory and/or graphics memory.

20. A non-transitory computer-readable medium having computer executable instructions embodied therein that, when executed, implement the method of claim 1.

21. A system for graphics processing, comprising

a memory storing executable instructions;

and

a graphics processing unit configured to implement a method by executing the instructions to cause the system to;

perform primitive assembly on vertices of a scene in a virtual space to compute projections of the vertices from the virtual space onto a viewport of the scene in a screen space of a display device containing a plurality of pixels, the plurality of pixels of the display device being subdivided into a plurality of subsections;

determine which pixels of the plurality of pixels are part of each primitive, that has been converted to screen space coordinates; and

using metadata associated with a subsection a primitive overlaps in processing pixels for the subsection to generate final pixel values for the viewport of the scene that is displayed on the display device, wherein the metadata determines a pixel resolution for the subsection, wherein the pixel resolution is a number of active pixels per unit area of the screen for the subsection, and wherein the pixel values are generated in such a way that parts of the scene in two different subsections have different pixel resolution and the pixel data stored in a memory is not changed wherein output screen areas are packed into a smaller memory footprint.

22. The system of claim 21, the metadata specifies a number of active pixels in the subsection and wherein the graphics processing unit is configured to use the metadata in processing the pixels for the subsection to generate the final pixel values by performing pixel processing only on the specified number of active pixels for the subsection.

23. The system of claim 22, wherein the metadata specifies that all the pixels for the subsection are active pixels when the pixel resolution for the subsection is a standard resolution and wherein the metadata specifies that less than all the pixels for the subsection are active pixels when the pixel resolution subsection is less than the standard resolution.

24. The system of claim 22, wherein the metadata includes scaling parameters and wherein the GPU is configured to use the metadata in processing pixels for the subsection by adjusting vertex locations in screen space, and texture space gradients and parameter barycentric interpolant values for the primitive to account for the pixel resolution for the subsection and wherein the GPU is configured to adjust the texture space gradients by performing per-pixel gradient scale correction in applying textures to account for different pixel resolutions for neighboring subsections of the plurality of subsections.

25. The system of claim 21, wherein the metadata specifies a standard resolution when the subsection is located at or near a center of the screen space and wherein the metadata specifies less than the standard resolution when the subsection is located at or near an edge or corner of the screen space.

26. The system of claim 21, wherein the display device is characterized by a field of view of 90 degrees or more.

27. The system of claim 21, wherein the display device is a head-mounted display device.

28. The system of claim 21, wherein the screen space corresponds to a curved screen.

29. The system of claim 21, further comprising the display device.

30. The system of claim 21, wherein the system is configured to determine a portion of a screen of the display device that a user is looking at and wherein the metadata is configured to vary the pixel resolution such that pixel resolution is highest for one or more subsections of the screen containing the portion the user is looking at.

31. The system of claim 21, wherein the system is configured to use static metadata for given optics and field of view of the display device.

32. The system of claim 21, wherein the metadata specifies a number of active pixels in the subsection and wherein the graphics processing unit is configured to use the metadata in processing the pixels for the subsection to generate the final pixel values by performing pixel processing only on the specified number of active pixels for the subsection, and wherein the metadata is configured to specify different numbers of active pixels for different subsections of the plurality of subsections.

33. The system of claim 21, wherein, each subsection of the plurality of subsections corresponds to a fixed size portion of the screen space.

34. The system of claim 21, wherein each subsection of the plurality of subsections corresponds to a variable size portion of the screen space.

35. The system of claim 21, wherein the metadata defines each subsection of the plurality of subsections by ranges of pixels in vertical and horizontal directions.

36. The system of claim 21, wherein the metadata defines each subsection of the plurality of subsections by coarse rasterization tiles of some size.

37. The system of claim 21, wherein output screen areas are packed into a smaller memory footprint by embedding additional information into the metadata to encode output memory offsets.

38. The system of claim 21, wherein a portion of the metadata associated with a particular subsection of the plurality of subsections includes information specifying a number of active pixels for the particular subsection.

39. The system of claim 21, wherein the metadata is stored in the form of a table in a memory and/or graphics memory.