Temporal Supersampling for Foveated Rendering Systems

• US 20180357749A1
• Filed: 02/26/2018
• Published: 12/13/2018
• Est. Priority Date: 06/09/2017
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Young, Andrew, Ho, Chris, Stafford, Jeffrey Roger
• Granted Patent: US 10,713,752 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 11/40 Filling a planar surface by adding surface attributes, e.g. colour or texture
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G09G 2310/0232 Special driving of display border areas
• CPC Class Codes: G09G 2340/0428 Gradation resolution change
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2360/08 Power processing, i.e. workload management for processors involved in display operations, such as CPUs or GPUs
• CPC Class Codes: G09G 2360/18 Use of a frame buffer in a display terminal, inclusive of the display panel
• CPC Class Codes: G09G 2370/02 Networking aspects
• CPC Class Codes: G09G 5/001 Arbitration of resources in a display system, e.g. control of access to frame buffer by video controller and/or main processor
• CPC Class Codes: G09G 5/391 Resolution modifying circuits, e.g. variable screen formats
• CPC Class Codes: G09G 5/42 characterised by the display of patterns using a display memory without fixed position correspondence between the display memory contents and the display position on the screen

Abstract

Methods and systems are provided for using temporal supersampling to increase a displayed resolution associated with peripheral region of a foveated rendering view. A method for enabling reconstitution of higher resolution pixels from a low resolution sampling region for fragment data is provided. The method includes an operation for receiving a fragment from a rasterizer of a GPU and for applying temporal supersampling to the fragment with the low resolution sampling region over a plurality of prior frames to obtain a plurality of color values. The method further includes an operation for reconstituting a plurality of high resolution pixels in a buffer that is based on the plurality of color values obtained via the temporal supersampling. Moreover, the method includes an operation for sending the plurality of high resolution pixels for display.

Claims

1. A method for drawing higher resolution pixels from a low resolution sampling region, comprising:

receiving a fragment from a rasterizer;

applying temporal supersampling to the fragment with the low resolution sampling region over a plurality of prior frames for obtaining a plurality color values;

reconstituting a plurality of high resolution pixels in a buffer based on the plurality of color values obtained via the temporal supersampling; and

sending, from the buffer, the plurality of high resolution pixels for presentation on a display.

2. The method of claim 1, wherein the temporal supersampling includes sampling at one location within the low resolution sampling region for each of the plurality of prior frames.

3. The method of claim 1, wherein the temporal supersampling includes sampling locations for the low resolution sampling region as determined by pixel reprojection.

4. The method of claim 1, wherein the temporal supersampling includes sampling locations for the low resolution sampling region as determined by jitter.

5. The method of claim 1, wherein the plurality of high resolution pixels is greater in number than the number of the plurality of prior frames.

6. The method of claim 1, wherein the reconstituting of the plurality of high resolution pixels further includes blending of the plurality of color values.

7. The method of claim 1, wherein the plurality of high resolution pixels is associated with a native resolution of the display or a resolution that is greater than the native resolution of the display.

8. The method claim 1, wherein the display is associated with a head mounted display (HMD).

9. The method of claim 1, wherein the low resolution sampling region is associated with a peripheral region of a foveated rendering view.

10. A graphics system, comprising:

a graphics processing unit (GPU) for applying temporal supersampling to a plurality of prior frames that include a low resolution sampling region, wherein the temporal supersampling obtains a plurality of color values;

a frame buffer for storing the plurality of prior frames that are rendered by the GPU; and

a display buffer in which a plurality of high resolution pixels that is reconstituted based on the plurality of color values obtained via the temporal supersampling of prior frames;

wherein the plurality of high resolution pixels is configured for presentation on a display.

11. The graphics systems of claim 10, wherein the temporal supersampling includes sampling at one location within the low resolution sampling region for each of the prior frames for the plurality of color values.

12. The graphics system of claim 10, wherein the temporal supersampling defines sampling locations for the low resolution sampling region as determined by pixel reprojection.

13. The graphics system of claim 10, wherein the temporal supersampling defines sampling locations for the low resolution sampling region as determined by jitter.

14. The graphics system of claim 10, wherein the plurality of high resolution pixels is greater in number that the number of the plurality of prior frames.

15. The graphics system of claim 10, wherein the reconstituting of the plurality of high resolution further includes blending of the plurality of color values.

16. The graphics system of claim 10, wherein the plurality of high resolution pixels is associated with a native resolution of the display or a resolution that is greater than the native resolution of the display.

17. The graphics system of claim 10, wherein the display is associated with a head mounted display (HMD).

18. The graphics system of claim 10, wherein the low resolution sampling region is associated with a peripheral region of a foveated rendering view.

19. A non-transitory computer-readable storage medium storing a computer program executable by a processor-based system, comprising:

program instructions for receiving a fragment from a rasterizer, the fragment associated with a low resolution sampling region;

program instructions for applying temporal supersampling to the fragment over a plurality of prior frames for obtaining a plurality of color values;

program instructions for reconstituting, in a buffer, a plurality of high resolution pixels associated with the low resolution sampling region, the plurality of high resolution pixels are based on the plurality of color values obtained via the temporal supersampling; and

program instructions for sending, from the buffer, the plurality of high resolution pixels for presentation on a display.

20. The non-transitory computer-readable medium of claim 19, wherein the temporal supersampling defines sampling locations for the low resolution sampling region over the plurality of prior frames as determined by pixel reprojection or as determined jitter, and wherein the plurality of high resolution pixels is associated with a native resolution of the display or a resolution that is greater than the native resolution of the display.