Dithered sampling patterns for temporal color averaging

• US 9,589,366 B2
• Filed: 06/18/2015
• Issued: 03/07/2017
• Est. Priority Date: 06/27/2014
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Golas, Abhinav, Ramani, Karthik, Brothers, John W.
• Primary Examiner(s): Sajous, Wesner
• Application Number:
  US14/743,728
• Publication Number:
  US 20150379734A1
• Time in Patent Office: 628 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/40 Filling a planar surface by adding surface attributes, e.g. colour or texture
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2200/12 involving antialiasing
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G09G 5/363 Graphics controllers

Abstract

Graphics processing is performed in which a decision is made in individual tiles whether or not to sample at a reduced sampling rate. A sampling pattern is selected from a set of sampling patterns having the same reduced sampling rate. The sampling pattern is dithered over a set of frames to reduce the visual appearance of visual artifacts via temporal color averaging.

Claims

1. A method of graphics processing, comprising:

selecting, on a frame-by-frame basis for a sequence of frames, a selection of a set of tiles in which the tiles are sparsely sampled at an average sample rate of less than one sample per pixel;

selecting, by a graphics processing unit, in each frame, a sampling pattern for each sparsely sampled tile selected from a set of different sampling patterns having the same average sampling rate; and

varying, by the graphics processing unit, the sampling pattern in individual tiles over a sequence of frames to temporally color average sampling artifacts.

2. The method of claim 1, wherein the sampling pattern is varied at a rate above a threshold for human perception to discern individual images.

3. The method of claim 1, further comprising selecting, on a frame-by-frame based for a sequence of frames, full sampling for a second set of tiles in which the tiles are sampled at a rate at least equal to one sample per pixel.

4. The method of claim 3, further comprising:

analyzing a scene to determine a minimum number of samples per tile in each region of an image and selecting a sample rate and a sample pattern for each tile.

5. The method of claim 4, wherein a sampling pattern is determined for each tile prior to pixel shading.

6. The method of claim 1, wherein in a particular frame, the same sampling pattern is utilized for all sparsely sampled tiles having the same sample rate.

7. The method of claim 1, wherein the sampling pattern is scheduled by a counter to ensure that the refresh rate is above a threshold human perception to discern individual images.

8. The method of claim 1, wherein the sampling pattern is varied over a sequence of frames in a schedule selected to ensure that every pixel is sampled at least once over a set of frames.

9. The method of claim 1, wherein the sampling pattern is stochastically varied over a sequence of frames.

10. A method of graphics processing, comprising:

determining by a graphics processing unit, in each frame, a required minimum sample rate for each tile of a frame and selecting a sample rate for each tile in the frame;

selecting by the graphics processing unit, in each frame, a sampling pattern for each tile that is sparsely sampled at a selected sampling rate of less than one sample per pixel, the sampling pattern being selected from a set of different sampling patterns having the same selected sampling rate but a different arrangement of sample locations; and

varying by the graphics processing unit the sampling pattern in individual tile locations over a sequence of frames to temporally color average sampling artifacts associated with sparse sampling.

11. The method of claim 10, wherein the sampling pattern is varied at a rate above a threshold for human perception to discern individual images.

12. The method of claim 11, wherein the sampling pattern is varied at a rate at least equal to 15 frames per second.

13. The method of claim 10, wherein in a particular frame, the same sampling pattern is utilized for all sparsely sampled tiles having the same sample rate.

14. The method of claim 10, wherein the sampling pattern is scheduled by a counter to ensure that the refresh rate is above a threshold human perception to discern individual images.

15. The method of claim 10, wherein the sampling pattern is varied over a sequence of frames in a schedule selected to ensure that every pixel is sampled at least once over a set of frames.

16. The method of claim 10, wherein the sampling pattern is stochastically varied over a sequence of frames.

17. A graphics system including a graphics pipeline, comprising:

an adaptive sample generator configured to adapt a sampling rate of individual tiles based on a required sampling rate; and

a dithering unit, the dithering unit varying a sampling pattern utilized by the adaptive sample generator to sample tiles at an average sample rate of less than one sample per pixel.

18. The graphics system of claim 17, wherein the sampling pattern is selected from a set of sample patterns having a different arrangement of sample locations and the sampling pattern is scheduled by a counter to ensure that the refresh rate is above a threshold human perception to discern individual images.

19. The graphics system of claim 17, wherein the sampling pattern is varied over a sequence of frames in a schedule selected to ensure that every pixel is sampled at least once over a set of frames.

20. The graphics system of claim 17, wherein the adaptive sample generator determines the required sampling rate in each tile based on a motion vector of pixels in the tile.

21. The graphics system of claim 20, wherein the adaptive sample generator determines the required sampling rate in each tile based on the combination of a motion vector of pixels in the tile and the presence of edges.

22. The graphics system of claim 20, wherein the sampling pattern is dithered, interpolation is used to reconstruct missing pixel data, and the dithering reduces a visual appearance of artifacts associated with interpolation.

23. The graphics system of claim 20, wherein the sampling pattern is dithered, advection is performed to use pixel data from a previous frame to generate pixel data for missing pixel values, and the dithering reduces a visual appearance of artifacts associated with advection.