MOTION BASED ADAPTIVE RENDERING

• US 20150379727A1
• Filed: 06/18/2015
• Published: 12/31/2015
• Est. Priority Date: 06/27/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: GOLAS, Abhinav, RAMANI, Karthik, BROTHERS, John W., ABRAHAM, Santosh, CHENG, Christopher T., ZHANG, Liangjun, CHOW, Ki Fung
• Granted Patent: US 9,928,610 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/40 Filling a planar surface by adding surface attributes, e.g. colour or texture
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2200/12 involving antialiasing
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G09G 5/363 Graphics controllers

Abstract

An apparatus, system and method is provided to determine a motion of pixels in local regions of a scene, classify the motion into a speed category, and make decisions on how to render blocks of pixels. In one implementation the motion in a tile is classified into at least three different speed regimes. If the pixels in a tile are in a quasi-static speed regime, a determination is made whether or not to reuse a fraction of pixels from the previous frame. If the pixels are determined to be in a high speed regime, a decision is made whether or not a sampling rate may be reduced.

Claims

1. A method of graphics processing, comprising:

determining, on a tile-by-tile basis, a relative speed of objects as measured in screen space co-ordinates or pixels in a current frame relative to a previous frame;

classifying each tile within the frame into one of at least three speed categories, the at least three categories including a quasi-static speed category, a medium speed category, and a high-speed category;

making a sampling decision for each tile, based at least in part on the speed category associated with each tile, including whether the tile is to be sampled at a full resolution sampling rate of at least one sample per pixel in the current frame or sampled at a lower rate in the current frame; and

rendering tiles of the current frame.

2. The method of claim 1, wherein for a tile classified in the quasi-static speed category the method further comprises reusing pixel data from a previous frame by mapping a location of at least one pixel in the current frame to a corresponding pixel in a previous frame and copying pixel data for the at least one pixel of the previous frame to the current frame.

3. The method of claim 2, wherein a subset of the pixels from a tile of a previous frame are copied to a tile of the current frame based on a pre-selected ratio of rendered to unrendered pixels.

4. The method of claim 2, further comprising performing a discrepancy check with respect to pixel color values of the previous frame as a condition for reusing pixel data from the previous frame.

5. The method of claim 1, wherein the speed is determined from a difference in vertex position data for rendered geometry between the current frame and a previous frame.

6. The method of claim 1, wherein for a tile classified in the at least one high speed regime making a sampling decision includes selecting a sampling pattern in which a number of rendered samples is less than the number of actual pixels associated with the tile.

7. The method of claim 1, further comprising performing edge detection in depth/Z and wherein the making a sampling decision for each tile is further based on whether edges are detected within individual tiles.

8. The method of claim 7, wherein:

for a tile in the at least one high speed regime not including an edge, selecting a sampling pattern in which a number of samples is less than the number of pixels associated with the tile; and

for a tile in the at least one high speed regime including an edge, selecting a sampling pattern in which a number of samples is at least equal to one sample per pixel.

9. The method of claim 1, wherein making a sampling decision further comprises making a decision for each tile whether or not to perform super-sampling.

10. The method of claim 9, wherein super-sampling is performed for a tile including an edge.

11. A method of graphics processing, comprising:

determining, on a tile-by-tile basis, a speed of objects as measured in screen space pixels in a current frame relative to a previous frame;

detecting, on a tile-by-tile basis, whether a tile includes an edge;

classifying each tile within the frame into one of at least three speed categories including a quasi-static speed category, an intermediate speed regime, and at least one high speed regime;

making a sampling rate decision for individual tiles based on the speed category associated with a tile and whether the tile includes an edge, the sampling decision including determining whether the tile is to be sampled at a full resolution sampling rate of at least one sample per pixel in the current frame or sampled at a lower rate in the current frame; and

rendering tiles of the current frame.

12. The method of claim 11, wherein for a tile classified in the quasi-static speed category the method further comprises reusing pixel data from a previous frame by mapping a location of at least one pixel in a current frame to a corresponding pixel in a previous frame and copying pixel data for the at least one pixel of the previous frame to the current frame.

13. The method of claim 12, wherein a subset of the pixels from a tile of a previous frame are copied to a tile of the current frame based on a pre-selected ratio of rendered to unrendered pixels.

14. The method of claim 12, further comprising performing a discrepancy check with respect to pixel color values of the previous frame as a condition for reusing pixel data from the previous frame.

15. The method of claim 11, wherein for a tile classified in the intermediate speed regime, every pixel is sampled at least once.

16. The method of claim 11, wherein for a tile classified in the at least one high speed regime selecting a sampling pattern in which a number of samples is less than the number of pixels associated with the tile and performing interpolation to determine a color at pixel locations not sampled.

17. The method of claim 11, wherein:

for a tile in the at least one high speed regime not including an edge, selecting a sampling pattern in which a number of samples is less than the number of pixels associated with the tile; and

for a tile in the at least one high speed regime including an edge, selecting a sampling pattern in which a number of samples is at least equal to one sample per pixel.

18. A graphics system including a graphics pipeline, comprising:

an adaptive sampling generator; and

a pixel shader coupled to the adaptive sampling generator;

the adaptive sampling generator configured to determine a required sample rate for each tile based at least in part on a speed of pixels of objects in each tile and selecting a sample pattern based on the required sample rate.

19. The graphics system of claim 18, wherein the adaptive sampling generator determines a sample pattern and a sample rate for an individual tile based on a combination of the speed of objects in the individual tile and whether or not the individual tile includes an edge.

20. The graphics system of claim 19 further comprising:

a velocity buffer to compare per-vertex coordinates of vertices of objects in a current frame to corresponding per-vertex coordinates in a previous frame and determine a per-pixel velocity of pixels in each tile;

an edge detector to detect a presence of edges in each tile; and

a sample generator to determine, for each tile to be shaded, a sample pattern and sampling rate based on a speed associated with pixels in each tile and the presence of edges.

21. The graphics system of claim 19, wherein a reduced sampling rate is selected for each tile not including an edge and having a speed above a threshold speed.

22. The graphics system of claim 19, further comprising a advection unit, wherein for a tile having a velocity below a quasi-static speed limit, a sample pattern having a reduced sampling rate is selected and the reconstruction unit fills in missing pixel data by reusing pixel data from a previous frame via advection.

23. The graphics system of claim 22, wherein the reconstruction unit identifies a location of a pixel in a previous frame by mapping a change in pixel coordinates.

24. The graphics system of claim 22, wherein for the tile having a velocity below the quasi-static speed limit a preselected subset of the pixels of the tile are rendered and the remainder of the pixels are reused from the previous frame based on a pre-selected ratio of rendered to unrendered pixels.

25. A method of graphics processing, comprising:

adaptively rendering at least one tile of a current frame based on speed determined by comparing the current frame to a previous frame.

26. The method of claim 25 wherein the adaptively rendering at least one tile of a current frame based on speed further comprises:

determining for the at least one tile of the current frame a relative speed of objects as measured in screen space co-ordinates or pixels in the current frame relative to a previous frame;

classifying each tile within the current frame into one of at least three speed categories, the at least three categories including a quasi-static speed category, a medium speed category, and a high-speed category;

making a sampling decision for each tile, based at least in part on the speed category associated with each tile, including whether the tile is to be sampled at a full resolution sampling rate of at least one sample per pixel in the current frame or sampled at a lower rate in the current frame; and

rendering tiles of the current frame.