Graphics processing with advection to reconstruct missing sample data points

• US 9,830,714 B2
• Filed: 06/18/2015
• Issued: 11/28/2017
• Est. Priority Date: 06/27/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Golas, Abhinav, Ramani, Karthik, Brothers, John W.
• Primary Examiner(s): Saini, Amandeep
• Application Number:
  US14/743,716
• Publication Number:
  US 20150379674A1
• Time in Patent Office: 894 Days
• Field of Search: 382100
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/40 Filling a planar surface by adding surface attributes, e.g. colour or texture
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2200/12 involving antialiasing
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G09G 5/363 Graphics controllers

Abstract

In a graphics processing system pixel data and vertex coordinate information from a previous frame is buffered and provided to the current frame. A decision is made in the current frame whether pixel data from the previous frame may be reused. In one implementation if the speed of pixels in a tile is less than a quasi-static speed threshold a decision is made whether or not to reuse a fraction of pixels from the previous frame.

Claims

1. A method of performing graphics processing in a graphics pipeline that is implemented using a graphics processing unit (GPU) including graphics hardware via advection, the method comprising:

buffering pixel data and vertex coordinate data generated for a previous frame n, into a buffer of a graphics pipeline processing a current frame n+1, where n is a positive integer;

updating a velocity of each pixel of a plurality of pixels of the current frame n+1;

determining, for a set of pixels of the plurality of pixels of the current frame n+1, corresponding positions in the previous frame n; and

selecting, in at least one tile, color data from at least one pixel of the previous frame n for reuse in the current frame n+1 by utilizing a tile sampling pattern to determine in each tile a selection of pixel locations shaded in the current frame n+1 and a selection of pixel locations in which pixel data from the previous frame n is reused.

2. The method of claim 1, wherein said updating the velocity of each pixel of the plurality of pixels of the current frame n+1 comprises performing a splatting operation during a Z-pass operation to determine corresponding per-pixel motion vectors.

3. The method of claim 1, further comprising:

shading a first subset of the pixels in a tile in the current frame n+1; and

reusing a second subset of the pixels from the previous frame n.

4. The method of claim 1, further comprising:

performing a speed test in each tile; and

performing said selecting in individual tiles when a speed associated with a motion vector is less than a threshold speed.

5. The method of claim 1, further comprising performing a discrepancy check in individual tiles comprising the at least one tile, wherein color data is not selected for corresponding ones of the individual tiles when the discrepancy check fails.

6. The method of claim 5, wherein the discrepancy check is performed for individual tiles that have advected pixels in the current frame n+1, and

wherein the discrepancy check determines whether to perform advection in a next frame n+2.

7. The method of claim 1, further comprising mapping by determining motion vectors for each pixel of the plurality of pixels by subtracting vertex coordinates across successive frames.

8. A method of performing graphics processing in a graphics pipeline that is implemented using a graphics processing unit (GPU) including graphics hardware, the method comprising:

buffering pixel data and vertex coordinate data generated for a first frame in a buffer of a graphics pipeline processing a second frame;

determining, for of a set of pixels in the second frame, corresponding positions in the first frame based on a translation associated with changes in vertex coordinates between the first frame and the second frame;

selecting color data from at least one pixel of the first frame for reuse in the second frame;

performing a discrepancy check for tiles that have advected pixels in the second frame;

disabling the selecting the color data from corresponding pixels if the discrepancy check fails; and

determining whether to perform advection in a third frame based on the discrepancy check.

9. The method of claim 8, further comprising performing stereoscopic rendering, wherein the first frame is a first eye image, and wherein the second frame is a second eye image.

10. The method of claim 8, further comprising performing non-stereoscopic rendering, wherein the first frame is a previous frame, and wherein the second frame is a current frame.

11. The method of claim 8, wherein the determining and the selecting are performed on a tile basis.

12. A graphics system comprising:

a graphics pipeline that is implemented using a graphics processing unit (GPU) including graphics hardware; and

an advection unit configured to;

buffer pixel data and vertex coordinate data generated for a first frame into a buffer of the graphics pipeline processing a second frame;

determine, for of a set of pixels in the second frame, corresponding positions in the first frame based on a translation associated with changes in vertex coordinates between the first frame and the second frame;

select color data from at least one pixel of the first frame for reuse in the second frame;

determine pixel locations on a tile-by-tile basis;

select color data on a tile-by-tile basis; and

utilize a tile sampling pattern to determine in each tile a selection of pixel locations shaded in the second frame and a selection of pixel locations in which pixel data from the first frame is reused.

13. The graphics system of claim 12, wherein a first half of the pixels in a tile are shaded in the second frame, and

wherein a second half of the pixels are reused from the first frame.

14. The graphics system of claim 12, wherein the advection unit is further configured to:

perform a speed test of objects on a tile-by-tile basis; and

not reuse pixel data from the first frame in a tile having a speed exceeding a threshold speed.

15. The graphics system of claim 14, wherein the advection unit is further configured to perform a discrepancy check as a condition for reusing pixel data of the first frame.

16. The graphics system of claim 12, wherein the advection unit is further configured to perform stereoscopic rendering,

wherein the first frame is a first eye image, and

wherein the second frame is a second eye image.

17. The graphics system of claim 12, wherein the advection unit is further configured to perform non-stereoscopic rendering,

wherein the first frame is a previous frame, and

wherein the second frame is a current frame.