RECONSTRUCTION OF MISSING DATA POINT FROM SPARSE SAMPLES DURING GRAPHICS PROCESSING USING CUBIC SPLINE POLYNOMIALS

• US 20150379692A1
• Filed: 06/18/2015
• Published: 12/31/2015
• Est. Priority Date: 06/27/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: GOLAS, Abhinav, RAMANI, Karthik, BROTHERS, John W.
• Granted Patent: US 9,589,367 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/40 Filling a planar surface by adding surface attributes, e.g. colour or texture
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2200/12 involving antialiasing
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G09G 5/363 Graphics controllers

Abstract

A graphics system includes a reconstruction unit that utilizes higher order polynomials, such as cubic splines, to reconstruct missing pixel data. The computational work to perform interpolation with higher order polynomials, such as cubic splines, is reduced by pre-calculating weights for each sparse sample pattern. The pre-calculated weights may be stored as stencils and used during runtime to perform interpolation.

Claims

1. A method of performing graphics processing, comprising:

selecting at least one tile of a graphics image to be sampled;

selecting a sample pattern to sample the at least one tile of a graphics image;

rendering pixel data of the sampled at least one tile; and

performing, at runtime, reconstruction of missing pixel data in the sampled at least one tile using cubic spline interpolation.

2. The method of claim 1, wherein the cubic spline interpolation is performed using pre-computed weights to compute pixel color values as a weighted sum of known pixel color values using cubic splines for reconstruction.

3. The method of claim 2, wherein the method further comprises accumulating products of the pre-computed weights and rendered color values with a multiplier/adder and normalizing the accumulated product.

4. The method of claim 2, further comprising storing the pre-computed weights of the sample pattern in a set of pre-computed stencils.

5. The method of claim 4, wherein the set of pre-computed stencils includes all of the pre-computed weights for each location of missing pixel data in the sample pattern.

6. The method of claim 4, further comprising providing an index into the pre-computed stencil.

7. The method of claim 1, wherein the sample pattern is selected from a set of pre-defined sample patterns having the same sample rate and each pre-defined sample pattern has an associated set of pre-computed weights to perform cubic spline interpolation for the selected sample pattern.

8. The method of claim 1, wherein the selecting at least one tile of the graphics image to be sampled comprises analyzing a scene and determining required sample rates in different tiles of the image and selecting tiles requiring a lower sampling rate to be sparsely sampled.

9. A method of performing graphics processing, comprising:

providing a set of pre-computed weights to perform cubic spline interpolation of missing pixel data;

selecting at least one tile of a graphics image to be sampled;

selecting the sample pattern to sample the at least one tile of the graphics image; and

rendering pixel data of the sampled at least one tile;

performing, at runtime, reconstruction of missing pixel data in the sampled at least one tile using the pre-computed weights to perform cubic spline interpolation of pixel color values as a weighted sum of known pixel color values.

10. The method of claim 9, wherein the performing comprises accumulating products of the pre-computed weights and samples color values with a multiplier/adder and normalizing the accumulated product.

11. The method of claim 9, wherein the pre-computed weights are stored in a set of pre-computed stencils including all of the pre-computed weights for each pixel location of the sample pattern.

12. The method of claim 11, further comprising providing an index into the set of pre-computed stencils.

13. The method of claim 9, wherein the sample pattern is selected from a set of pre-defined sample patterns having the same sample rate and further comprising selecting a pre-computed stencil having pre-computed weights stored for the selected sample pattern

14. A graphics system including a graphics pipeline, comprising:

a memory storing a set of pre-computed weights to compute pixel color values as a weighted sum of known pixel color values using cubic spline interpolation for reconstruction; and

a reconstruction submodule configured to access the pre-computed weights from the memory and perform, at runtime, reconstruction of missing pixel data in sampled tiles using cubic spline interpolation.

15. The graphics system of claim 14 including a multiplier/adder, wherein the reconstruction unit accumulates products of the pre-computed weights and samples color values with the multiplier/adder and normalizing the accumulated product.

16. The graphics system of claim 14, wherein the reconstruction is performed on a tile basis.

17. The graphics system of claim 16, wherein the pre-computed weights are stored as a set of stencils.

18. The graphics system of claim 17, wherein the set of pre-computed stencils includes all of the pre-computed weights for each pixel location of the sample pattern.

19. The graphics system of claim 14, wherein the graphics system includes a set of sampling patterns having a reduced average sampling rate and each respective sampling pattern has a corresponding set of pre-computed stencils including the pre-computed weights to perform cubic spline interpolation.

20. A method of performing graphics processing, comprising:

providing a set of pre-computed weights to perform higher order polynomial interpolation of missing pixel data, wherein the order of the polynomial is at least three;

selecting at least one tile of a graphics image to be sampled;

selecting the sample pattern to sample the at least one tile of the graphics image;

rendering pixel data of the sampled at least one tile; and

performing, at runtime, reconstruction of missing pixel data in the sampled at least one tile using the pre-computed weights to perform higher order polynomial interpolation of pixel color values as a weighted sum of known pixel color values.