Denoising filter

• US 10,249,026 B2
• Filed: 07/29/2016
• Issued: 04/02/2019
• Est. Priority Date: 07/31/2015
• Status: Active Grant

• Current Assignee: Imagination Technologies Limited (Imagination Technologies Group Ltd.)
• Original Assignee: Imagination Technologies Limited (Imagination Technologies Group Ltd.)
• Inventors: Csefalvay, Szabolcs
• Primary Examiner(s): Thomas, Mia M
• Application Number:
  US15/223,642
• Publication Number:
  US 20170032500A1
• Time in Patent Office: 977 Days
• CPC Class Codes: G06T 15/06 Ray-tracing
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing

Abstract

A pixel filter receives a sequence of pixels, each pixel having an associated pixel value. A first recursive filter operation is performed in a first direction through the sequence of pixels to form a first filtered pixel value for each pixel, and a second recursive filter operation is performed in a second direction through the sequence of pixels to form a second filtered pixel value for each pixel. The first and second filtered pixel values for a given pixel are determined in dependence on the pixel value at that pixel and the filtered pixel value preceding that pixel in their respective direction of operation, the filtered pixel value of the preceding pixel being scaled by a measure of similarity between data associated with that pixel and its preceding pixel. For each pixel of the sequence, the first and second filtered pixel values are combined to generate a filter output for the pixel.

Claims

1. A pixel filter, comprising:

an input arranged to receive a sequence of pixels, each pixel having an associated pixel value;

a filter module arranged to perform a first recursive filter operation in a first direction through the sequence of pixels so as to form a first filtered pixel value for each pixel, and perform a second recursive filter operation in a second direction through the sequence of pixels so as to form a second filtered pixel value for each pixel, wherein the first and second recursive filter operations form a respective filtered pixel value for a given pixel in dependence on the pixel value at that pixel and the filtered pixel value preceding that pixel in their respective direction of operation, the filtered pixel value of the preceding pixel being scaled by a measure of similarity between data associated with that pixel and its preceding pixel; and

filter logic configured to, for each pixel of the sequence, combine the first and second filtered pixel values formed in respect of the pixel by the first and second recursive filter operations so as to generate a filter output for the pixel.

2. A pixel filter as claimed in claim 1, the second direction being opposite to the first direction.

3. A pixel filter as claimed in claim 1, the first and second recursive filter operations being configured to scale the filtered pixel value of the preceding pixel such that its contribution to the filtered pixel value is greater when the measure of similarity is indicative of high similarity than when the measure of similarity is indicative of low similarity.

4. A pixel filter as claimed in claim 1, wherein the input is further arranged to receive a further sequence of pixels, the further sequence of pixels comprising at least one pixel value from the filter output, and the filter module is further arranged to perform the first and second recursive filter operations on the further sequence of pixels.

5. A pixel filter as claimed in claim 4, wherein the sequence of pixels represents one of:

a row of pixels selected from a rectangular array of pixels, and the further sequence of pixels represents a column of pixels selected from a rectangular array of pixels; and

a column of pixels selected from a rectangular array of pixels, and the further sequence of pixels represents a row of pixels selected from a rectangular array of pixels.

6. A pixel filter as claimed in claim 1, the filter module being configured to, in forming a filtered pixel value for each pixel of the sequence, weight each pixel value by a measure of confidence in that pixel value.

7. A pixel filter as claimed in claim 6, wherein each pixel value and the measure of confidence in that pixel value represent a coordinate set, and the filter module is configured to concurrently operate on the components of each coordinate set so as to generate a filtered pixel value and filtered measure of confidence.

8. A pixel filter as claimed in claim 6, the pixel values being generated by path or ray tracing and the measure of confidence of each pixel value prior to filtering being indicative of the number of light rays which contributed to that pixel value.

9. A pixel filter as claimed in claim 1, the filter logic being configured to combine the first and second filtered pixel values for each pixel by calculating a weighted average of the first and second filtered pixel values, the weight of each filtered pixel value being formed in dependence on a measure of confidence in the pixel values which contributed to that filtered pixel value.

10. A pixel filter as claimed in claim 1, wherein the filter module comprises one of:

a recursive filter block configured to perform both the first and second recursive filter operations, such that the first recursive filter operation is performed in a first pass through the recursive filter block and the second recursive filter operation is performed in a second pass through the recursive filter block; and

a first recursive filter block and a second recursive filter block arranged to operate concurrently, and wherein the first recursive filter block is configured to perform the first recursive filter operation and the second recursive filter block is configured to perform the second recursive filter operation.

11. A pixel filter as claimed in claim 1, further comprising a preliminary filter adapted to filter the pixel values so as to form a prefiltered pixel value for each pixel, and wherein the data associated with that pixel and its preceding pixel comprises, respectively, the prefiltered pixel value at that pixel and the prefiltered pixel value at the preceding pixel with respect to the direction of operation of the respective recursive filter operation.

12. A pixel filter as claimed in claim 11, wherein the preliminary filter has a filter radius at least an order of magnitude smaller than the first and second recursive filter operations.

13. A pixel filter as claimed in claim 11, wherein the preliminary filter is adapted to form a prefiltered pixel value at a pixel by forming a weighted mean of the pixel value and a predetermined set of pixel values adjacent to that pixel, the pixel value having a higher weight than the set of adjacent pixel values.

14. A pixel filter as claimed in claim 1, wherein each recursive filter comprises at least one of:

a recursive filter having an infinite filter radius;

an exponential filter; and

a recursive filter including a preliminary filter arranged to form prefiltered pixel values for use at that recursive filter.

15. A pixel filter as claimed in claim 1, the pixels representing a raster image rendered so as to represent a three-dimensional scene and wherein the data associated with that pixel and its preceding pixel comprises one or more of:

a pixel characteristic; and

a scene characteristic of the three-dimensional scene at the points represented in the image by the pixel and preceding pixel.

16. A pixel filter as claimed in claim 15, wherein the scene characteristic comprises one or more of:

a vector describing scene geometry, a normal vector describing the orientation of a surface in the scene, and a scene depth representing a measure of distance of a surface in the scene from a defined reference point, line or plane.

17. A pixel filter as claimed in claim 16, wherein the measure of similarity comprises at least one of:

a measure of the angle between normal vectors of surfaces in the scene represented by the pixel and preceding pixel; and

a measure of the difference in scene depth between the surfaces in the scene represented by the pixel and preceding pixel.

18. A non-transitory computer readable storage medium having stored thereon computer readable instructions that, when executed at a computer system for generating a manifestation of an integrated circuit, cause the computer system to generate a manifestation of a filter as claimed in claim 1.

19. A method for filtering a sequence of pixels, each pixel having an associated pixel value, the method comprising:

recursively filtering the sequence of pixels in first and second directions so as to form first and second filtered pixel values for each pixel by, for each pixel;

reading a filtered pixel value for the preceding pixel of the sequence with respect to the direction of operation of the recursive filter;

scaling the filtered pixel value of the preceding pixel of the sequence by a measure of similarity between data associated with the pixel and the preceding pixel; and

forming a filtered pixel value for the pixel in dependence on the pixel value of the pixel and on the scaled filtered pixel value for the preceding pixel; and

combining the first and second filtered pixel values formed in respect of each pixel of each sequence so as to generate a filter output for the pixel.

20. A non-transitory computer readable storage medium having stored thereon computer readable instructions that, when executed at a processor, cause the processor to perform a method as claimed in claim 19.