Convolution filtering in a graphics processor

• US 20070292047A1
• Filed: 06/14/2006
• Published: 12/20/2007
• Est. Priority Date: 06/14/2006
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Jiao, Guofang, Du, Yun, Yu, Chun, Chen, Lingjun
• Granted Patent: US 8,644,643 B2
• Time in Patent Office: Days
• US Class Current: 382/279
• CPC Class Codes: G06F 17/153 Multidimensional correlation or convolution
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 5/20 using local operators

Abstract

Techniques for performing convolution filtering using hardware normally available in a graphics processor are described. Convolution filtering of an arbitrary H×W grid of pixels is achieved by partitioning the grid into smaller sections, performing computation for each section, and combining the intermediate results for all sections to obtain a final result. In one design, a command to perform convolution filtering on a grid of pixels with a kernel of coefficients is received, e.g., from a graphics application. The grid is partitioned into multiple sections, where each section may be 2×2 or smaller. Multiple instructions are generated for the multiple sections, with each instruction performing convolution computation on at least one pixel in one section. Each instruction may include pixel position information and applicable kernel coefficients. Instructions to combine the intermediate results from the multiple instructions are also generated.

Claims

1. An apparatus configured to receive a command to perform convolution filtering on a grid of pixels, to partition the grid into multiple sections, and to generate multiple instructions for the multiple sections, each instruction performing convolution computation on at least one pixel in one section.

2. The apparatus of claim 1 and further configured to generate instructions to combine intermediate results from the multiple instructions for the multiple sections.

3. The apparatus of claim 1, wherein each of the multiple sections is 2×

2 or smaller.

4. The apparatus of claim 1 and further configured to determine position of each pixel in the grid and to include pixel position information for the at least one pixel in each instruction.

5. The apparatus of claim 4 wherein the pixel position information comprises a reference position, at least one horizontal offset, and at least one vertical offset for the at least one pixel.

6. The apparatus of claim 1 and further configured to receive a kernel of coefficients for the grid of pixels and to include at least one coefficient from the kernel in each instruction.

7. A method comprising:

receiving a command to perform convolution filtering on a grid of pixels;

partitioning the grid into multiple sections;

generating multiple instructions for the multiple sections, each instruction performing convolution computation on at least one pixel in one section; and

generating instructions to combine intermediate results from the multiple instructions for the multiple sections.

8. The method of claim 7, further comprising:

determining position of each pixel in the grid; and

including pixel position information for the at least one pixel in each instruction.

9. The method of claim 7, further comprising:

receiving a kernel of coefficients for the grid of pixels; and

including at least one coefficient from the kernel in each instruction.

10. An apparatus comprising:

means for receiving a command to perform convolution filtering on a grid of pixels;

means for partitioning the grid into multiple sections;

means for generating multiple instructions for the multiple sections, each instruction performing convolution computation on at least one pixel in one section; and

means for generating instructions to combine intermediate results from the multiple instructions for the multiple sections.

11. The apparatus of claim 10, further comprising:

means for determining position of each pixel in the grid; and

means for including pixel position information for the at least one pixel in each instruction.

12. The apparatus of claim 10, further comprising:

means for receiving a kernel of coefficients for the grid of pixels; and

means for including at least one coefficient from the kernel in each instruction.

13. A processor readable media for storing instructions operable to:

receive a command to perform convolution filtering on a grid of pixels;

partition the grid into multiple sections;

generate multiple instructions for the multiple sections, each instruction performing convolution computation on at least one pixel in one section; and

generate instructions to combine intermediate results from the multiple instructions for the multiple sections.

14. The processor readable media of claim 13, and further for storing instructions operable to:

determine position of each pixel in the grid; and

include pixel position information for the at least one pixel in each instruction.

15. The processor readable media of claim 13, and further for storing instructions operable to:

receive a kernel of coefficients for the grid of pixels; and

include at least one coefficient from the kernel in each instruction.

16. An graphics processor comprising:

a first processing unit configured to receive a set of instructions for convolution filtering of a grid of pixels, to dispatch a plurality of instructions in the set, to receive intermediate results for the dispatched instructions, and to combine the intermediate results to generate a final result for the convolution filtering of the grid of pixels; and

a second processing unit configured to receive the instructions dispatched by the shader core, to perform computation on at least one pixel in the grid for each instruction, and to provide an intermediate result for each instruction.

17. The graphics processor of claim 16, wherein the first processing unit is a shader core and the second processing unit is a texture engine.

18. The graphics processor of claim 16, wherein each dispatched instruction covers a 2×

2 or smaller section of the grid.

19. The graphics processor of claim 16, wherein for each dispatched instruction the second processing unit is configured to retrieve the at least one pixel from memory, to multiply the at least one pixel with at least one coefficient received in the instruction, and to accumulate at least one result of the multiply to generate the intermediate result for the instruction.

20. The graphics processor of claim 19, wherein for each dispatched instruction the second processing unit is further configured to compute at least one position of the at least one pixel based on a reference position and horizontal and vertical offsets received in the instruction, and to retrieve the at least one pixel from the memory at the at least one position.

21. The graphics processor of claim 19, wherein the convolution filtering is performed with a kernel of coefficients, and wherein the at least one coefficient received in each instruction is closest to the at least one pixel among the coefficients in the kernel.

22. The graphics processor of claim 16, wherein for each dispatched instruction the second processing unit is configured to retrieve a pixel from memory, to derive an interpolated coefficient for the pixel based on coefficients received in the instruction, and to multiply the pixel with the interpolated coefficient to generate the intermediate result for the instruction.

23. The graphics processor of claim 22, wherein the convolution filtering is performed with a kernel of coefficients, and wherein the coefficients received in each instruction are closest to the pixel among the coefficients in the kernel.

24. The graphics processor of claim 22, wherein for each dispatched instruction the second processing unit is further configured to derive the interpolated coefficient based on four coefficients using bilinear interpolation.

25. The graphics processor of claim 22, wherein for each dispatched instruction the second processing unit is further configured to compute position of the pixel based on a reference position and horizontal and vertical offsets received in the instruction, and to retrieve the pixel from the memory at the computed position.

26. The graphics processor of claim 16, wherein the second processing unit comprises a cache operative to store pixels, and wherein for each dispatched instruction the second processing unit is configured to retrieve the at least one pixel from the cache and to perform a cache fill if the at least one pixel is not located in the cache.

27. The graphics processor of claim 16, wherein the first processing unit comprises a buffer operative to store a kernel of coefficients used for convolution filtering.

28. The graphics processor of claim 27, wherein the buffer is further operative to store horizontal and vertical offsets for each of the pixels in the grid.

29. The graphics processor of claim 16, wherein each dispatched instruction comprises a reference position, up to two horizontal offsets and up to two vertical offsets for up to four pixels, up to four coefficients, and a mask identifying the up to four pixels in the instruction.

30. The graphics processor of claim 16, wherein each dispatched instruction comprises a reference position, a horizontal offset and a vertical offset for a pixel, and four coefficients for the pixel.

31. A method comprising:

receiving a set of instructions for convolution filtering of a grid of pixels;

dispatching a plurality of instructions in the set;

performing computation on at least one pixel in the grid for each dispatched instruction to obtain an intermediate result for the dispatched instruction; and

combining intermediate results for the plurality of dispatched instructions to generate a final result.

32. The method of claim 31, wherein the performing computation on the at least one pixel in the grid for each dispatched instruction comprises

retrieving the at least one pixel from memory,

multiplying the at least one pixel with at least one coefficient received in the instruction, and

accumulating at least one result of the multiply to generate the intermediate result for the instruction.

33. The method of claim 31, wherein the performing computation on the at least one pixel in the grid for each dispatched instruction comprises

retrieving a pixel from memory,

deriving an interpolated coefficient for the pixel based on coefficients received in the instruction, and

multiplying the pixel with the interpolated coefficient to generate the intermediate result for the instruction.

34. An apparatus configured to receive a command to perform convolution filtering on a grid of pixels with a kernel of coefficients in accordance with one of multiple modes comprising a first mode and a second mode, to multiply the pixels in the grids with closest coefficients in the kernel if the first mode is selected, and to multiply the pixels in the grids with interpolated coefficients derived from the coefficients in the kernel if the second mode is selected.

35. The apparatus of claim 34 and configured to multiply the pixels in the grids in accordance with a set of instructions, each instruction covering up to four pixels in the grid for the first mode.

36. The apparatus of claim 34 and configured to multiply the pixels in the grids in accordance with a set of instructions, each instruction covering a single pixel in the grid for the second mode.

37. A method comprising:

receiving a command to perform convolution filtering on a grid of pixels with a kernel of coefficients in accordance with one of multiple modes comprising a first mode and a second mode;

multiplying the pixels in the grids with closest coefficients in the kernel if the first mode is selected; and

multiplying the pixels in the grids with interpolated coefficients derived from the coefficients in the kernel if the second mode is selected.