Interpolation filter support for sub-pixel resolution in video coding

• US 8,705,622 B2
• Filed: 04/08/2009
• Issued: 04/22/2014
• Est. Priority Date: 04/10/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Ye, Yan, Karczewicz, Marta
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Xu, Xiaolan
• Application Number:
  US12/420,270
• Publication Number:
  US 20090257493A1
• Time in Patent Office: 1,840 Days
• Field of Search: 375/240
• US Class Current: 375/240.12
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/192 the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/463 by compressing encoding parameters before transmission
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51

Abstract

This disclosure describes filtering techniques applied by an encoder and a decoder during the prediction stage of a video encoding and/or decoding process. The filtering techniques may enhance the accuracy of predictive data used during fractional interpolation, and may improve predictive data of integer blocks of pixels. There are several aspects to this disclosure, including a useful twelve-pixel filter support that may be used for interpolation, techniques that use coefficient symmetry and pixel symmetry to reduce the amount of data needed to be sent between an encoder and a decoder to configure the filter support for interpolation, and techniques for filtering data at integer pixel locations in a manner that is similar to sub-pixel interpolation. Other aspects of this disclosure concern techniques for encoding information in the bitstream to convey the type of filter used, and possibly the filter coefficients used. Predictive coding of filter coefficients is also described.

Claims

1. A method comprising:

obtaining a block of pixels, wherein the block of pixels includes integer pixel values corresponding to integer pixel positions within the block of pixels;

computing sub-pixel values for sub-pixel positions associated with the block of pixels based on the integer pixel values, wherein computing the sub-pixel values comprises applying an interpolation filter that defines a two-dimensional array of filter support positions corresponding to a set of twelve or more integer pixel positions that surround the sub-pixel positions in a radial shape, and wherein the interpolation filter is used during an inter-prediction video coding process; and

generating a prediction block based on at least some of the sub-pixel values.

2. The method of claim 1, wherein the sub-pixel values comprise fifteen half-pel and quarter-pel values, wherein the interpolation filter that defines the two-dimensional array of filter support positions is applied to define nine of the fifteen half-pel and quarter-pel values.

3. The method of claim 2, wherein computing the sub-pixel values further comprises applying linear interpolation filters that define one-dimensional arrays of filter support positions for six of the fifteen half-pel and quarter-pel values.

4. The method of claim 2, wherein the set of twelve or more integer pixel positions that surround the sub-pixel positions are a set of spatially closest integer pixel positions to the nine half-pel and quarter-pel values defined by the interpolation filter.

5. The method of claim 4, wherein the set of twelve or more integer pixel positions that surround the sub-pixel positions include four integer-pixel positions surrounding the nine half-pel and quarter-pel values defined by the interpolation filter and eight integer-pixel positions surrounding the four integer-pixel positions.

6. The method of claim 1, wherein each of the filter support positions in the two-dimensional array of filter support positions is located within a distance of two integer-pixel positions relative to the sub-pixel positions.

7. The method of claim 1, wherein the method forms part of a video encoding process or a video decoding process.

8. The method of claim 1, wherein the method forms part of a video encoding process, the method further comprising:

encoding a plurality of sets of coefficient values of the interpolation filter; and

transmitting the coefficient values of the interpolation filter as part of an encoded bitstream.

9. The method of claim 8, wherein each of the plurality of sets of coefficient values define different weightings for the twelve or more integer pixel positions.

10. The method of claim 9, wherein pixel symmetry exists such that M sets of coefficient values define N different weightings for the twelve or more integer pixel positions, wherein M and N are positive integers and N is greater than M.

11. An apparatus comprising a video coder that:

obtains a block of pixels, wherein the block of pixels includes integer pixel values corresponding to integer pixel positions within the block of pixels;

computes sub-pixel values for sub-pixel positions associated with the block of pixels based on the integer pixel values, wherein the video coder applies an interpolation filter that defines a two-dimensional array of filter support positions corresponding to a set of twelve or more integer pixel positions that surround the sub-pixel positions in a radial shape, and wherein the interpolation filter is used during an inter-prediction video coding process; and

generates a prediction block based on at least some of the sub-pixel values.

12. The apparatus of claim 11, wherein the sub-pixel values comprise fifteen half-pel and quarter-pel values, wherein the interpolation filter that defines the two-dimensional array of filter support positions is applied to define nine of the fifteen half-pel and quarter-pel values.

13. The apparatus of claim 12, wherein the video coder applies linear interpolation filters that define one-dimensional arrays of filter support positions for six of the fifteen half-pel and quarter-pel values.

14. The apparatus of claim 12, wherein the set of twelve or more integer pixel positions that surround the sub-pixel positions are a set of spatially closest integer pixel positions to the nine half-pel and quarter-pel values defined by the interpolation filter.

15. The apparatus of claim 14, wherein the set of twelve or more integer pixel positions that surround the sub-pixel positions include four integer-pixel positions surrounding the nine half-pel and quarter-pel values defined by the interpolation filter and eight integer-pixel positions surrounding the four integer-pixel positions.

16. The apparatus of claim 11, wherein each of the filter support positions in the two-dimensional array of filter support positions is located within a distance of two integer-pixel positions relative to the sub-pixel positions.

17. The apparatus of claim 11, wherein the video coder comprises a video encoding-device or a video decoding device.

18. The apparatus of claim 11, wherein the apparatus further comprises a transmitter, wherein:

the video coder encodes a plurality of sets of coefficient values of the interpolation filter; and

the transmitter transmits the coefficient values of the interpolation filter as part of an encoded bitstream.

19. The apparatus of claim 18, wherein each of the plurality of sets of coefficient values define different weightings for the twelve or more integer pixel positions.

20. The apparatus of claim 19, wherein pixel symmetry exists such that M sets of coefficient values define N different weightings for the twelve or more integer pixel positions, wherein M and N are positive integers and N is greater than M.

21. The apparatus of claim 11, wherein the video coder comprises an integrated circuit.

22. The apparatus of claim 11, wherein the video coder comprises a microprocessor.

23. The apparatus of claim 11, wherein the apparatus comprises a wireless communication device that includes the video coder.

24. A device comprising:

means for obtaining a block of pixels, wherein the block of pixels includes integer pixel values corresponding to integer pixel positions within the block of pixels;

means for computing sub-pixel values for sub-pixel positions associated with the block of pixels based on the integer pixel values, wherein means for computing the sub-pixel values comprises means for applying an interpolation filter that defines a two-dimensional array of filter support positions corresponding to a set of twelve or more integer pixel positions that surround the sub-pixel positions in a radial shape, and wherein the interpolation filter is used during an inter-prediction video coding process; and

means for generating a prediction block based on at least some of the sub-pixel values.

25. The device of claim 24, wherein the sub-pixel values comprise fifteen half-pel and quarter-pel values, wherein the interpolation filter that defines the two-dimensional array of filter support positions is applied to define nine of the fifteen half-pel and quarter-pel values.

26. The device of claim 25, wherein means for computing the sub-pixel values further comprises means for applying linear interpolation filters that define one-dimensional arrays of filter support positions for six of the fifteen half-pel and quarter-pel values.

27. The device of claim 25, wherein the set of twelve or more integer pixel positions that surround the sub-pixel positions are a set of spatially closest integer pixel positions to the nine half-pel and quarter-pel values defined by the interpolation filter.

28. The device of claim 27, wherein the set of twelve or more integer pixel positions that surround the sub-pixel positions include four integer-pixel positions surrounding the nine half-pel and quarter-pel values defined by the interpolation filter and eight integer-pixel positions surrounding the four integer-pixel positions.

29. The device of claim 24, wherein each of the filter support positions in the two-dimensional array of filter support positions is located within a distance of two integer-pixel positions relative to the sub-pixel positions.

30. The device of claim 24, wherein the device comprises a video encoding-device or a video decoding-device.

31. The device of claim 24, wherein the device executes a video encoding process, the device further comprising:

means for encoding a plurality of sets of coefficient values of the interpolation filter; and

means for transmitting the coefficient values of the interpolation filter as part of an encoded bitstream.

32. The device of claim 31, wherein each of the plurality of sets of coefficient values define different weightings for the twelve or more integer pixel positions.

33. The method of claim 32, wherein pixel symmetry exists such that M sets of coefficient values define N different weightings for the twelve or more integer pixel positions, wherein M and N are positive integers and N is greater than M.

34. A non-transitory computer-readable storage medium comprising instructions that when executed by a processor cause the processor to:

obtain a block of pixels, wherein the block of pixels includes integer pixel values corresponding to integer pixel positions within the block of pixels;

compute sub-pixel values for sub-pixel positions associated with the block of pixels based on the integer pixel values, wherein in computing the sub-pixel values the instructions cause the processor to apply an interpolation filter that defines a two-dimensional array of filter support positions corresponding to a set of twelve or more integer pixel positions that surround the sub-pixel positions in a radial shape, and wherein the interpolation filter is used during an inter-prediction video coding process; and

generate a prediction block based on at least some of the sub-pixel values.

35. The non-transitory computer-readable storage medium of claim 34, wherein the sub-pixel values comprise fifteen half-pel and quarter-pel values, wherein the interpolation filter that defines the two-dimensional array of filter support positions is applied to define nine of the fifteen half-pel and quarter-pel values.

36. The non-transitory computer-readable storage medium of claim 35, wherein in computing the sub-pixel values, the instructions cause the processor to apply linear interpolation filters that define one-dimensional arrays of filter support positions for six of the fifteen half-pel and quarter-pel values.

37. The non-transitory computer-readable storage medium of claim 35, wherein the set of twelve or more integer pixel positions that surround the sub-pixel positions are a set of spatially closest integer pixel positions to the nine half-pel and quarter-pel values defined by the interpolation filter.

38. The non-transitory computer-readable storage medium of claim 37, wherein the set of twelve or more integer pixel positions that surround the sub-pixel positions include four integer-pixel positions surrounding the nine half-pel and quarter-pel values defined by the interpolation filter and eight integer-pixel positions surrounding the four integer-pixel positions.

39. The non-transitory computer-readable storage medium of claim 34, wherein each of the filter support positions in the two-dimensional array of filter support positions is located within a distance of two integer-pixel positions relative to the sub-pixel positions.

40. The non-transitory computer-readable storage medium of claim 34, wherein the instructions forms part of a video encoding software program or a video decoding software program.

41. The non-transitory computer-readable storage medium of claim 34, wherein the instructions forms part of a video encoding software program, wherein the instructions cause the processor to:

encode a plurality of sets of coefficient values of the interpolation filter; and

output the coefficient values of the interpolation filter as part of an encoded bitstream.

42. The non-transitory computer-readable storage medium of claim 41, wherein each of the plurality of sets of coefficient values define different weightings for the twelve or more integer pixel positions.

43. The non-transitory computer-readable storage medium of claim 41, wherein pixel symmetry exists such that M sets of coefficient values define N different weightings for the twelve or more integer pixel positions, wherein M and N are positive integers and N is greater than M.