OFFSET CALCULATION IN SWITCHED INTERPOLATION FILTERS

• US 20100074332A1
• Filed: 07/24/2009
• Published: 03/25/2010
• Est. Priority Date: 09/23/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Karczewicz, Marta, Ye, Yan, Chen, Peisong, Motta, Giovanni
• Granted Patent: US 8,750,378 B2
• Time in Patent Office: Days
• US Class Current: 375/240.120
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

This disclosure describes techniques for adding offset to predictive video blocks during video coding. In one example, a method of encoding a video block includes interpolating a first block of predictive values based on a first reference video unit within a first list of reference data, and a second block of predictive values based on a second reference video unit within a second list of reference data, calculating, for sub-integer pixel positions, a first offset value based on the first block and the current video block, and a second offset value based on the first offset value and the second block, determining a final block of offset values based on the first block of predictive values, the second block of predictive values, the first offset values, and the second offset values, and encoding the current video block based on the final block of offset values.

Claims

1. A video encoding method for encoding a current video block, the method comprising:

interpolating, via an encoder, a first block of predictive values associated with a sub-integer pixel position based on a first reference video unit within a first list of reference data, and a second block of predictive values associated with the sub-integer pixel position based on a second reference video unit within a second list of reference data;

calculating, via an encoder, for sub-integer pixel positions, a first offset value based on the first block of predictive values and the current video block, and a second offset value based on the first offset value and the second block of predictive values;

determining, via an encoder, a final block of offset values based on the first block of predictive values, the second block of predictive values, the first offset values, and the second offset values; and

encoding, via an encoder, the current video block based on the final block of offset values.

2. The method of claim 1, wherein interpolating the first block of predictive values and the second block of predictive values is performed by fixed interpolation filters.

3. The method of claim 1, wherein determining a final block of offset values comprises:

applying weights to at least one of the first offset value and the second offset value.

4. The method of claim 1, wherein determining a final block of offset values comprises:

adding together the first block of predictive values, the second block of predictive values, the first offset values, and the second offset values to produce an offset predictive sum;

dividing the offset predictive sum by two to produce a prediction error;

accumulating the prediction error for each sub-integer pixel position; and

dividing the accumulated prediction error for each sub-integer pixel position by a number of contributions to the prediction error to produce a final offset value for each sub-integer pixel position.

5. The method of claim 1, wherein determining a final block of offset values comprises:

adding the first block of predictive values and the first offset values to produce a first offset predictive sum;

adding the second block of predictive values and the second offset values to produce a second offset predictive sum;

multiplying the first offset predictive sum by a first weight to produce a first weighted offset predictive sum, and the second offset predictive sum by a second weight to produce a second weighted offset predictive sum;

adding the first weighted offset predictive sum and the second weighted offset predictive sum to produce a final weighted offset predictive sum;

dividing the final weighted offset predictive sum by a normalization term to produce a prediction error;

accumulating the prediction error for each sub-integer pixel position; and

dividing the accumulated prediction error for each sub-integer pixel position by a number of contributions to the prediction error to produce a final offset value for each sub-integer pixel position.

6. The method of claim 1, further comprising encoding the final block of offset values as part of a coded bitstream that includes the coded current video block.

7. The method of claim 6, further comprising transmitting the coded bitstream from a source device to a destination device.

8. The method of claim 6, wherein encoding the final block of offset values comprises assigning a first number of bits to an integer portion of an offset value of the final block of offset values and assigning a second number of bits to a fractional portion of the offset value of the final block of offset values, and wherein the first and second numbers of bits are determined based on a magnitude of the integer portion.

9. The method of claim 6, wherein the final block of offset values includes one offset value for integer pixel locations and several offset values for different sub-integer pixel locations.

10. The method of claim 9, wherein the final block of offset values includes fifteen different offset values for fifteen different sub-integer pixel locations.

11. An apparatus that encodes video data, the apparatus comprising a video encoder that:

interpolates a first block of predictive values associated with a sub-integer pixel position based on a first reference video unit within a first list of reference data, and a second block of predictive values associated with the sub-integer pixel position based on a second reference video unit within a second list of reference data;

calculates, for sub-integer pixel positions, a first offset value based on the first block of predictive values and the current video block, and a second offset value based on the first offset value and the second block of predictive values;

determines a final block of offset values based on the first block of predictive values, the second block of predictive values, the first offset values, and the second offset values; and

encodes the current video block based on the final block of offset values.

12. The apparatus of claim 11, wherein the video encoder comprises at least one fixed interpolation filter that interpolates the first block of predictive values and the second block of predictive values.

13. The apparatus of claim 11, wherein, in order to determine the final block of offset values, the video encoder applies weights to at least one of the first offset value and the second offset value.

14. The apparatus of claim 11, wherein, in order to determine the final block of offset values, the video encoder adds together the first block of predictive values, the second block of predictive values, the first offset values, and the second offset values to produce an offset predictive sum, divides the offset predictive sum by two to produce a prediction error, accumulates the prediction error for each sub-integer pixel position, and divides the accumulated prediction error for each sub-integer pixel position by a number of contributions to the prediction error to produce a final offset value for each sub-integer pixel position.

15. The apparatus of claim 11, wherein, in order to determine the final block of offset values, the video encoder:

adds the first block of predictive values and the first offset values to produce a first offset predictive sum,

adds the second block of predictive values and the second offset values to produce a second offset predictive sum,

multiplies the first offset predictive sum by a first weight to produce a first weighted offset predictive sum, and the second offset predictive sum by a second weight to produce a second weighted offset predictive sum;

adds the first weighted offset predictive sum and the second weighted offset predictive sum to produce a final weighted offset predictive sum;

divides the final weighted offset predictive sum by a normalization term to produce a prediction error;

accumulates the prediction error for each sub-integer pixel position; and

divides the accumulated prediction error for each sub-integer pixel position by a number of contributions to the prediction error to produce a final offset value for each sub-integer pixel position.

16. The apparatus of claim 11, wherein the video encoder encodes the final block of offset values as part of a coded bitstream that includes the coded current video block.

17. The apparatus of claim 16, wherein the video encoder assigns a first number of bits to an integer portion of an offset value of the final block of offset values and assigns a second number of bits to a fractional portion of an offset value of the final block of offset values, and wherein the first and second numbers of bits are determined based on a magnitude of the integer portion.

18. The apparatus of claim 16, wherein the final block of offset values includes one offset value for integer pixel locations and several offset values for different sub-integer pixel locations.

19. The apparatus of claim 18, wherein the final block of offset values includes fifteen different offset values for fifteen different sub-integer pixel locations.

20. The apparatus of claim 11, wherein the video encoder forms part of a wireless communication device.

21. The apparatus of claim 11, wherein the video encoder comprises an integrated circuit device.

22. A computer-readable medium comprising instructions encoded on the computer-readable medium that, upon execution, cause a processor within a video encoder to:

interpolate a first block of predictive values associated with a sub-integer pixel position based on a first reference video unit within a first list of reference data, and a second block of predictive values associated with the sub-integer pixel position based on a second reference video unit within a second list of reference data;

calculate, for sub-integer pixel positions, a first offset value based on the first block of predictive values and the current video block, and a second offset value based on the first offset value and the second block of predictive values;

determine a final block of offset values based on the first block of predictive values, the second block of predictive values, the first offset values, and the second offset values; and

encode the current video block based on the final block of offset values.

23. The computer-readable medium of claim 22, wherein the instructions that cause the processor to interpolate the first block of predictive values and the second block of predictive values comprises instructions to cause the processor to select a set of interpolation filters from a set of fixed interpolation filters.

24. The computer-readable medium of claim 22, wherein the instructions that cause a processor to determine a final block of offset values comprise instructions that cause a processor to apply weights to at least one of the first offset value and the second offset value.

25. The computer-readable medium of claim 22, wherein the instructions that cause a processor to determine a final block of offset values comprise instructions that cause a processor to:

add together the first block of predictive values, the second block of predictive values, the first offset values, and the second offset values to produce an offset predictive sum; and

divide the offset predictive sum by two to produce a prediction error;

accumulate the prediction error for each sub-integer pixel position; and

divide the accumulated prediction error for each sub-integer pixel position by a number of contributions to the prediction error to produce a final offset value for each sub-integer pixel position.

26. The computer-readable medium of claim 22, wherein the instructions that cause a processor to determine a final block of offset values comprise instructions that cause a processor to:

add the first block of predictive values and the first offset values to produce a first offset predictive sum;

add the second block of predictive values and the second offset values to produce a second offset predictive sum;

multiply the first offset predictive sum by a first weight to produce a first weighted offset predictive sum, and the second offset predictive sum by a second weight to produce a second weighted offset predictive sum;

add the first weighted offset predictive sum and the second weighted offset predictive sum to produce a final weighted offset predictive sum;

divide the final weighted offset predictive sum by a normalization term to produce a prediction error;

accumulate the prediction error for each sub-integer pixel position; and

divide the accumulated prediction error for each sub-integer pixel position by a number of contributions to the prediction error to produce a final offset value for each sub-integer pixel position.

27. The computer-readable medium of claim 22, wherein the instructions that cause a processor to determine a final block of offset values comprise instructions that cause a processor to:

encode the final block of offset values as part of a coded bitstream that includes the coded current video block.

28. The computer-readable medium of claim 27, further comprising instructions that cause transmission of the coded bitstream from a source device to a destination device.

29. The computer-readable medium of claim 27, wherein the instructions that cause a processor to encode the final block of offset values comprise instructions that cause a processor to:

assign a first number of bits to an integer portion of an offset value of the final block of offset values and assign a second number of bits to a fractional portion of the offset value of the final block of offset values, wherein the first and second numbers of bits are determined based on a magnitude of the integer portion.

30. The computer-readable medium of claim 27, wherein the final block of offset values includes one offset value for integer pixel locations and several offset values for different sub-integer pixel locations.

31. The computer-readable medium of claim 30, wherein the at least one final offset value includes fifteen different offset values for fifteen different sub-integer pixel locations.

32. An apparatus that encodes video data, the apparatus comprising:

means for interpolating a first block of predictive values associated with a sub-integer pixel position based on a first reference video unit within a first list of reference data, and a second block of predictive values associated with the sub-integer pixel position based on a second reference video unit within a second list of reference data;

means for calculating, for sub-integer pixel positions, a first offset value based on the first block of predictive values and the current video block, and a second offset value based on the first offset value and the second block of predictive values;

means for determining a final block of offset values based on the first block of predictive values, the second block of predictive values, the first offset values, and the second offset values; and

means for encoding the current video block based on the final block of offset values.

33. The apparatus of claim 32, wherein the means for interpolating the first block of predictive values and the second block of predictive values is performed by fixed interpolation filters.

34. The apparatus of claim 32, wherein the means for determining a final block of offset values comprises:

means for applying weights to at least one of the first offset value and the second offset value.

35. The apparatus of claim 32, wherein the means for determining a final block of offset values comprises:

means for adding together the first block of predictive values, the second block of predictive values, the first offset values, and the second offset values to produce an offset predictive sum;

means for dividing the offset predictive sum by two to produce a prediction error;

means for accumulating the prediction error for each sub-integer pixel position; and

means for dividing the accumulated prediction error for each sub-integer pixel position by a number of contributions to the prediction error to produce a final offset value for each sub-integer pixel position.

36. The apparatus of claim 32, wherein the means for determining a final block of offset values comprises:

means for adding the first block of predictive values and the first offset values to produce a first offset predictive sum;

means for adding the second block of predictive values and the second offset values to produce a second offset predictive sum;

means for multiplying the first offset predictive sum by a first weight to produce a first weighted offset predictive sum, and the second offset predictive sum by a second weight to produce a second weighted offset predictive sum;

means for adding the first weighted offset predictive sum and the second weighted offset predictive sum to produce a final weighted offset predictive sum;

means for dividing the final weighted offset predictive sum by a normalization term to produce a prediction error;

means for accumulating the prediction error for each sub-integer pixel position; and

means for dividing the accumulated prediction error for each sub-integer pixel position by a number of contributions to the prediction error to produce a final offset value for each sub-integer pixel position.

37. The apparatus of claim 32, further comprising means for encoding the final block of offset values as part of a coded bitstream that includes the coded current video block.

38. The apparatus of claim 37, further comprising means for transmitting the coded bitstream from a source device to a destination device.

39. The apparatus of claim 37, wherein the means for encoding the final block of offset values comprises means for assigning a first number of bits to an integer portion of an offset value of the final block of offset values and assigning a second number of bits to a fractional portion of the offset value of the final block of offset values, and wherein the first and second numbers of bits are determined based on a magnitude of the integer portion.

40. The apparatus of claim 37, wherein the final block of offset values includes one offset value for integer pixel locations and several offset values for different sub-integer pixel locations.

41. The apparatus of claim 40, wherein the final block of offset values includes fifteen different offset values for fifteen different sub-integer pixel locations.

42. A video encoding method comprising:

encoding, via an encoder, as a first pass, a current frame using sequence filters, without using offsets;

if all reference frames with an id=0 are separated from the current frame by the same number of frames, then;

encoding, via an encoder, as a picture order content (POC) pass, the current frame using sequence filters, without using offsets, on POC weighted reference frames;

if the rate-distortion characteristic of the POC pass defines a lower rate distortion cost value than the rate-distortion characteristic of the first pass, then;

using the POC weighted reference frames for encoding; and

storing the rate-distortion characteristic of the POC pass as a variable;

if the rate-distortion characteristic of the first pass defines a lower rate distortion cost value than the rate-distortion characteristic of the POC pass, then storing the rate-distortion characteristic of the first pass as the variable;

if all reference frames with an id=0 are not separated from the current frame by the same number of frames, then storing the rate-distortion characteristic of the first pass as the variable;

computing, via an encoder, frame filters and offsets on the current frame;

encoding, via an encoder, as a second pass, the current frame using the frame filters and offsets computed;

if the rate-distortion characteristic of the second pass defines a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then;

storing the rate-distortion characteristic of the second pass as the variable; and

increasing a quantization step by one and encoding, as a third pass, the current frame using the frame filters and offsets computed;

if the rate-distortion characteristic of the second pass does not define a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then increasing the quantization step by one and encoding, as the third pass, the current frame using the sequence filters, without using offsets;

if the rate-distortion characteristic of the third pass defines a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then adding the encoding of the third pass to the bitstream;

if the rate-distortion characteristic of the third pass does not define a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then adding the encoding of the variable to the bitstream; and

computing, via an encoder, sequence filters to encode the next frame.

43. An apparatus that encodes video data, the apparatus comprising a video encoder that:

encodes, as a first pass, a current frame using sequence filters, without using offsets;

if all reference frames with an id=0 are separated from the current frame by the same number of frames, then;

encodes, as a picture order content (POC) pass, the current frame using sequence filters, without using offsets, on POC weighted reference frames;

if the rate-distortion characteristic of the POC pass defines a lower rate distortion cost value than the rate-distortion characteristic of the first pass, then;

uses the POC weighted reference frames to encode; and

stores the rate-distortion characteristic of the POC pass as a variable;

if the rate-distortion characteristic of the first pass defines a lower rate distortion cost value than the rate-distortion characteristic of the POC pass, then stores the rate-distortion characteristic of the first pass as the variable;

if all reference frames with an id=0 are not separated from the current frame by the same number of frames, then stores the rate-distortion characteristic of the first pass as the variable;

computes frame filters and offsets on the current frame;

encodes, as a second pass, the current frame using the frame filters and offsets computed;

if the rate-distortion characteristic of the second pass defines a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then;

stores the rate-distortion characteristic of the second pass as the variable; and

increases a quantization step by one and encodes, as a third pass, the current frame using the frame filters and offsets computed;

if the rate-distortion characteristic of the second pass does not define a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then increases the quantization step by one and encodes, as the third pass, the current frame using the sequence filters, without using offsets;

if the rate-distortion characteristic of the third pass defines a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then adds the encoding of the third pass to the bitstream;

if the rate-distortion characteristic of the third pass does not define a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then adds the encoding of the variable to the bitstream; and

computes sequence filters to encode the next frame.

44. A computer-readable medium comprising instructions encoded on the computer-readable medium that, upon execution, cause a processor within a video encoder to:

encode, as a first pass, a current frame using sequence filters, without using offsets;

if all reference frames with an id=0 are separated from the current frame by the same number of frames, then;

encode, as a picture order content (POC) pass, the current frame using sequence filters, without using offsets, on POC weighted reference frames;

if the rate-distortion characteristic of the POC pass defines a lower rate distortion cost value than the rate-distortion characteristic of the first pass, then;

use the POC weighted reference frames to encode; and

store the rate-distortion characteristic of the POC pass as a variable;

if the rate-distortion characteristic of the first pass defines a lower rate distortion cost value than the rate-distortion characteristic of the POC pass, then store the rate-distortion characteristic of the first pass as the variable;

if all reference frames with an id=0 are not separated from the current frame by the same number of frames, then stores the rate-distortion characteristic of the first pass as the variable;

compute frame filters and offsets on the current frame;

encode, as a second pass, the current frame using the frame filters and offsets computed;

if the rate-distortion characteristic of the second pass defines a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then;

store the rate-distortion characteristic of the second pass as the variable; and

increase a quantization step by one and encodes, as a third pass, the current frame using the frame filters and offsets computed;

if the rate-distortion characteristic of the second pass does not define a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then increase the quantization step by one and encodes, as the third pass, the current frame using the sequence filters, without using offsets;

if the rate-distortion characteristic of the third pass defines a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then add the encoding of the third pass to the bitstream;

if the rate-distortion characteristic of the third pass does not define a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then add the encoding of the variable to the bitstream; and

compute sequence filters to encode the next frame.

45. An apparatus that encodes video data, the apparatus comprising:

means for encoding, via an encoder, as a first pass, a current frame using sequence filters, without using offsets;

if all reference frames with an id=0 are separated from the current frame by the same number of frames, then;

means for encoding, via an encoder, as a picture order content (POC) pass, the current frame using sequence filters, without using offsets, on POC weighted reference frames;

if the rate-distortion characteristic of the POC pass defines a lower rate distortion cost value than the rate-distortion characteristic of the first pass, then;

means for using the POC weighted reference frames for encoding; and

means for storing the rate-distortion characteristic of the POC pass as a variable;

if the rate-distortion characteristic of the first pass defines a lower rate distortion cost value than the rate-distortion characteristic of the POC pass, then means for storing the rate-distortion characteristic of the first pass as the variable;

if all reference frames with an id=0 are not separated from the current frame by the same number of frames, then means for storing the rate-distortion characteristic of the first pass as the variable;

means for computing, via an encoder, frame filters and offsets on the current frame;

means for encoding, via an encoder, as a second pass, the current frame using the frame filters and offsets computed;

if the rate-distortion characteristic of the second pass defines a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then;

means for storing the rate-distortion characteristic of the second pass as the variable; and

means for increasing a quantization step by one and encoding, as a third pass, the current frame using the frame filters and offsets computed;

if the rate-distortion characteristic of the second pass does not define a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then means for increasing the quantization step by one and encoding, as the third pass, the current frame using the sequence filters, without using offsets;

if the rate-distortion characteristic of the third pass defines a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then means for adding the encoding of the third pass to the bitstream;

if the rate-distortion characteristic of the third pass does not define a lower rate distortion cost value than the rate-distortion characteristic stored in the variable, then means for adding the encoding of the variable to the bitstream; and

means for computing, via an encoder, sequence filters to encode the next frame.