Adaptive video encoding using a perceptual model

• US 8,422,546 B2
• Filed: 05/25/2005
• Issued: 04/16/2013
• Est. Priority Date: 05/25/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lin, Chih-Lung, Xia, Minghui, Hsu, Pohsiang, Wu, Shou-Jen
• Primary Examiner(s): AN, SHAWN S
• Application Number:
  US11/137,924
• Publication Number:
  US 20060268990A1
• Time in Patent Office: 2,883 Days
• Field of Search: 375/240, 375/240.04, 375/240.03, 375/240.02, 375/240.06, 375/240.12, 382/251, 382/239, 382/238, 382/240.12, 382/240
• US Class Current: 375/240
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A video encoder includes a region detector module that classifies blocks of video frames. An adaptive filter module applies a median filter to a block based upon a block classification assigned by the region detector module. An adaptive quantization module quantizes a block according to a quantization method adaptively determined based upon a block classification assigned by the region detection module. In one example, a video encoder adaptively determines a median filter selected using a block classification. In another example, a video encoder adaptively determines whether to drop an isolated last transform coefficient based on the block classification, and/or applies a dead-zone selected using the block classification.

Claims

1. In a computing device that implements a video encoder, a method comprising, with the computing device, for each of plural blocks to be encoded using adaptive inter-picture compression:

classifying the block according to content of the block, wherein the classifying the block includes;

determining a block gradient for the block to be encoded using the adaptive inter-picture compression, including;

downsampling luminance samples for the block; and

determining a pixel gradient for each of plural pixels within the block, the pixel gradient being between one or more samples for the pixel and one or more corresponding samples for surrounding pixels, wherein the pixel gradient is quantified as a sum of differences between the samples for the pixel and the one or more corresponding samples for surrounding pixels, wherein the sum of differences is determined based upon both the luminance samples and chrominance samples for the block, and wherein the downsampling of the luminance samples is performed before the sum of differences is determined; and

evaluating the block gradient to determine a block value; and

adaptively encoding the block using adaptive inter-picture compression and depending at least in part upon the block classification, wherein the adaptive encoding includes using two or more of;

adaptive median filtering depending at least in part on the block classification, selective dropping of a last non-zero transform coefficient depending at least in part on the block classification,

adaptive dead zone re-sizing depending at least in part on the block classification, or

quantization step size adjustment depending at least in part on the block classification.

2. In a computing device that implements a video encoder, a method comprising, with the computing device, for each of plural blocks:

classifying the block according to its content, wherein the classifying the block includes;

determining a block gradient, including determining a pixel gradient for each of plural pixels within the block, wherein the pixel gradient is quantified as a sum of differences between samples from the pixel and corresponding samples from neighboring pixels, and wherein the samples include both luminance and chrominance samples; and

evaluating the block gradient to determine a block value; and

adaptively encoding the block using adaptive inter-picture compression and depending at least in part upon the block classification, wherein the adaptive encoding includes using selective dropping of a last non-zero transform coefficient, wherein the selective dropping of a last non-zero coefficient depends at least in part on the block classification and a number of consecutive zero coefficients preceding the last non-zero coefficient in a zig-zag scan, and wherein the adaptive encoding further includes using one or more of;

adaptive median filtering,

adaptive dead zone re-sizing, or

quantization step size adjustment.

3. The method of claim 2 wherein evaluating the block gradient to determine the block value comprises ordering pixel gradient values numerically and selecting a median value as the block value.

4. The method of claim 2 wherein the adaptive median filtering includes changing filter dimensions depending on the block classification, and wherein the adaptive median filtering is performed on a motion compensation prediction residual as part of the adaptive inter-picture compression of the block.

5. The method of claim 2 wherein the determining the pixel gradient comprises using only pixel data from the plural pixels within the block.

6. The method of claim 2 wherein the block value is determined based on an average of pixel gradients.

7. The method of claim 2 wherein the block value is determined based on a median of pixel gradients.

8. The method of claim 2 wherein the adaptive encoding includes using the quantization step size adjustment, and wherein the quantization step size adjustment includes applying differential quantization depending at least in part on the block classification.

9. The method of claim 2 wherein the adaptive encoding further comprises applying adaptive quantization dependent upon:

a characterization of the block, and

a frequency characterization of a coefficient within a block.

10. The method of claim 2 wherein the adaptive encoding includes using the adaptive dead zone re-sizing, and wherein the adaptive dead zone re-sizing includes switching between plural quantizer dead zone sizes depending at least in part on the block classification.

11. A video decoder, decoding an output of an encoder performing the method of claim 2.

12. In a computing device that implements a video encoder, a method comprising:

with the computing device that implements the video encoder, for each of plural blocks of video data, classifying the block according to content of the block and adaptively encoding the block based at least in part upon the block classification, wherein the adaptive encoding includes selective dropping of a last non-zero transform coefficient depending at least in part on the block classification, wherein the selective dropping of a last non-zero coefficient further depends at least in part on a number of consecutive zero coefficients preceding the last non-zero coefficient in a zig-zag scan, and wherein the adaptive encoding further includes using one or more of;

adaptive median filtering depending at least in part on the block classification,

adaptive dead zone re-sizing depending at least in part on the block classification, or

quantization step size adjustment depending at least in part on the block classification.

13. The method of claim 12, wherein the classifying the block includes:

determining a block gradient for the block, including determining a pixel gradient for each of plural pixels of the block, the pixel gradient between one or more samples for the pixel and one or more corresponding samples from surrounding pixels; and

evaluating the block gradient to determine a block value.

14. The method of claim 13 wherein the block classification is based at least in part on the block value, the block classification comprising one of smooth, texture, and edge.

15. The method of claim 13 wherein the block value is determined based on an average of pixel gradients.

16. The method of claim 13 wherein the block value is determined based on a median of pixel gradients.

17. The method of claim 12 wherein the adaptive median filtering includes changing filter dimensions depending on the block classification, and wherein the adaptive median filtering is performed on a motion compensation prediction residual as part of the adaptive encoding.

18. The method of claim 12 wherein the adaptive encoding includes using the quantization step size adjustment, and wherein the quantization step size adjustment includes applying differential quantization depending at least in part on the block classification.

19. The method of claim 12 wherein the adaptive encoding further comprises applying adaptive quantization dependent upon:

a characterization of the block, and

a frequency characterization of a coefficient within a block.

20. The method of claim 12 wherein the adaptive encoding includes using the adaptive dead zone re-sizing, and wherein the adaptive dead zone re-sizing includes switching between plural quantizer dead zone sizes depending at least in part on the block classification.

21. A computing device that implements a video encoder, the computing device comprising a processor, memory and storage that stores computer-executable instructions for causing the processor to perform a method comprising, for each of plural blocks of video data:

classifying the block according to content of the block and adaptively encoding the block based at least in part upon the block classification, wherein the adaptive encoding includes selective dropping of a last non-zero transform coefficient depending at least in part on the block classification, wherein the selective dropping of a last non-zero coefficient further depends at least in part on a number of consecutive zero coefficients preceding the last non-zero coefficient in a zig-zag scan, and wherein the adaptive encoding further includes using one or more of;

adaptive median filtering depending at least in part on the block classification,

adaptive dead zone re-sizing depending at least in part on the block classification, or

quantization step size adjustment depending at least in part on the block classification.

22. A computing device that implements a video encoder, the computing device comprising a processor, memory and storage that stores computer-executable instructions for causing the processor to perform a method comprising, for each of plural non-encoded blocks to be encoded using adaptive inter-picture compression:

classifying the block according to content of the block, wherein the classifying the block includes;

determining a block gradient for the block to be encoded using the adaptive inter-picture compression, including;

downsampling luminance samples for the block; and

determining a pixel gradient for each of plural pixels within the block, the pixel gradient being between one or more samples for the pixel and one or more corresponding samples for surrounding pixels, wherein the pixel gradient is quantified as a sum of differences between the one or more samples for the pixel and the one or more corresponding samples for surrounding pixels, wherein the sum of differences is determined based upon both the luminance samples and chrominance samples for the block, and wherein the downsampling of the luminance samples is performed before the sum of differences is determined; and

evaluating the block gradient to determine a block value; and

adaptively encoding the block using the adaptive inter-picture compression and depending at least in part upon the block classification, wherein the adaptive encoding includes using two or more of;

adaptive median filtering depending at least in part on the block classification,

selective dropping of a last non-zero transform coefficient depending at least in part on the block classification,

adaptive dead zone re-sizing depending at least in part on the block classification, or

quantization step size adjustment depending at least in part on the block classification.

23. In a computing device that implements a video encoder, a method comprising, with the computing device, for each of plural non-encoded blocks to be encoded using adaptive inter-picture compression:

classifying the block according to its content, wherein the classifying the block includes;

determining a block gradient, including determining a pixel gradient for each of plural pixels within the block, the pixel gradient between one or more samples for the pixel and one or more corresponding samples from surrounding pixels; and

evaluating the block gradient to determine a block value, wherein the evaluating comprises ordering pixel gradient values numerically and selecting a median value as the block value; and

adaptively encoding the block using the adaptive inter-picture compression and depending at least in part upon the block classification, wherein the adaptive encoding includes using selective dropping of a last non-zero coefficient, wherein the selective dropping of a last non-zero coefficient depends at least in part on a number of consecutive zero coefficients preceding the last non-zero coefficient in a zig-zag scan, and wherein the adaptive encoding further includes using one or more of;

adaptive median filtering,

adaptive dead zone re-sizing, or

quantization step size adjustment.

24. The computing device of claim 23 wherein the block value is determined based on an average of pixel gradients.

25. The computing device of claim 23 wherein the block value is determined based on a median of pixel gradients.

26. The computing device of claim 23 wherein the adaptive encoding further comprises applying adaptive quantization dependent upon:

a characterization of the block, and

a frequency characterization of a coefficient within a block.