Adaptive video encoding using a perceptual model

• US 20060268990A1
• Filed: 05/25/2005
• Published: 11/30/2006
• Est. Priority Date: 05/25/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wu, Shou-Jen, Lin, Chih-Lung, Xia, Minghui, Hsu, Pohsiang
• Granted Patent: US 8,422,546 B2
• Time in Patent Office: Days
• US Class Current: 375/240.240
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A video encoder includes a region detector module that classifies blocks of video frames. An adaptive filter module applies a median filter to a block based upon a block classification assigned by the region detector module. An adaptive quantization module quantizes a block according to a quantization method adaptively determined based upon a block classification assigned by the region detection module. In one example, a video encoder adaptively determines a median filter selected using a block classification. In another example, a video encoder adaptively determines whether to drop an isolated last transform coefficient based on the block classification, and/or applies a dead-zone selected using the block classification.

Claims

1. A computerized method comprising:

for each of plural blocks of video data, classifying the block according to content of the block and adaptively encoding the block based at least in part upon the block classification, wherein the adaptive encoding includes using two or more of;

adaptive median filtering depending at least in part on the block classification,

selective dropping of a last non-zero transform coefficient depending at least in part on the block classification,

adaptive dead zone re-sizing depending at least in part on the block classification, or

quantization step size adjustment depending at least in part on the block classification.

2. The method of claim 1, wherein the classifying the block includes:

determining a block gradient for the block, including determining a pixel gradient for each of plural pixels of the block, the pixel gradient between one or more samples for the pixel and one or more corresponding samples from surrounding pixels; and

evaluating the block gradient to determine a block value.

3. The method of claim 2 wherein the block classification is based at least in part on the block value, the block classification comprising one of smooth, texture, and edge.

4. The method of claim 2 wherein the block value is determined based on an average of pixel gradients.

5. The method of claim 2 wherein the block value is determined based on a median of pixel gradients.

6. The method of claim 2, wherein the pixel gradient is quantified as a sum of differences, and wherein the sum of differences is determined based upon both luminance and chrominance samples.

7. The method of claim 6, further including downsampling the luminance samples for the block before the sum of differences is determined.

8. The method of claim 2, wherein the determining the pixel gradient comprises using only pixel data from the plural pixels of the block.

9. The method of claim 2, wherein evaluating the block gradient to determine the block value comprises ordering pixel gradient values numerically and selecting a median value as the block value.

10. The method of claim 1 wherein the adaptive median filtering includes changing filter dimensions depending on the block classification, and wherein the adaptive median filtering is performed on a motion compensation prediction residual.

11. The method of claim 1 wherein the selective dropping of a last non-zero coefficient further depends at least in part on

a number of consecutive zero coefficients preceding the last non-zero coefficient in a zig-zag scan.

12. The method of claim 1 wherein the quantization step size adjustment includes applying differential quantization depending at least in part on the block classification.

13. The method of claim 1 wherein the adaptive encoding further comprises, applying adaptive quantization dependent upon,

a characterization of the block, and

a frequency characterization of a coefficient within a block.

14. The method of claim 1 wherein the adaptive dead zone re-sizing includes switching between plural quantizer dead zone sizes depending at least in part on the block classification.

15. A video decoder, decoding an output of an encoder performing the method of claim 1.

16. A computer readable medium including computer executable instructions for performing the method of claim 1.

17. A video encoder comprising:

a region detector module for classifying blocks of video frames;

an adaptive filter module for median filtering motion compensation error data for one or more of the blocks, wherein the median filtering is based at least in part upon block classifications assigned by the region detector module; and

an adaptive quantization module for differentially quantizing the motion compensation error data based at least in part upon the block classifications assigned by the region detection module.

18. The video encoder of claim 17 wherein the adaptive quantization module is also for:

selectively dropping isolated last non-zero transform coefficients based at least in part on the block classifications, and/or

changing quantization dead-zones based at least in part on the block classifications.

19. A computer-readable program carrying medium having thereon computer-executable instructions comprising:

instructions for determining, for each of plural blocks of video data, pixel gradients for pixels within the block, the pixel gradients comprising sums of differences between video data of a subject pixel and plural surrounding pixels;

instructions for determining, for each of the plural blocks, a perceptual characterization of the block based on the plural pixel gradients for the block; and

instructions for adaptively encoding the plural blocks based on the perceptual characterizations of the plural blocks.

20. The computer-readable program carrying medium of claim 19 wherein the computer-executable instructions for adaptively encoding comprise:

instructions for applying a median filter selected using one of the perceptual characterizations;

instructions for dropping a trailing transform coefficient based at least in part on one of the perceptual characterizations; and

/or

instructions for applying in quantization a dead-zone selected using one of the perceptual characterizations.