Adaptive updates in motion-compensated temporal filtering

• US 20060008038A1
• Filed: 05/10/2005
• Published: 01/12/2006
• Est. Priority Date: 07/12/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wu, Feng, Xu, Jizheng, Song, Li
• Granted Patent: US 8,442,108 B2
• Time in Patent Office: Days
• US Class Current: 375/350
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/615 using motion compensated temporal filtering [MCTF]
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets

Abstract

Techniques and tools are described for scalable video coding and decoding. For example, a video encoder and decoder perform adaptive update operations in lifting-based motion-compensated temporal filtering and inverse motion-compensated temporal filtering, respectively. The encoder, in a prediction stage, computes a motion-compensated prediction for an odd image and computes a high-pass image as the difference between the odd image and the prediction. For an update stage, the encoder performs motion compensation on the high-pass image (and potentially another high-pass image) to compute a motion-compensated update. Based upon perceptual criteria, the encoder adaptively combines the update with an even image to form a low-pass image. The decoder, for an update stage, performs motion compensation on one or more high-pass images to compute a motion-compensated update. Based upon perceptual criteria, the encoder adaptively combines the update with a low-pass image to reconstruct an even frame.

Claims

1. A method of motion-compensated temporal filtering or inverse motion-compensated temporal filtering that includes a prediction stage and an update stage, the method comprising:

adapting update information, wherein the adapting is based at least in part upon a model of the human visual system; and

applying results of the adapting in encoding or decoding.

2. The method of claim 1 wherein the model is a just noticeable difference model.

3. The method of claim 1 wherein an encoder and a decoder use the same metric for the adapting.

4. The method of claim 1 wherein the adapting includes setting one or more image-dependent thresholds.

5. The method of claim 4 wherein the one or more image-dependent thresholds vary depending on variance of sample values in a spatial neighborhood.

6. The method of claim 4 wherein the one or more image-dependent thresholds are lower for flat areas and higher for noisy areas.

7. The method of claim 4 wherein the one or more image-dependent thresholds vary depending on a contrast adjustment tuning parameter, and wherein the one or more image-dependent thresholds are further modified by a strength parameter.

8. The method of claim 1 wherein the adapting comprises:

passing through those parts of the update information less than the absolute value of an applicable threshold; and

limiting other parts of the update information to the applicable threshold as modified by sign of the other parts of the update information.

9. The method of claim 1 further comprising, during encoding:

in the prediction stage, for each of one or more first images,

computing a motion-compensated prediction for the first image with reference to one or more second images; and

computing a high-pass image as the difference between the first image and the motion-compensated prediction; and

performing motion compensation on the one or more high-pass images, wherein the one or more motion-compensated high-pass images contribute to the update information, wherein the adapting include combining the update information with one of the one or more second images to form a low-pass image, and wherein the applying includes subsequent spatial decomposition and/or entropy encoding.

10. The method of claim 1 further comprising, during decoding, performing motion compensation on one or more high-pass images, wherein the one or more motion-compensated high-pass images contribute to the update information, wherein the adapting include combining the update information with a low-pass image, and wherein the applying includes subsequent motion compensated-prediction and/or display.

11. A computer-readable medium storing computer-executable instructions for causing a computer system programmed thereby to perform a method comprising:

based at least in part on one or more perceptual criteria, computing one or more update thresholds for lifting-based motion-compensated temporal filtering or inverse motion-compensated temporal filtering; and

thresholding update information according to the one or more update thresholds.

12. The computer-readable medium of claim 11 wherein the one or more update thresholds are computed per image for each of plural images to be updated.

13. The computer-readable medium of claim 11 wherein the one or more update thresholds vary depending on variance of sample values in a spatial neighborhood.

16. The computer-readable medium of claim 11 wherein the one or more update thresholds vary depending on a strength parameter.

17. The method of claim 18 wherein the thresholding comprises:

passing through those parts of the update information less than an applicable update threshold of the one or more update thresholds; and

limiting other parts of the update information to the applicable update threshold as modified by sign of the other parts of the update information.

18. A video encoder including the computer-readable medium of claim 11.

19. A video decoder including the computer-readable medium of claim 11.

14. The computer-readable medium of claim 111 wherein the one or more update thresholds are lower for flat areas and higher for noisy areas.

15. The computer-readable medium of claim 111 wherein the one or more update thresholds vary depending on a contrast adjustment tuning parameter.

20. A tool comprising one of more buffers for storing video picture information and means for adaptively performing update operations during motion-compensated temporal filtering or inverse motion-compensated temporal filtering in view of update thresholds computed based upon perceptual criteria and buffered video picture information.