Spatial extrapolation of pixel values in intraframe video coding and decoding

• US 7,116,830 B2
• Filed: 12/17/2002
• Issued: 10/03/2006
• Est. Priority Date: 12/17/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Srinivasan, Sridhar
• Primary Examiner(s): Mancuso, Joseph
• Assistant Examiner(s): Ge, Yuzhen
• Application Number:
  US10/322,171
• Publication Number:
  US 20030138150A1
• Time in Patent Office: 1,386 Days
• Field of Search: 382/236
• US Class Current: 382/236
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/122 Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/129 Scanning of coding units, e.g. zig-zag scan of transform coefficients or flexible macroblock ordering [FMO]
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/15 by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/46 : Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/50 : using predictive coding H04N19/61 takes precedence
• CPC Class Codes: H04N 19/513 : Processing of motion vectors
• CPC Class Codes: H04N 19/52 : by predictive encoding
• CPC Class Codes: H04N 19/523 : with sub-pixel accuracy
• CPC Class Codes: H04N 19/527 : Global motion vector estimation
• CPC Class Codes: H04N 19/533 : Motion estimation using multistep search, e.g. 2D-log search or one-at-a-time search [OTS]
• CPC Class Codes: H04N 19/547 : Motion estimation performed in a transform domain
• CPC Class Codes: H04N 19/57 : Motion estimation characterised by a search window with variable size or shape
• CPC Class Codes: H04N 19/59 : involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/593 : involving spatial prediction techniques
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 19/625 : using discrete cosine transform [DCT]
• CPC Class Codes: H04N 19/63 : using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/70 : characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/80 : Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/82 : involving filtering within a prediction loop
• CPC Class Codes: H04N 19/86 : involving reduction of coding artifacts, e.g. of blockiness
• CPC Class Codes: H04N 19/895 : in combination with error concealment

Abstract

Techniques and tools for spatial extrapolation of pixel values in intraframe video encoding and/or decoding are described. For example, to predict the pixel values for a block of pixels, a video encoder or decoder uses spatial extrapolation from the pixel values of pixels in a causal neighborhood for the block of pixels.

Claims

1. In a computer system, a computer-implemented method of processing a current block of pixels of a video frame of a video sequence, the method comprising:

predicting a spatial extrapolation mode for the current block based upon plural contextual spatial extrapolation modes, the plural contextual spatial extrapolation modes including a first contextual spatial extrapolation mode for a first causal block above the current block in the video frame, the plural contextual spatial extrapolation modes further including a second contextual spatial extrapolation mode for a second causal block left of the current block in the video frame; and

computing an actual spatial extrapolation mode for the current block, wherein the computing is based at least in part upon the predicted spatial extrapolation mode for the current block, and wherein the computing includes evaluating one or more spatial extrapolation modes in a rank ordering starting with the predicted spatial extrapolation mode.

2. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 1 during video encoding.

3. The method of claim 1 further comprising outputting information indicating the actual spatial extrapolation mode.

4. The method of claim 3 wherein the output information is a rank order index indicating a differential between the actual spatial extrapolation mode and the predicted spatial extrapolation mode in a rank ordering.

5. The method of claim 3 further comprising entropy encoding the information before the outputting.

13. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 1 during video encoding.

14. The method of claim 1 further comprising outputting information indicating the actual spatial extrapolation mode.

15. The method of claim 14 wherein the output information is a rank order index indicating a differential between the actual spatial extrapolation mode and the predicted spatial extrapolation mode in a rank ordering.

16. The method of claim 14 further comprising entropy encoding the information before the outputting.

17. The method of claim 1 wherein the computing includes evaluating one or more spatial extrapolation modes in a rank ordering starting with the predicted spatial extrapolation mode.

18. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 1 during video decoding.

19. The method of claim 1 further comprising receiving information indicating the actual spatial extrapolation mode, wherein the computing includes evaluating the received information in view of the predicted spatial extrapolation mode.

20. The method of claim 19 wherein the received information is a rank order index indicating a differential between the actual spatial extrapolation mode and the predicted spatial extrapolation mode in a rank ordering.

21. The method of claim 19 further comprising entropy decoding the received information.

6. In a computer system, a computer-implemented method of processing a current block of pixels of a video frame of a video sequence, the method comprising:

receiving information indicating an actual spatial extrapolation mode for the current block;

predicting a spatial extrapolation mode for the current block based upon plural contextual spatial extrapolation modes, the plural contextual spatial extrapolation modes including a first contextual spatial extrapolation mode for a first causal block above the current block in the video frame, the plural contextual spatial extrapolation modes further including a second contextual spatial extrapolation mode for a second causal block left of the current block in the video frame; and

computing the actual spatial extrapolation mode for the current block, wherein the computing is based at least in part upon the predicted spatial extrapolation mode for the current block, wherein the computing includes evaluating the received information in view of the predicted spatial extrapolation mode, and wherein the received information is a rank order index indicating a differential between the actual spatial extrapolation mode and the predicted spatial extrapolation mode in a rank ordering.

7. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 6 during video decoding.

8. The method of claim 6 further comprising entropy decoding the received information.

9. In a computer system, a computer-implemented method of processing a current block of pixels of a video frame of a video sequence, the method comprising:

predicting a spatial extrapolation mode for the current block based upon plural contextual spatial extrapolation modes, the plural contextual spatial extrapolation modes including a first contextual spatial extrapolation mode for a first causal block above the current block in the video frame, the plural contextual spatial extrapolation modes further including a second contextual spatial extrapolation mode for a second causal block left of the current block in the video frame;

computing an actual spatial extrapolation mode for the current block, wherein the computing is based at least in part upon the predicted spatial extrapolation mode for the current block; and

outputting information indicating the actual spatial extrapolation mode, wherein the output information is a rank order index indicating a differential between the actual spatial extrapolation mode and the predicted spatial extrapolation mode in a rank ordering.

10. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 9 during video encoding.

11. The method of claim 9 further comprising entropy encoding the information before the outputting.

12. In a computer system, a computer-implemented method of processing a current block of pixels of a video frame of a video sequence, the method comprising:

predicting a spatial extrapolation mode for the current block based upon plural contextual spatial extrapolation modes, the plural contextual spatial extrapolation modes including a first contextual spatial extrapolation mode for a first causal block above the current block in the video frame, the plural contextual spatial extrapolation modes further including a second contextual spatial extrapolation mode for a second causal block left of the current block in the video frame, wherein if the first contextual spatial extrapolation mode equals the second contextual spatial extrapolation mode, the predicted spatial extrapolation mode is set to the first contextual spatial extrapolation mode; and

computing an actual spatial extrapolation mode for the current block, wherein the computing is based at least in part upon the predicted spatial extrapolation mode for the current block.