Scalable video coding

• US 8,422,555 B2
• Filed: 07/09/2007
• Issued: 04/16/2013
• Est. Priority Date: 07/11/2006
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Wang, Xianglin, Ridge, Justin
• Primary Examiner(s): Dastouri, Mehrdad
• Assistant Examiner(s): ELAHI, SHAN E
• Application Number:
  US11/775,129
• Publication Number:
  US 20080056356A1
• Time in Patent Office: 2,108 Days
• Field of Search: 375/240.12, 37524001-24018
• US Class Current: 375/240.12
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/513 Processing of motion vectors
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

A system and method for predicting an enhancement layer macroblock. A base layer frame is divided into intra-coded and inter-coded regions. If any portion of the enhancement layer macroblock is covered by both an intra-coded base layer macroblock and an inter-coded base layer macroblock, predictions utilizing the intra-coded and inter-coded macroblocks are established independently to generate at least two prediction values. The at least two prediction values are then combined to give a prediction from which the enhancement layer block is coded. Various embodiments serve to smooth the boundary effect between intra-coded regions and inter-coded regions inside the inter-layer prediction for extended spatial scalability.

Claims

1. A method comprising:

forming a first prediction value for a first portion of an enhancement layer macroblock, representing at least a portion of a video frame within a scalable bit stream, by performing a residual prediction from a first set of base layer macroblocks;

forming a second prediction value for a second portion of the enhancement layer macroblock by deriving reconstructed and upsampled values from a second set of base layer macroblocks;

performing a filtering operation on a boundary between the first and second portions of the enhancement layer macroblock, wherein all base layer macroblocks are treated as inter-coded base layer macroblocks for the derivation of the reconstructed and upsampled values to form a prediction value for the whole enhancement layer macroblock, and wherein the filtering operation comprises a blending operation between the first prediction value and the second prediction value; and

encoding the enhancement layer macroblock utilizing a combination of the first and second prediction values.

2. The method of claim 1, wherein at least the first set of base layer macroblocks cover the first portion of the enhancement layer macroblock and at least the second set of base layer macroblocks cover the second portion of the enhancement layer macroblock.

3. The method of claim 1, wherein the first set of base layer macroblocks are inter-coded, and the second set of base layer macroblocks are intra-coded.

4. The method of claim 1, wherein the residual prediction is performed from both the first and second set of base layer macroblocks, forming the first prediction value for both the first and second portions of the enhancement layer macroblocks.

5. The method of claim 4, wherein the deriving of the reconstructed and upsampled values from the second set of base layer macroblocks as a second prediction occurs after the residual prediction is performed for both the first and second set of base layer macroblocks, the second prediction value replacing the first prediction value only for the second portion of the enhancement layer macroblock.

6. The method of claim 1, wherein the blending operation comprises a weighted average operation.

7. The method of claim 1, further comprising, before deriving the reconstructed and upsampled values, adjusting the boundary of the second set of intra-coded macroblocks from the base layer to the nearest coding block boundary at the enhancement layer so that the derived second prediction covers an area with its boundary aligned with the coding block boundary at the enhancement layer.

8. The method of claim 7, wherein the adjusting of the boundary of the second set of intra-coded macroblocks is based on the percentage of area that a coding block at the enhancement layer is covered by the second set of intra-coded macroblocks from the base layer.

9. A computer program product, embodied in a non-transitory computer-readable medium, comprising computer code configured to perform the processes of claim 1.

10. An encoding apparatus comprising:

a processor; and

a memory operatively connected to the processor and including;

computer code for forming a first prediction value for a first portion of an enhancement layer macroblock by performing a residual prediction from a first set of base layer macroblocks;

computer code for forming a second prediction value for a second portion of the enhancement layer macroblock by deriving reconstructed and upsampled values from a second set of base layer macroblocks;

computer code for performing a filtering operation on a boundary between the first and second portions of the enhancement layer macroblock, wherein all base layer macroblocks are treated as inter-coded base layer macroblocks for the derivation of the reconstructed and upsampled values to form a prediction value for the whole enhancement layer macroblock, and wherein the filtering operation comprises a blending operation between the first prediction value and the second prediction value; and

computer code for encoding the enhancement layer macroblock utilizing a combination of the first and second prediction values.

11. The encoding apparatus of claim 10, wherein at least the first set of base layer macroblocks cover the first portion of the enhancement layer macroblock and at least the second set of base layer macroblocks cover the second portion of the enhancement layer macroblock.

12. The encoding apparatus of claim 10, wherein the first set of base layer macroblocks are inter-coded, and the second set of base layer macroblocks are intra-coded.

13. The encoding apparatus of claim 10, wherein the residual prediction is performed from both the first and second set of base layer macroblocks, forming the first prediction value for both the first and second portions of the enhancement layer macroblocks.

14. The encoding apparatus of claim 13, wherein the deriving of the reconstructed and upsampled values from the second set of base layer macroblocks as a second prediction occurs after the residual prediction is performed for both the first and second set of base layer macroblocks, the second prediction value replacing the first prediction value only for the second portion of the enhancement layer macroblock.

15. The encoding apparatus of claim 10, wherein the blending operation comprises a weighted average operation.

16. The encoding apparatus of claim 10, further comprising, before deriving the reconstructed and upsampled values, adjusting the boundary of the second set of intra-coded macroblocks from the base layer to the nearest coding block boundary at the enhancement layer so that the derived second prediction covers an area with its boundary aligned with the coding block boundary at the enhancement layer.

17. The encoding apparatus of claim 16, wherein the adjusting of the boundary of the second set of intra-coded macroblocks is based on the percentage of area that a coding block at the enhancement layer is covered by the second set of intra-coded macroblocks from the base layer.

18. An apparatus, comprising:

means for forming a first prediction value for a first portion of an enhancement layer macroblock by performing a residual prediction from a first set of base layer macroblocks;

means for forming a second prediction value for a second portion of the enhancement layer macroblock by deriving reconstructed and upsampled values from a second set of base layer macroblocks;

means for performing a filtering operation on a boundary between the first and second portions of the enhancement layer macroblock, wherein all base layer macroblocks are treated as inter-coded base layer macroblocks for the derivation of the reconstructed and upsampled values to form a prediction value for the whole enhancement layer macroblock, and wherein the filtering operation comprises a blending operation between the first prediction value and the second prediction value; and

means for encoding the enhancement layer macroblock utilizing a combination of the first and second prediction values.

19. A method comprising:

receiving a scalable bit stream comprising an enhancement layer macroblock representing at least a portion of a video frame;

deriving a first prediction value for a first portion of the enhancement layer macroblock by performing a residual prediction from a first set of base layer macroblocks;

deriving a second prediction value for a second portion of the enhancement layer macroblock formed from reconstructed and upsampled values of a second set of base layer macroblocks; and

decoding the enhancement layer macroblock utilizing a combination of the first and second prediction values,

wherein a boundary between the first and second portions of the enhancement layer macroblock has undergone a filtering operation that comprises a blending operation between the first prediction value and the second prediction value, wherein all base layer macroblocks are treated as inter-coded base layer macroblocks during derivation of the reconstructed and upsampled values to derive a prediction value for the whole enhancement layer macroblock, and wherein the filtering operation.

20. The method of claim 19, wherein at least a first set of base layer macroblocks cover the first portion of the enhancement layer macroblock and at least a second set of base layer macroblocks cover the second portion of the enhancement layer macroblock.

21. The method of claim 20, wherein the first set of base layer macroblocks are inter-coded, and the second set of base layer macroblocks are intra-coded.

22. A computer program product, embodied in a non-transitory computer-readable medium, comprising computer code configured to perform the processes of claim 19.

23. A decoding apparatus comprising:

a processor; and

a memory operatively connected to the processor and including;

computer code for receiving a scalable bit stream comprising an enhancement layer macroblock representing at least a portion of a video frame;

computer code for deriving a first prediction value for a first portion of the enhancement layer macroblock by performing a residual prediction from a first set of base layer macroblocks;

computer code for deriving a second prediction value for a second portion of the enhancement layer macroblock formed from reconstructed and upsampled values of a second set of base layer macroblocks; and

computer code for decoding the enhancement layer macroblock utilizing a combination of the first and second prediction values,

wherein a boundary between the first and second portions of the enhancement layer macroblock has undergone a filtering operation that comprises a blending operation between the first prediction value and the second prediction value, wherein all base layer macroblocks are treated as inter-coded base layer macroblocks during derivation of the reconstructed and upsampled values to derive a prediction value for the whole enhancement layer macroblock, and wherein the filtering operation.

24. The decoding apparatus of claim 23, wherein at least a first set of base layer macroblocks cover the first portion of the enhancement layer macroblock and at least a second set of base layer macroblocks cover the second portion of the enhancement layer macroblock.

25. The decoding apparatus of claim 24, wherein the first set of base layer macroblocks are inter-coded, and the second set of base layer macroblocks are intra-coded.

26. The decoding apparatus of claim 23, wherein the boundary that has undergone the filtering operation is between the inter-coded base layer macroblocks and the intra-coded base layer macroblocks.

27. An apparatus, comprising:

means for receiving a scalable bit stream comprising an enhancement layer macroblock representing at least a portion of a video frame;

means for deriving a first prediction value for a first portion of the enhancement layer macroblock by performing a residual prediction from a first set of base layer macroblocks;

means for deriving a second prediction value for a second portion of the enhancement layer macroblock formed from reconstructed and up sampled values of a second set of base layer macroblocks; and

means for decoding the enhancement layer macroblock utilizing a combination of the first and second prediction values,

wherein a boundary between the first and second portions of the enhancement layer macroblock has undergone a filtering operation that comprises a blending operation between the first prediction value and the second prediction value, wherein all base layer macroblocks are treated as inter-coded base layer macroblocks during derivation of the reconstructed and upsampled values to derive a prediction value for the whole enhancement layer macroblock, and wherein the filtering operation.

28. The apparatus of claim 27, wherein the boundary that has undergone the filtering operation is between the inter-coded base layer macroblocks and the intra-coded base layer macroblocks.