INTER-LAYER PREDICTION FOR EXTENDED SPATIAL SCALABILITY IN VIDEO CODING

• US 20080165855A1
• Filed: 01/07/2008
• Published: 07/10/2008
• Est. Priority Date: 01/08/2007
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Wang, Xianglin, Ridge, Justin
• Granted Patent: US 9,049,456 B2
• Time in Patent Office: Days
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/109 among a plurality of temporal predictive coding modes
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/513 Processing of motion vectors
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution

Abstract

An improved system and method for providing improved inter-layer prediction for extended spatial scalability in video coding, as well as improving inter-layer prediction for motion vectors in the case of extended spatial scalability. In various embodiments, for the prediction of macroblock mode, the actual reference frame index and motion vectors from the base layer are used in determining if two blocks should be merged. Additionally, multiple representative pixels in a 4×4 block can be used to represent each 4×4 block in a virtual base layer macroblock. The partition and motion vector information for the relevant block in the virtual base layer macroblock can be derived from all of the partition information and motion vectors of those 4×4 blocks.

Claims

1. A method for encoding an enhancement layer block representing at least a portion of a video frame within a scalable bit stream, comprising:

deriving a coding mode indicator specifying a partition of the enhancement layer block based at least in part on a plurality of reference frame index values corresponding to a plurality of reference layer blocks, wherein the reference layer blocks that have the same reference frame index are merged to derive the partition size; and

encoding the enhancement layer block utilizing the coding mode indicator.

2. The method of claim 1, wherein the reference layer blocks that have the same motion vector information are merged to derive the partition size.

3. The method of claim 1, wherein the reference layer blocks that have motion vector information having a difference metric less than a predetermined threshold are merged to derive the partition size.

4. The method of claim 3, wherein the difference metric is derived by a sum of absolute difference of motion vector components, and wherein the threshold value is equal to one.

5. The method of claim 1, wherein a plurality of representative pixels are used to select partition and motion vector information for each respective reference layer block.

6. The method of claim 1, wherein, if the plurality of reference blocks have different reference frame indices, then the motion vector information from the reference block with the lowest reference frame indices is selected.

7. A computer program product, embodied in a computer-readable medium, comprising computer code for performing the processes of claim 1.

8. An apparatus, comprising:

a processor; and

a memory unit communicatively connected to the processor and including;

computer code for deriving a coding mode indicator specifying a partition of an enhancement layer block based at least in part on a plurality of reference frame index values corresponding to a plurality of reference layer blocks, wherein the reference layer blocks that have the same reference frame index are merged to derive the partition size; and

computer code for encoding the enhancement layer block utilizing the coding mode indicator.

9. The apparatus of claim 8, wherein the reference layer blocks that have the same motion vector information are merged to derive the partition size.

10. The apparatus of claim 8, wherein the reference layer blocks that have motion vector information having a difference metric less than a predetermined threshold are merged to derive the partition size.

11. The apparatus of claim 10, wherein the difference metric is derived by a sum of absolute difference of motion vector components, and wherein the threshold value is equal to one.

12. The apparatus of claim 8, wherein a plurality of representative pixels are used to select partition and motion vector information for each respective reference layer block.

13. The apparatus of claim 8, wherein, if the plurality of reference blocks have different reference frame indices, then the motion vector information from the reference block with the lowest reference frame indices is selected.

14. An apparatus, comprising:

means for deriving a coding mode indicator specifying a partition of an enhancement layer block based at least in part on a plurality of reference frame index values corresponding to a plurality of reference layer blocks, wherein the reference layer blocks that have the same reference frame index are merged to derive the partition size; and

encoding the enhancement layer block utilizing the coding mode indicator.

15. A method for decoding an enhancement layer block representing at least a portion of a video frame within a scalable bit stream, comprising:

deriving a coding mode indicator specifying a partition of the enhancement layer block based at least in part on a plurality of reference frame index values corresponding to a plurality of reference layer blocks, wherein the reference layer blocks that have the same reference frame index are merged to derive the partition size; and

decoding the enhancement layer block utilizing the coding mode indicator.

16. The method of claim 15, wherein the reference layer blocks that have the same motion vector information are merged to derive the partition size.

17. The method of claim 15, wherein the reference layer blocks that have motion vector information having a difference metric less than a predetermined threshold are merged to derive the partition size.

18. The method of claim 17, wherein the difference metric is derived by a sum of absolute difference of motion vector components, and wherein the threshold value is equal to one.

19. The method of claim 15, wherein a plurality of representative pixels are used to select partition and motion vector information for the respective reference layer block.

20. The method of claim 15, wherein, if the plurality of reference blocks have different reference frame indices, then the motion vector information from the block with the lowest reference frame indices is selected.

21. A computer program product, embodied in a computer-readable medium, comprising computer code for performing the processes of claim 15.

22. An apparatus, comprising:

a processor; and

a memory unit communicatively connected to the processor and including;

computer code for deriving a coding mode indicator specifying a partition of an enhancement layer block based at least in part on a plurality of reference frame index values corresponding to a plurality of reference layer blocks, wherein the reference layer blocks that have the same reference frame index are merged to derive the partition size; and

decoding the enhancement layer block utilizing the coding mode indicator.

23. The apparatus of claim 22, wherein the reference layer blocks that have the same motion vector information are merged to derive the partition size.

24. The apparatus of claim 22, wherein the reference layer blocks that have motion vector information having a difference metric less than a predetermined threshold are merged to derive the partition size.

25. The apparatus of claim 24, wherein the difference metric is derived by a sum of absolute difference of motion vector components, and wherein the threshold value is equal to one.

26. The apparatus of claim 22, wherein a plurality of representative pixels are used to select partition and motion vector information for the respective reference layer block.

27. The apparatus of claim 22, wherein, if the plurality of reference blocks have different reference frame indices, then the motion vector information from the block with the lowest reference frame indices is selected.

28. An apparatus, comprising:

means for deriving a coding mode indicator specifying a partition of an enhancement layer block based at least in part on a plurality of reference frame index values corresponding to a plurality of reference layer blocks, wherein the reference layer blocks that have the same reference frame index are merged to derive the partition size; and

means for decoding the enhancement layer block utilizing the coding mode indicator.

29. A method for encoding an enhancement layer block representing at least a portion of a video frame within a scalable bit stream, comprising:

deriving partition and motion vector information for the enhancement layer block based upon partition and motion vector information from a plurality of reference blocks; and

encoding the enhancement layer block utilizing the partition and motion vector information.

30. The method of claim 29, wherein, if each of the mapped reference blocks have different reference frame indices, then the motion vector information from the reference block with the lowest reference frame index is selected to derive the partition and motion vector information for the enhancement layer block.

31. The method of claim 29, wherein, if more than one of the mapped blocks have the smallest reference frame indices, then an average of the motion vector information for the respective reference blocks with the smallest reference frame indices is selected to derive the partition and motion vector information for the enhancement layer block.

32. The method of claim 31, wherein the average comprises a weighted average using weighting coefficients, and wherein the weighting coefficient for the motion vector in each mapped reference block is determined using at least one of a size of the area the motion vector represents within the reference block;

a delta motion vector for the motion vector;

a partition size for the reference block;

a block type of the reference block; and

a block mode of the reference layer block.

33. A computer program product, embodied in a computer-readable medium, comprising computer code for performing the processes of claim 29.

34. An apparatus, comprising:

a processor; and

a memory unit communicatively connected to the processor and including;

computer code for deriving partition and motion vector information for an enhancement layer block based upon partition and motion vector information from a plurality of reference blocks; and

computer code for encoding the enhancement layer block utilizing the partition and motion vector information.

35. The apparatus of claim 34, wherein, if each of the mapped reference blocks have different reference frame indices, then the motion vector information from the reference block with the lowest reference frame index is selected to derive the partition and motion vector information for the enhancement layer block.

36. The apparatus of claim 34, wherein, if more than one of the mapped blocks have the smallest reference frame indices, then an average of the motion vector information for the respective reference blocks with the smallest reference frame indices is selected to derive the partition and motion vector information for the enhancement layer block.

37. The apparatus of claim 36, wherein the average comprises a weighted average using weighting coefficients, and wherein the weighting coefficient for the motion vector in each mapped reference block is determined using at least one of a size of the area the motion vector represents within the reference block;

a delta motion vector for the motion vector;

a partition size for the reference block;

a block type of the reference block; and

a block mode of the reference layer block.

38. An apparatus, comprising:

means for deriving partition and motion vector information for an enhancement layer block based upon partition and motion vector information from a plurality of reference blocks; and

means for encoding the enhancement layer block utilizing the partition and motion vector information.

39. A method of decoding an enhancement layer block representing at least a portion of a video frame within a scalable bit stream, comprising:

deriving partition and motion vector information for the enhancement layer block based upon partition and motion vector information from a plurality of reference blocks; and

decoding the enhancement layer block utilizing the partition and motion vector information.

40. The method of claim 39, wherein, if each of the mapped reference blocks have different reference frame indices, then the motion vector information from the reference block with the lowest reference frame index is selected to derive the partition and motion vector information for the enhancement layer block.

41. The method of claim 39, wherein, if more than one of the mapped blocks have the smallest reference frame indices, then an average of the motion vector information for the respective reference blocks with the smallest reference frame indices is selected to derive the partition and motion vector information for the enhancement layer block.

42. The method of claim 41, wherein the average comprises a weighted average using weighting coefficients, and wherein the weighting coefficient for the motion vector in each mapped reference block is determined using at least one of a size of the area the motion vector represents within the reference block;

a delta motion vector for the motion vector;

a partition size for the reference block;

a block type of the reference block; and

a block mode of the reference layer block.

43. A computer program product, embodied in a computer-readable medium, comprising computer code for performing the processes of claim 39.

44. An apparatus, comprising:

a processor; and

a memory unit communicatively connected to the processor and including;

computer code for deriving partition and motion vector information for an enhancement layer block based upon partition and motion vector information from a plurality of reference blocks; and

computer code for decoding the enhancement layer block utilizing the partition and motion vector information.

45. The apparatus of claim 44, wherein, if each of the mapped reference blocks have different reference frame indices, then the motion vector information from the reference block with the lowest reference frame index is selected to derive the partition and motion vector information for the enhancement layer block.

46. The apparatus of claim 44, wherein, if more than one of the mapped blocks have the smallest reference frame indices, then an average of the motion vector information for the respective reference blocks with the smallest reference frame indices is selected to derive the partition and motion vector information for the enhancement layer block.

47. The apparatus of claim 46, wherein the average comprises a weighted average using weighting coefficients, and wherein the weighting coefficient for the motion vector in each mapped reference block is determined using at least one of a size of the area the motion vector represents within the reference block;

a delta motion vector for the motion vector;

a partition size for the reference block;

a block type of the reference block; and

a block mode of the reference layer block.

48. An apparatus, comprising:

means for deriving partition and motion vector information for an enhancement layer block based upon partition and motion vector information from a plurality of reference blocks; and

means for decoding the enhancement layer block utilizing the partition and motion vector information.