Video coding with fine granularity spatial scalability

• US 20070160133A1
• Filed: 08/18/2006
• Published: 07/12/2007
• Est. Priority Date: 01/11/2006
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Ye, Yan, Bao, Yiliang
• Granted Patent: US 8,315,308 B2
• Time in Patent Office: Days
• US Class Current: 375/240.100
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/34 Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

The disclosure is directed to video coding techniques that support spatial scalability using a generalized fine granularity scalability (FGS) approach. Various degrees of spatial scalability can be achieved by sending spatially scalable enhancement layers in a generalized FGS format. Spatially scalable enhancement bitstreams can be arbitrarily truncated to conform to network conditions, channel conditions and/or decoder capabilities. Coding coefficients and syntax elements for spatial scalability can be embedded in a generalized FGS format. For good network or channel conditions, and/or enhanced decoder capabilities, additional bits received via one or more enhancement layers permit encoded video to be reconstructed with increased spatial resolution and continuously improved video quality across different spatial resolutions. The techniques permit spatial scalability layers to be coded as FGS layers, rather than discrete layers, permitting arbitrary scalability. The techniques may include features to curb error propagation that may otherwise arise due to partial decoding.

Claims

1. A video coding method comprising:

decoding a fine granularity scalability (FGS) base layer to reconstruct a base layer video block defining video at a first spatial resolution;

at least partially decoding one or more FGS enhancement layers to reconstruct an enhancement layer video block defining video at a second spatial resolution greater or equal to the first spatial resolution; and

predicting an intra-coded video block based on a weighted sum of a first prediction block formed from the base layer video block and a second prediction block formed from neighboring pixels in the enhancement layer video block.

2. The method of claim 1, wherein the second spatial resolution is greater than the first spatial resolution, the method further comprising upsampling the base layer video block, and wherein the first prediction block is formed from the upsampled base layer video block.

3. The method of claim 2, further comprising generating video at the second spatial resolution based on the decoded FGS base layer and the at least partially decoded FGS enhancement layers.

4. The method of claim 2, wherein the FGS base layer defines the video with a first quality level, and at least one of the FGS enhancement layers defines the video with a second quality level greater than the first quality level.

5. The method of claim 4, further comprising generating video at the second quality level based on the decoded FGS base layer and the at least partially decoded FGS enhancement layers.

6. The method of claim 4, further comprising generating video at the second spatial resolution based on the decoded FGS base layer and the at least partially decoded FGS enhancement layers, and generating the video at the second quality level based on the decoded FGS base layer and the at least partially decoded FGS enhancement layers.

7. The method of claim 2, further comprising predicting the intra-coded video block without using video information derived from the same frame in the same layer as the respective intra-coded video block.

8. The method of claim 2, further comprising predicting the intra-coded video block using only pixels from the base layer.

9. The method of claim 2, further comprising decoding a block in the FGS enhancement layer using a special DC mode in which a DC coefficient is predicted from a default value known to both an encoder and a decoder and AC coefficients are decoded without a prediction calculated from neighboring blocks.

10. The method of claim 2, further comprising applying only a modified spatial direct mode, which does not use information from a reference frame, to an enhancement layer if the enhancement layer is coded in a bi-predictive (B) slice.

11. The method of claim 2, further comprising upsampling a significance map from the base layer, and using the upsampled significance map to decode one or more of the enhancement layers.

12. The method of claim 1, further comprising resetting a significance map from the base layer, and decoding coefficients only as zero coefficients or nonzero significant coefficients with no refinement coefficients in one or more of the enhancement layers.

13. The method of claim 2, further comprising receiving a 1-bit syntax element with at least one of the enhancement layers to signal the use of FGS coding for spatial scalability.

14. A video coding device including a decoder that decodes a fine granularity scalability (FGS) base layer to reconstruct a base layer video block defining video at a first spatial resolution, at least partially decodes one or more FGS enhancement layers to reconstruct an enhancement layer video block defining video at a second spatial resolution greater than or equal to the first spatial resolution, and predicts an intra-coded video block based on a weighted sum of a first prediction block formed from the base layer video block and a second prediction block formed from neighboring pixels in the enhancement layer video block.

15. The device of claim 14, wherein the second spatial resolution is greater than the first spatial resolution, the decoder upsamples the base layer video block, and wherein the first prediction block is formed from the upsampled base layer video block.

16. The device of claim 15, wherein the decoder generates video at the second spatial resolution based on the decoded FGS base layer and the at least partially decoded FGS enhancement layers.

17. The device of claim 15, wherein the FGS base layer defines the video with a first quality level, and at least one of the FGS enhancement layers defines the video with a second quality level greater than the first quality level.

18. The device of claim 17, wherein the decoder generates video at the second quality level based on the decoded FGS base layer and the at least partially decoded FGS enhancement layers.

19. The device of claim 17, wherein the decoder generates video at the second spatial resolution based on the decoded FGS base layer and the at least partially decoded FGS enhancement layers, and generates the video at the second quality level based on the decoded FGS base layer and the at least partially decoded FGS enhancement layers.

20. The device of claim 17, wherein the decoder predicts the intra-coded video block without using video information derived from the same frame in the same layer as the respective intra-coded video block.

21. The device of claim 15, wherein the decoder predicts the intra-coded video block using only pixels from the base layer.

22. The device of claim 15, wherein the decoder decodes a block in the FGS enhancement layer using a special DC mode in which a DC coefficient is predicted from a default value known to both an encoder and a decoder and AC coefficients are decoded without a prediction calculated from neighboring blocks.

23. The device of claim 15, wherein the decoder applies only a spatial direct mode, without using information from a reference frame, to an enhancement layer if the enhancement layer is coded in a bi-predictive (B) slice.

24. The device of claim 15, wherein the decoder upsamples a significance map from the base layer, and uses the upsampled significance map to decode one or more of the enhancement layers.

25. The device of claim 14, wherein the decoder resets a significance map from the base layer, and decodes coefficients only as zero coefficients or nonzero significant coefficients with no refinement coefficients in one or more of the enhancement layers.

26. The device of claim 15, wherein the decoder receives a 1-bit syntax element with at least one of the enhancement layers to signal the use of FGS coding for spatial scalability.

27. A computer-readable medium comprising instructions that cause a processor to:

decode a fine granularity scalability (FGS) base layer to reconstruct a base layer video block defining video at a first spatial resolution;

at least partially decode one or more FGS enhancement layers to reconstruct an enhancement layer video block defining video at a second spatial resolution greater than or equal to the first spatial resolution; and

predict an intra-coded video block based on a weighted sum of a first prediction block formed from the base layer video block and a second prediction block formed from neighboring pixels in the enhancement layer video block.

28. The computer-readable medium of claim 27, wherein the second spatial resolution is greater than the first spatial resolution, the instructions further causing the processor to upsample the base layer video block, and wherein the first prediction block is formed from the upsampled base layer video block.

29. The computer-readable medium of claim 28, wherein the instructions cause the processor to generate video at the second spatial resolution based on the decoded FGS base layer and the at least partially decoded FGS enhancement layers.

30. The computer-readable medium of claim 29, wherein the FGS base layer defines the video with a first quality level, and at least one of the FGS enhancement layers defines the video with a second quality level greater than the first quality level.

31. The computer-readable medium of claim 30, wherein the instructions cause the processor to generate video at the second quality level based on the decoded FGS base layer and the at least partially decoded FGS enhancement layers.

32. The computer-readable medium of claim 30, wherein the instructions cause the processor to generate video at the second spatial resolution based on the decoded FGS base layer and the at least partially decoded FGS enhancement layers, and generate the video at the second quality level based on the decoded FGS base layer and the at least partially decoded FGS enhancement layers.

33. The computer-readable medium of claim 28, wherein the instructions cause the processor to predict the intra-coded video block without using video information derived from the same frame in the same layer as the respective intra-coded video block.

34. The computer-readable medium of claim 28, wherein the instructions cause the processor to predict the intra-coded video block using only pixels from the base layer.

35. The computer-readable medium of claim 28, wherein the instructions cause the processor to decode a block in the FGS enhancement layer using a special DC mode in which a DC coefficient is predicted from a default value known to both an encoder and a decoder and AC coefficients are decoded without a prediction calculated from neighboring blocks.

36. The computer-readable medium of claim 28, wherein the instructions cause the processor to apply only a spatial direct mode, without using information from a reference frame, to an enhancement layer if the enhancement layer is coded in a bi-predictive (B) slice.

37. The computer-readable medium of claim 28, wherein the instructions cause the processor to upsample a significance map from the base layer, and use the upsampled significance map to decode one or more of the enhancement layers.

38. The computer-readable medium of claim 27, wherein the instructions cause the processor to reset a significance map from the base layer, and decode coefficients only as zero coefficients or nonzero significant coefficients with no refinement coefficients in one or more of the enhancement layers.

39. The computer-readable medium of claim 28, wherein the instructions cause the processor to receive a 1-bit syntax element with at least one of the enhancement layers to signal the use of FGS coding for spatial scalability.

40. A method comprising:

decoding a fine granularity scalability (FGS) base layer including base layer video blocks defining video information at a first spatial resolution;

at least partially decoding one or more FGS enhancement layers including enhancement layer video blocks defining video information at a second spatial resolution greater than the first spatial resolution; and

predicting intra-coded video blocks without using video information derived from the same frame in the same layer as the respective intra-coded video blocks.

41. The computer-readable medium of claim 40, further comprising predicting the intra-coded video block using only pixels from the base layer.

42. The method of claim 40, further comprising decoding a block in the FGS enhancement layer using a special DC mode in which a DC coefficient is predicted from a default value known to both an encoder and a decoder and AC coefficients are decoded without a prediction calculated from neighboring blocks.

43. The method of claim 40, further comprising applying only a spatial direct mode, without using information from a reference frame, to an enhancement layer if the enhancement layer is coded in a bi-predictive (B) slice.

44. The method of claim 40, further comprising upsampling a significance map from the base layer, and using the upsampled significance map to decode one or more of the enhancement layers.

45. The method of claim 40, further comprising resetting a significance map from the base layer, and decoding coefficients only as zero coefficients or nonzero significant coefficients with no refinement coefficients in one or more of the enhancement layers.

46. A device comprising a decoder that decodes a fine granularity scalability (FGS) base layer including base layer video blocks defining video information at a first spatial resolution, at least partially decodes one or more FGS enhancement layers including enhancement layer video blocks defining video information at a second spatial resolution greater than the first spatial resolution, and predicts intra-coded video blocks without using video information derived from the same frame in the same layer as the respective intra-coded video blocks.

47. The device of claim 46, wherein the decoder predicts the intra-coded video block using only pixels from the base layer.

48. The device of claim 46, wherein the decoder decodes a block in the FGS enhancement layer using a special DC mode in which a DC coefficient is predicted from a default value known to both an encoder and a decoder and decodes AC coefficients without a prediction calculated from neighboring blocks.

49. The device of claim 46, wherein the decoder applies only a spatial direct mode, without using information from a reference frame, to an enhancement layer if the enhancement layer is coded in a bi-predictive (B) slice.

50. The device of claim 46, wherein the decoder upsamples a significance map from the base layer, and uses the upsampled significance map to decode one or more of the enhancement layers.

51. The device of claim 46, wherein the decoder resets a significance map from the base layer, and decodes coefficients only as zero coefficients or nonzero significant coefficients with no refinement coefficients in one or more of the enhancement layers.

52. A method comprising:

decoding a fine granularity scalability (FGS) base layer including base layer video blocks defining video information at a first spatial resolution;

decoding one or more FGS enhancement layers including enhancement layer video blocks defining video information at a second spatial resolution greater than the first spatial resolution; and

decoding each of the blocks in the FGS enhancement layers using a special DC mode in which a DC coefficient is predicted from a default value known to both an encoder and a decoder and AC coefficients are decoded without a prediction calculated from neighboring blocks.

53. The method of claim 52, wherein the FGS base layer defines the video information with a first quality level, and at least one of the FGS enhancement layers defines the video information with a second quality level greater than the first quality level.

54. The method of claim 52, further comprising receiving a 1-bit syntax element with at least one of the enhancement layers to signal the use of FGS coding for spatial scalability.

55. A device comprising a decoder that decodes a fine granularity scalability (FGS) base layer including base layer video blocks defining video information at a first spatial resolution, encodes one or more FGS enhancement layers including enhancement layer video blocks defining video information at a second spatial resolution greater than the first spatial resolution, and decoding each of the blocks in the FGS enhancement layers using a special DC mode in which a DC coefficient is predicted from a default value known to both an encoder and a decoder and AC coefficients are decoded without a prediction calculated from neighboring blocks.

56. The method of claim 55, wherein the FGS base layer defines the video information with a first quality level, and at least one of the FGS enhancement layers defines the video information with a second quality level greater than the first quality level.

57. The method of claim 55, further comprising receiving a 1-bit syntax element with at least one of the enhancement layers to signal the use of FGS coding for spatial scalability.