Scalable video coding and decoding with sample bit depth and chroma high-pass residual layers

• US 20090219994A1
• Filed: 02/29/2008
• Published: 09/03/2009
• Est. Priority Date: 02/29/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sun, Shijun, Srinivasan, Sridhar, Regunathan, Shankar, Tu, Chengjie
• Granted Patent: US 8,953,673 B2
• Time in Patent Office: Days
• US Class Current: 375/240.80
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/36 Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution

Abstract

Techniques and tools are described for scalable video encoding and decoding. In some embodiments, an input frame is downsampled in terms of sample depth and chroma sampling rate, encoded, and output from the encoder as a base layer bitstream. The base layer bitstream is also reconstructed and upsampled to produce a reconstructed bitstream which is subtracted from the original input frame to produce a residual layer. The residual layer is split and encoded as a sample depth residual layer bitstream and a chroma high-pass residual layer bitstream. To recover the encoded input frame, a decoder receives one or more of these bitstreams, decodes them, and combines them to form a reconstructed image. The use of separate codecs is allowed for the base layer and the enhancement layers, without inter-layer dependencies.

Claims

1. A method of encoding video comprising:

encoding base layer video to produce at least part of a base layer bitstream;

encoding chroma high-pass residual layer video to produce at least part of a chroma enhancement layer bitstream;

encoding sample depth residual layer video to produce at least part of a sample depth enhancement layer bitstream; and

outputting the at least part of the base layer bitstream, the at least part of the chroma enhancement layer bitstream, and the at least part of the sample depth enhancement layer bitstream.

2. The method of claim 1 further comprising:

before the encoding the base layer video, producing the base layer video from input video; and

reconstructing the base layer video.

3. The method of claim 2 wherein each of the base layer video, the chroma high-pass residual layer video, the sample depth residual layer video and the reconstructed base layer video includes one or more pictures, and wherein:

the input video has a first bit depth and a first chroma sub-sampling rate;

the producing the base layer video comprises downsampling the input video to a second bit depth and a second chroma sub-sampling rate; and

the reconstructing the base layer video comprises upsampling to the first bit depth and the first chroma sub-sampling rate.

4. The method of claim 3 wherein downsampling comprises at least one of tone mapping and chroma down-sampling, and wherein the upsampling comprises at least one of chroma upsampling and inverse tone mapping.

5. The method of claim 1 wherein the encoding the base layer video and the encoding the sample depth residual layer video use different encoders.

6. The method of claim 1 wherein the encoding the base layer video and the encoding the sample depth residual layer video use the same encoder.

7. The method of claim 1 wherein the encoding the chroma high-pass residual layer video comprises encoding the chroma high-pass residual layer video with a chroma high-pass encoder.

8. The method of claim 1 further comprising:

producing overall residual layer video from input video and a reconstructed version of the base layer video; and

decomposing the overall residual layer video into the chroma high-pass residual layer video and the sample depth residual layer video.

9. The method of claim 8 wherein the step of decomposing the overall residual comprises wavelet decomposition.

10. The method of claim 8 further comprising remapping the sample depth residual layer video from a first bit depth to a second bit depth lower than the first bit depth before the encoding of the sample depth residual layer video.

11. The method of claim 1 wherein at least one of the encoding the chroma high-pass residual layer video and the encoding the sample depth residual layer video includes performing motion compensation.

12. A method comprising:

receiving at least part of a base layer bitstream and at least part of a chroma enhancement layer bitstream;

decoding the at least part of the base layer bitstream to produce base layer video;

decoding the at least part of the chroma enhancement layer bitstream to produce chroma high-pass residual layer video; and

combining the base layer video and the chroma high-pass residual layer video.

13. The method of claim 12 wherein the base layer video has a first chroma sub-sampling rate, the method further comprising upsampling from the first chroma sub-sampling rate to a second chroma sub-sampling rate.

14. The method of claim 12 further comprising:

composing the chroma high-pass residual layer video with one or more other residual layers of video prior to the combining, wherein the composing comprises wavelet synthesis.

15. The method of claim 12 further comprising:

receiving at least part of a sample depth enhancement layer bitstream; and

decoding the at least part of the sample depth enhancement layer bitstream to produce sample depth residual layer video.

16. The method of claim 15 further comprising:

on a picture-by-picture basis, combining the sample depth residual layer video with the base layer video and the chroma high-pass residual layer video.

17. The method of claim 16 wherein a different decoding loop includes each of the decoding the at least part of the base layer bitstream, the decoding the at least part of the chroma enhancement layer bitstream, and the decoding the at least part of the sample depth enhancement layer bitstream, wherein the decoding loops are independent of each other, and wherein the combining the sample depth residual layer video with the base layer video and the chroma high-pass residual layer video occurs as part of post-processing after and outside of the decoding loops.

18. A method comprising:

receiving at least part of a base layer bitstream and at least part of a sample depth enhancement layer bitstream;

decoding the at least part of the base layer bitstream to produce base layer video;

decoding the at least part of the sample depth enhancement layer bitstream to produce sample depth residual layer video; and

combining the base layer video and the sample depth residual layer video.

19. The method of claim 18 further comprising:

before the combining, remapping the sample depth residual layer video from a first bit depth to a second bit depth higher than the first bit depth.

20. The method of claim 18 wherein the base layer video has a first bit depth, the method further comprising:

upsampling from the first bit depth to a second bit depth higher than the first bit depth.