Scalable video coding and decoding with sample bit depth and chroma high-pass residual layers

• US 8,953,673 B2
• Filed: 02/29/2008
• Issued: 02/10/2015
• Est. Priority Date: 02/29/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Tu, Chengjie, Srinivasan, Sridhar, Regunathan, Shankar, Sun, Shijun
• Primary Examiner(s): Lee, Jae Y
• Assistant Examiner(s): GUADALUPE CRUZ, AIXA AMYR
• Application Number:
  US12/074,030
• Publication Number:
  US 20090219994A1
• Time in Patent Office: 2,538 Days
• Field of Search: None
• US Class Current: 375/240.08
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/36 Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution

Abstract

Techniques and tools are described for scalable video encoding and decoding. In some embodiments, an input frame is downsampled in terms of sample depth and chroma sampling rate, encoded, and output from the encoder as a base layer bitstream. The base layer bitstream is also reconstructed and upsampled to produce a reconstructed bitstream which is subtracted from the original input frame to produce a residual layer. The residual layer is split and encoded as a sample depth residual layer bitstream and a chroma high-pass residual layer bitstream. To recover the encoded input frame, a decoder receives one or more of these bitstreams, decodes them, and combines them to form a reconstructed image. The use of separate codecs is allowed for the base layer and the enhancement layers, without inter-layer dependencies.

Claims

1. A method of encoding video comprising:

encoding base layer video to produce at least part of a base layer bitstream;

determining chroma high-pass residual layer video and sample depth residual layer video from input video and a reconstructed version of the base layer video;

remapping samples of the sample depth residual layer video from a first range to a second range;

encoding the chroma high-pass residual layer video to produce at least part of a chroma enhancement layer bitstream;

encoding the remapped samples of the sample depth residual layer video to produce at least part of a sample depth enhancement layer bitstream; and

outputting the at least part of the base layer bitstream, the at least part of the chroma enhancement layer bitstream, and the at least part of the sample depth enhancement layer bitstream.

2. The method of claim 1 further comprising:

before the encoding the base layer video, producing the base layer video from the input video; and

reconstructing the base layer video.

3. The method of claim 2 wherein each of the base layer video, the chroma high-pass residual layer video, the sample depth residual layer video and the reconstructed version of the base layer video includes one or more pictures, and wherein:

the input video has a first bit depth and a first chroma sub-sampling rate;

the producing the base layer video comprises downsampling the input video to a second bit depth and a second chroma sub-sampling rate; and

the reconstructing the base layer video comprises upsampling to the first bit depth and the first chroma sub-sampling rate.

4. The method of claim 3 wherein downsampling comprises at least one of tone mapping and chroma down-sampling, and wherein the upsampling comprises at least one of chroma upsampling and inverse tone mapping.

5. The method of claim 3 wherein the downsampling comprises tone mapping, and wherein the upsampling comprises inverse tone mapping.

6. The method of claim 1 wherein the encoding the base layer video and the encoding the remapped samples of the sample depth residual layer video use different encoders.

7. The method of claim 1 wherein the encoding the base layer video and the encoding the remapped samples of the sample depth residual layer video use the same encoder.

8. The method of claim 1 wherein the encoding the chroma high-pass residual layer video comprises encoding the chroma high-pass residual layer video with a chroma high-pass encoder.

9. The method of claim 1 wherein the determining chroma high-pass residual layer video and sample depth residual layer video comprises:

producing overall residual layer video from the input video and the reconstructed version of the base layer video; and

decomposing the overall residual layer video into the chroma high-pass residual layer video and the sample depth residual layer video.

10. The method of claim 9 wherein the step of decomposing the overall residual comprises wavelet decomposition.

11. The method of claim 1 wherein at least one of the encoding the chroma high-pass residual layer video and the encoding the remapped samples of the sample depth residual layer video includes performing motion compensation.

12. A method comprising:

receiving at least part of a base layer bitstream, at least part of a chroma enhancement layer bitstream, and at least part of a sample depth enhancement layer bitstream;

decoding the at least part of the base layer bitstream to produce base layer video;

upsampling samples of the base layer video from a first chroma sub-sampling rate to a second chroma sub-sampling rate using fractional sample interpolation;

decoding the at least part of the chroma enhancement layer bitstream to produce chroma high-pass residual layer video;

decoding the at least part of the sample depth enhancement layer bitstream to produce sample depth residual layer video;

remapping decoded samples of the sample depth residual layer video from a first range to a second range; and

combining the base layer video, the sample depth enhancement layer video, and the chroma high-pass residual layer video.

13. The method of claim 12 further comprising:

composing the chroma high-pass residual layer video with one or more other residual layers of video prior to the combining, the one or more other residual layers of video including the sample depth enhancement layer video, wherein the composing comprises wavelet synthesis.

14. The method of claim 12 further comprising:

on a picture-by-picture basis, combining the sample depth residual layer video with the base layer video and the chroma high-pass residual layer video.

15. The method of claim 14 wherein a different decoding loop includes each of the decoding the at least part of the base layer bitstream, the decoding the at least part of the chroma enhancement layer bitstream, and the decoding the at least part of the sample depth enhancement layer bitstream, wherein the decoding loops are independent of each other, and wherein the combining the sample depth residual layer video with the base layer video and the chroma high-pass residual layer video occurs as part of post-processing after and outside of the decoding loops.

16. The method of claim 12 further comprising, prior to the combining, remapping decoded samples of the chroma high-pass residual layer video to another representation.

17. A method comprising:

receiving at least part of a base layer bitstream and at least part of a sample depth enhancement layer bitstream;

decoding the at least part of the base layer bitstream to produce base layer video;

decoding the at least part of the sample depth enhancement layer bitstream to produce sample depth residual layer video;

remapping decoded samples of the sample depth residual layer video from a first range to a second range; and

combining the base layer video and the sample depth residual layer video.

18. The method of claim 17 wherein the base layer video has a first bit depth, the method further comprising:

upsampling decoded samples of the base layer video from the first bit depth to a second bit depth higher than the first bit depth.

19. The method of claim 18 wherein the upsampling the decoded samples of the base layer video to the second bit depth includes inverse tone mapping.

20. A computing system adapted to perform a method of video decoding, the computing system including memory and a processor, the method comprising:

receiving at least part of a base layer bitstream and at least part of a sample depth enhancement layer bitstream;

decoding the at least part of the base layer bitstream to produce base layer video;

decoding the at least part of the sample depth enhancement layer bitstream to produce sample depth residual layer video;

remapping decoded samples of the sample depth residual layer video from a first range to a second range; and

combining the base layer video and the sample depth residual layer video.