Conversion operations in scalable video encoding and decoding

• US 10,250,905 B2
• Filed: 01/12/2017
• Issued: 04/02/2019
• Est. Priority Date: 08/25/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Sun, Shijun, Regunathan, Shankar, Tu, Chengjie, Lin, Chih-Lung
• Primary Examiner(s): Prince, Jessica M
• Application Number:
  US15/404,999
• Publication Number:
  US 20170127085A1
• Time in Patent Office: 810 Days
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

Techniques and tools for conversion operations between modules in a scalable video encoding tool or scalable video decoding tool are described. For example, given reconstructed base layer video in a low resolution format (e.g., 4:2:0 video with 8 bits per sample) an encoding tool and decoding tool adaptively filter the reconstructed base layer video and upsample its sample values to a higher sample depth (e.g., 10 bits per sample). The tools also adaptively scale chroma samples to a higher chroma sampling rate (e.g., 4:2:2). The adaptive filtering and chroma scaling help reduce energy in inter-layer residual video by making the reconstructed base layer video closer to input video, which typically makes compression of the inter-layer residual video more efficient. The encoding tool also remaps sample values of the inter-layer residual video to adjust dynamic range before encoding, and the decoding tool performs inverse remapping after decoding.

Claims

1. One or more computer storage media having stored thereon computer software instructions for causing a processing unit, when programmed thereby, to perform operations for scalable video processing, the one or more computer storage media being selected from the group consisting of non-volatile memory, magnetic disk, CD ROM, and DVD, the operations comprising:

selecting a type of chroma upsampling, wherein the type of chroma upsampling is an interpolation filter selected from among multiple interpolation filters, each of the multiple interpolation filters having different coefficients for cubic interpolation;

receiving base layer video after reconstruction of the base layer video, the reconstructed base layer video having a luma channel and plural chroma channels with a first chroma sampling rate; and

scaling each of the plural chroma channels of the reconstructed base layer video to a second chroma sampling rate different than the first chroma sampling rate using the selected type of chroma upsampling, the selected type of chroma upsampling being indicated by one or more chroma scaling parameters that are signaled as side information.

2. The one or more computer storage media of claim 1 wherein the scaling further comprises clipping results of the using the selected type of chroma upsampling.

3. The one or more computer storage media of claim 1 wherein the operations further comprise:

determining inter-layer residual video that represents differences between the reconstructed base layer video at the second chroma sampling rate and input video at the second chroma sampling rate; and

encoding the inter-layer residual video to produce an enhancement layer bit stream.

4. The one or more computer storage media of claim 1 wherein the operations further comprise:

reconstructing inter-layer residual video that represents differences between the reconstructed base layer video at the second chroma sampling rate and input video at the second chroma sampling rate; and

combining the reconstructed base layer video at the second chroma sampling rate and the reconstructed inter-layer residual video.

5. The one or more computer storage media of claim 1 wherein the first chroma sampling rate is 4:

2;

0, and wherein the second chroma sampling rate is 4;

2;

2 or 4;

4;

4.

6. The one or more computer storage media of claim 1 wherein the one or more chroma scaling parameters change on a picture-by-picture basis.

7. The one or more computer storage media of claim 1 wherein the operations further comprise one or more of:

scaling the reconstructed base layer video from a first spatial resolution to a second spatial resolution higher than the first spatial resolution; and

scaling sample values of the reconstructed base layer video from a first sample depth to a second sample depth higher than the first sample depth.

8. The one or more computer storage media of claim 1 wherein the operations further comprise:

performing inverse tone mapping, to a wider color gamut, on sample values of the reconstructed base layer video.

9. A system comprising one or more processing units and memory, wherein the system implements a scalable video encoding tool configured to perform operations comprising:

selecting a type of chroma upsampling, wherein the type of chroma upsampling is an interpolation filter selected from among multiple interpolation filters, each of the multiple interpolation filters having different filter coefficients for cubic interpolation;

receiving base layer video after reconstruction of the base layer video, the reconstructed base layer video having a luma channel and plural chroma channels with a first chroma sampling rate;

scaling each of the plural chroma channels of the reconstructed base layer video to a second chroma sampling rate different than the first chroma sampling rate using the selected type of chroma upsampling;

determining inter-layer residual video that represents differences between the reconstructed base layer video at the second chroma sampling rate and input video at the second chroma sampling rate; and

encoding the inter-layer residual video to produce an enhancement layer bit stream, the enhancement layer bit stream including one or more chroma scaling parameters that are signaled as side information, the one or more chroma scaling parameters indicating the selected type of chroma upsampling.

10. The system of claim 9 wherein the first chroma sampling rate is 4:

2;

0, and wherein the second chroma sampling rate is 4;

2;

2 or 4;

4;

4.

11. The system of claim 9 wherein the one or more chroma scaling parameters change on a picture-by-picture basis.

12. The system of claim 9 wherein the operations further comprise one or more of:

scaling the reconstructed base layer video from a first spatial resolution to a second spatial resolution higher than the first spatial resolution; and

scaling sample values of the reconstructed base layer video from a first sample depth to a second sample depth higher than the first sample depth.

13. The system of claim 9 wherein the operations further comprise:

performing inverse tone mapping, to a wider color gamut, on sample values of the reconstructed base layer video.

14. The system of claim 9 wherein the scaling further comprises clipping results of the using the selected type of chroma upsampling.

15. In a system that includes one or more processing units and memory, the system implementing a scalable video decoding tool, a method comprising:

receiving an enhancement layer bit stream that includes one or more chroma scaling parameters signaled as side information;

selecting a type of chroma upsampling, the one or more chroma scaling parameters indicating the selected type of chroma upsampling, wherein the type of chroma upsampling is an interpolation filter selected from among multiple interpolation filters, each of the multiple interpolation filters having different filter coefficients for cubic interpolation;

receiving base layer video after reconstruction of the base layer video, the reconstructed base layer video having a luma channel and plural chroma channels with a first chroma sampling rate;

scaling each of the plural chroma channels of the reconstructed base layer video to a second chroma sampling rate different than the first chroma sampling rate using the selected type of chroma upsampling;

reconstructing, using at least part of the enhancement layer bitstream, inter-layer residual video that represents differences between the reconstructed base layer video at the second chroma sampling rate and input video at the second chroma sampling rate; and

combining the reconstructed base layer video at the second chroma sampling rate and the reconstructed inter-layer residual video.

16. The method of claim 15 wherein the first chroma sampling rate is 4:

2;

0, and wherein the second chroma sampling rate is 4;

2;

2 or 4;

4;

4.

17. The method of claim 15 wherein the one or more chroma scaling parameters change on a picture-by-picture basis.

18. The method of claim 15 further comprising one or more of:

scaling the reconstructed base layer video from a first spatial resolution to a second spatial resolution higher than the first spatial resolution; and

scaling sample values of the reconstructed base layer video from a first sample depth to a second sample depth higher than the first sample depth.

19. The method of claim 15 further comprising:

performing inverse tone mapping, to a wider color gamut, on sample values of the reconstructed base layer video.

20. The method of claim 15 wherein the scaling further comprises clipping results of the using the selected type of chroma up sampling.