Signaling reference layers for 3D color prediction for color gamut scalability

• US 10,104,385 B2
• Filed: 04/16/2015
• Issued: 10/16/2018
• Est. Priority Date: 04/17/2014
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Li, Xiang, Chen, Jianle, Karczewicz, Marta
• Primary Examiner(s): Parikh, Dakshesh D
• Application Number:
  US14/688,700
• Publication Number:
  US 20150304664A1
• Time in Patent Office: 1,279 Days
• Field of Search: None
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/36 Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding

Abstract

Techniques are described for performing constrained three-dimensional (3D) color prediction for color gamut scalability in video coding. Color prediction techniques for color gamut scalability may be used by video coders to generate inter-layer reference pictures when a color gamut for a reference layer of video data is different than a color gamut for an enhancement layer of the video data, or when a bit depth of the reference layer is different than a bit depth of the enhancement layer. According to the techniques, a video coder may perform 3D color prediction with constrained bit depths of input and output color components of the 3D lookup table. According to further techniques, in the case of multiple layers, a video coder may perform 3D color prediction with constrained application to reference pictures in only one or more identified reference layers.

Claims

1. A method of decoding video data, the method comprising:

receiving at least one reference layer identifier (ID) for a three-dimensional (3D) lookup table for color gamut scalability, wherein the 3D lookup table is defined for an enhancement layer of the video data, and wherein the at least one reference layer ID identifies at least one reference layer of a plurality of reference layers of the video data to which the 3D lookup table is to be applied;

applying the 3D lookup table to color components of a reference layer picture in the at least one reference layer identified by the at least one reference layer ID to convert the color components from a first color gamut for the reference layers of the video data to a second color gamut for the enhancement layer of the video data;

generating an inter-layer reference picture for an enhancement layer picture in the enhancement layer of the video data based on the converted color components; and

decoding video blocks of the enhancement layer picture based on the inter-layer reference picture generated using the 3D lookup table.

2. The method of claim 1, wherein receiving the at least one reference layer ID for the 3D lookup table comprises receiving exactly one reference layer ID identifying exactly one reference layer to which the 3D lookup table is to be applied.

3. The method of claim 2, wherein applying the 3D lookup table comprises applying the 3D lookup table to color components of a reference layer picture in the exactly one reference layer identified by the exactly one reference layer ID.

4. The method of claim 1, wherein receiving the at least one reference layer ID for the 3D lookup table comprises receiving two or more reference layer IDs identifying two or more reference layers to which the 3D lookup table is to be applied.

5. The method of claim 4,

wherein applying the 3D lookup table comprises applying the 3D lookup table to color components of a reference layer picture in each of the two or more reference layers identified by the two or more reference layer IDs to convert the color components of the reference layer pictures of the two or more reference layers;

wherein generating the inter-layer reference picture comprises generating the inter-layer reference picture based on a combination of the converted color components from the two or more reference layers.

6. The method of claim 1, wherein receiving the at least one reference layer ID comprises receiving a first reference layer ID identifying a first reference layer having first color components of a first reference layer picture to which the 3D lookup table is applied to generate a first inter-layer reference picture for a first enhancement layer picture in the enhancement layer, the method further comprising:

receiving a second reference layer ID identifying a second reference layer of the plurality of reference layers of the video data, the second reference layer being different than the first reference layer;

applying the 3D lookup table to second color components of a second reference layer picture in the second reference layer to convert the second color components of the second reference layer picture of the second reference layer;

generating a second inter-layer reference picture for a second enhancement layer picture in the enhancement layer of the video data based on the converted color components, the second enhancement layer picture being different than the first enhancement layer picture; and

decoding video blocks of the second enhancement layer picture based on the second inter-layer reference picture generated using the 3D lookup table.

7. A method of encoding video data, the method comprising:

determining at least one reference layer of a plurality of reference layers of the video data for a three-dimensional (3D) lookup table for color gamut scalability, wherein the 3D lookup table is defined for an enhancement layer of the video data;

applying the 3D lookup table to color components of a reference layer picture in the at least one reference layer to convert the color components from a first color gamut for the reference layers of the video data to a second color gamut for the enhancement layer of the video data;

generating an inter-layer reference picture for an enhancement layer picture in the enhancement layer of the video data based on the converted second color components;

encoding video blocks of the enhancement layer picture based on the inter-layer reference picture generated using the 3D lookup table; and

signaling at least one reference layer identifier (ID) identifying the at least one reference layer of the plurality of reference layers of the video data to which the 3D lookup table is applied.

8. The method of claim 7, wherein signaling the at least one reference layer ID comprises signaling exactly one reference layer ID identifying exactly one reference layer to which the 3D lookup table is applied.

9. The method of claim 8,

wherein determining the at least one reference layer comprises determining the exactly one reference layer for the 3D lookup table;

and

wherein applying the 3D lookup table comprises applying the 3D lookup table to color components of a reference layer picture in the exactly one reference layer.

10. The method of claim 7, wherein signaling the at least one reference layer ID comprises signaling two or more reference layer IDs identifying two or more reference layers to which the 3D lookup table is applied.

11. The method of claim 10,

wherein determining the at least one reference layer comprises determining the two or more reference layers for the 3D lookup table;

wherein applying the 3D lookup table comprises applying the 3D lookup table to color components of a reference layer picture in each of the two or more reference layers to convert the color components of the reference layer pictures of the two or more reference layers; and

wherein generating the inter-layer reference picture comprises generating the inter-layer reference picture based on a combination of the converted color components from the two or more reference layers.

12. The method of claim 7, wherein signaling the at least one reference layer ID comprises signaling a first reference layer ID identifying a first reference layer having first color components of a first reference layer picture to which the 3D lookup table is applied to generate a first inter-layer reference picture for a first enhancement layer picture in the enhancement layer, the method further comprising:

determining a second reference layer ID identifying a second reference layer of the plurality of reference layers of the video data, the second reference layer being different than the first reference layer;

applying the 3D lookup table to second color components of a second reference layer picture in the second reference layer to convert the second color components of the second reference layer picture of the second reference layer;

generating a second inter-layer reference picture for a second enhancement layer picture in the enhancement layer of the video data based on the converted second color components, the second enhancement layer picture being different than the first enhancement layer picture;

encoding video blocks of the second enhancement layer picture based on the second inter-layer reference picture generated using the 3D lookup table; and

signaling the second reference layer ID identifying the second reference layer of the plurality of reference layers of the video data to which the 3D lookup table is applied.

13. A video decoding device comprising:

a memory configured to store video data; and

one or more processors in communication with the memory and configured to;

receive at least one reference layer identifier (ID) for a three-dimensional (3D) lookup table for color gamut scalability, wherein the 3D lookup table is defined for an enhancement layer of the video data, and wherein the at least one reference layer ID identifies at least one reference layer of a plurality of reference layers of the video data to which the 3D lookup table is to be applied;

apply the 3D lookup table to color components of a reference layer picture in the at least one reference layer identified by the at least one reference layer ID to convert the color components from a first color gamut for the reference layers of the video data to a second color gamut for the enhancement layer of the video data;

generate an inter-layer reference picture for an enhancement layer picture in the enhancement layer of the video data based on the converted color components; and

decode video blocks of the enhancement layer picture based on the inter-layer reference picture generated using the 3D lookup table.

14. The device of claim 13, wherein the one or more processors are configured to receive exactly one reference layer ID identifying exactly one reference layer to which the 3D lookup table is to be applied.

15. The device of claim 14, the one or more processors are configured to apply the 3D lookup table to color components of a reference layer picture in the exactly one reference layer identified by the exactly one reference layer ID.

16. The device of claim 13, wherein the one or more processors are configured to receive two or more reference layer IDs identifying two or more reference layers to which the 3D lookup table is to be applied.

17. The device of claim 16, wherein the one or more processors are configured to:

apply the 3D lookup table to color components of a reference layer picture in each of the two or more reference layers identified by the two or more reference layer IDs to convert the color components of the reference layer pictures of the two or more reference layers; and

generate the inter-layer reference picture based on a combination of the converted color components from the two or more reference layers.

18. The device of claim 13, wherein the one or more processors are configured to:

receive a first reference layer ID identifying a first reference layer of the plurality of reference layers having first color components of a first reference layer picture to which the 3D lookup table is applied to generate a first inter-layer reference picture for a first enhancement layer picture in the enhancement layer;

receive a second reference layer ID identifying a second reference layer of the plurality of reference layers of the video data, the second reference layer being different than the first reference layer;

apply the 3D lookup table to second color components of a second reference layer picture in the second reference layer to convert the second color components of the second reference layer picture of the second reference layer;

generate a second inter-layer reference picture for a second enhancement layer picture in the enhancement layer of the video data based on the converted second color components, the second enhancement layer picture being different than the first enhancement layer picture; and

decode video blocks of the second enhancement layer picture based on the second inter-layer reference picture generated using the 3D lookup table.

19. A video encoding device comprising:

a memory configured to store video data; and

one or more processors in communication with the memory and configured to;

determine at least one reference layer of a plurality of reference layers of the video data for a three-dimensional (3D) lookup table for color gamut scalability, wherein the 3D lookup table is defined for an enhancement layer of the video data;

apply the 3D lookup table to color components of a reference layer picture in the at least one reference layer to convert the color components from a first color gamut for the reference layers of the video data to a second color gamut for the enhancement layer of the video data;

generate an inter-layer reference picture for an enhancement layer picture in the enhancement layer of the video data based on the converted color components;

encode video blocks of the enhancement layer picture based on the inter-layer reference picture generated using the 3D lookup table; and

signal at least one reference layer identifier (ID) identifying the at least one reference layer of the plurality of reference layers of the video data to which the 3D lookup table is applied.

20. The device of claim 19, wherein the one or more processors are configured to signal exactly one reference layer ID identifying exactly one reference layer to which the 3D lookup table is applied.

21. The device of claim 20, wherein the one or more processors are configured to:

determine the exactly one reference layer for the 3D lookup table; and

apply the 3D lookup table to color components of a reference layer picture in the exactly one reference layer.

22. The device of claim 19, wherein the one or more processors are configured to signal two or more reference layer IDs identifying two or more reference layers to which the 3D lookup table is applied.

23. The device of claim 22, wherein the one or more processors are configured to:

determine the two or more reference layers for the 3D lookup table;

apply the 3D lookup table to color components of a reference layer picture in each of the two or more reference layers to convert the color components of the reference layer pictures of the two or more reference layers; and

generate the inter-layer reference picture based on a combination of the converted color components from the two or more reference layers.

24. The device of claim 19, wherein the one or more processors are configured to:

signal a first reference layer ID identifying a first reference layer of the plurality of reference layers having first color components of a first reference layer picture to which the 3D lookup table is applied to generate a first inter-layer reference picture for a first enhancement layer picture in the enhancement layer;

determine a second reference layer ID identifying a second reference layer of the plurality of reference layers of the video data, the second reference layer being different than the first reference layer;

apply the 3D lookup table to second color components of a second reference layer picture in the second reference layer to convert the second color components of the second reference layer picture of the second reference layer;

generate a second inter-layer reference picture for a second enhancement layer picture in the enhancement layer of the video data based on the converted second color components, the second enhancement layer picture being different than the first enhancement layer picture;

encode video blocks of the second enhancement layer picture based on the second inter-layer reference picture generated using the 3D lookup table; and

signal the second reference layer ID identifying the second reference layer of the plurality of reference layers of the video data to which the 3D lookup table is applied.

25. A video decoding device comprising:

means for receiving at least one reference layer identifier (ID) for a three-dimensional (3D) lookup table for color gamut scalability, wherein the 3D lookup table is defined for an enhancement layer of the video data, and wherein the at least one reference layer ID identifies at least one reference layer of a plurality of reference layers of the video data to which the 3D lookup table is to be applied;

means for applying the 3D lookup table to color components of a reference layer picture in the at least one reference layer identified by the at least one reference layer ID to convert the color components from a first color gamut for the reference layers of the video data to a second color gamut for the enhancement layer of the video data;

means for generating an inter-layer reference picture for an enhancement layer picture in the enhancement layer of the video data based on the converted color components; and

means for decoding video blocks of the enhancement layer picture based on the inter-layer reference picture generated using the 3D lookup table.

26. A non-transitory computer-readable storage medium storing instructions for decoding video data that, when executed, cause one or more processors to:

receive at least one reference layer identifier (ID) for a three-dimensional (3D) lookup table for color gamut scalability, wherein the 3D lookup table is defined for an enhancement layer of the video data, and wherein the at least one reference layer ID identifies at least one reference layer of a plurality of reference layers of the video data to which the 3D lookup table is to be applied;

apply the 3D lookup table to color components of a reference layer picture in the at least one reference layer identified by the at least one reference layer ID to convert the color components from a first color gamut for the reference layers of the video data to a second color gamut for the enhancement layer of the video data;

generate an inter-layer reference picture for an enhancement layer picture in the enhancement layer of the video data based on the converted color components; and

decode video blocks of the enhancement layer picture based on the inter-layer reference picture generated using the 3D lookup table.