Multi-Layer Backwards-Compatible Video Delivery for Enhanced Dynamic Range and Enhanced Resolution Formats

• US 20130314495A1
• Filed: 05/10/2013
• Published: 11/28/2013
• Est. Priority Date: 05/24/2012
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Chen, Tao, Hulyalkar, Samir N.
• Granted Patent: US 9,357,197 B2
• Time in Patent Office: Days
• US Class Current: 348/43
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding

Abstract

A sequence of enhanced dynamic range (EDR) images and a sequence of standard dynamic range images are encoded using a backwards-compatible SDR high-definition (HD) base layer and one or more enhancement layers. The EDR and SDR video signals may be of the same resolution (e.g., HD) or at different resolutions (e.g., 4K and HD) and are encoded using a dual-view-dual-layer (DVDL) encoder to generate a coded base layer (BL) and a coded enhancement layer (EL). The DVDL encoder includes a reference processing unit (RPU) which is adapted to compute a reference stream based on the coded BL stream. The RPU operations include post-processing, normalization, inverse normalization, and image registration. Decoders for decoding the coded BL and EL streams to generate a backwards compatible 2D SDR stream and additional 2D or 3D SDR or EDR streams, are also described.

Claims

1. A method comprising:

accessing a first video signal (302) and a second video signal (304), wherein the second video signal has higher resolution and higher dynamic range than the first video signal; and

encoding the first video signal and the second video signal using a dual-view-dual-layer (DVDL) encoder (300) to generate a coded base layer (BL) signal (312) and a coded enhancement layer (EL) signal (334), wherein the DVDL encoder comprises a reference processing unit (RPU) (320), the RPU unit computing a third video signal (324) based on the first video signal, wherein computing the third video signal comprises performing a standard dynamic range (SDR) to enhanced dynamic range (EDR) prediction step (530) and an image registration step (540).

2. The method of claim 1, wherein the image registration step comprises:

accessing an input image (535); and

transforming the input image (535) to generate an output image (560), wherein the output image matches the size and characteristics of a target image.

3. The method of claim 2, wherein transforming the input image comprises scaling an active area of the input image to match the size of an active area of the target image.

4. The method of claim 1, wherein the first signal is a standard dynamic range (SD) signal at a high-definition resolution and the second signal is an EDR signal at a 4K resolution.

16. An encoder performing the method as recited in claim 1.

5. A method comprising:

accessing a coded BL stream (312) and a coded EL stream (334);

decoding the coded BL stream and the coded EL stream with a DVDL decoder (350) to generate a first video signal (362) and a second video signal (384), wherein the second video signal has higher resolution and higher dynamic range than the first video signal, wherein the DVDL decoder comprises an RPU (370), the RPU unit computing a third video signal (374) based on the first video signal, wherein computing the third video signal comprises an SDR to EDR prediction step (530) and an image registration step (540).

6. A method comprising:

accessing an SDR video signal (902), a first view of a first 3D EDR video signal (904), and a second view of a second 3D EDR video signal (906), wherein the second view of the second 3D EDR video signal has a higher resolution than the first view of the first 3D EDR video signal;

applying a normalization process to the SDR video signal and the first view of the first EDR video signal to generate a normalized SDR* signal (912);

encoding the normalized SDR* signal with an encoder (920) to generate a coded BL stream (922). applying an inverse normalization process and an image registration process to a signal based on the normalized SDR* signal to generate an estimate EDR video signal with the same resolution as the second view of the second 3D EDR video signal; and

encoding using a DVDL encoder the estimate EDR video signal and the second view of the second EDR video signal to generate a coded EL stream (942).

7. The method of claim 6, wherein the SDR signal has an HD resolution, the first view of the first EDR signal has an HD resolution, and the second view of the second EDR signal has a 4K resolution.

17. An encoder performing the method as recited in claim 6.

19. A decoder performing the method as recited in claim 6.

8. A method comprising:

accessing a coded BL stream (922) and a coded EL stream (942);

decoding the coded BL stream with a first video decoder (980) to generate a first normalized (SDR*) video stream (952);

applying an inverse normalization process (960) and an image registration process to the first normalized video stream to generate an SDR output video stream (964) and a first view of an EDR output video stream (962-1), wherein the first view of the EDR output video stream has higher resolution than the resolution of the SDR output video stream;

decoding with a second video decoder (990) the coded EL stream (942) to generate a second view of the EDR video stream, wherein the decoding by the second video decoder is based on both the coded BL stream and the first view of the EDR stream.

9. The method of claim 8 wherein the SDR output video has an HD resolution and the EDR output video has a 4K resolution.

10. The method of claim 8 wherein the first video decoder is an AVC decoder and the second video decoder is an HEVC decoder.

15. A method comprising:

receiving a coded BL stream (1022), a first coded EL stream (1042), and a second coded EL stream (1044) encoded according to the method of claim 9;

decoding the coded BL stream using a first decoder (1080) to generate a first view of an SDR stream (1062-1);

using an RPU and a first second decoder, generating a second view of the SDR stream (1062-2) based on the second coded EL stream and an RPU output;

using the RPU and the second video decoder, generating an EDR video stream (1064) based on the first coded EL stream and the RPU output, wherein the EDR video stream has a higher resolution than the SDR stream.

20. A decoder performing the method as recited in claim 8.

21. A decoder performing the method as recited in claim 15.

11. A method comprising:

accessing a 3D SDR video stream comprising a first view (1002-1) and a second view (1002-2);

accessing one view of a 3D EDR video stream (1004), where the 3D EDR video stream has a higher resolution than the 3D SDR stream;

encoding with a first encoder (1022) the first view of the 3D SDR video stream to generate a coded BL stream (1022);

decoding (1030) the coded BL stream to generate a decoded BL video stream (1032);

generating a reconstructed SDR stream (1016) based on the decoded BL video stream;

applying an inverse normalization process and an image registration process to the reconstructed SDR stream to generate a second EDR signal (1014);

using a second video encoder, generating a first coded EL stream (1042) based on the one view of the 3D EDR video stream (1004) and the second EDR signal (1014); and

using the second video encoder, generating a second coded EL stream (1044) based on the second view of the 3D SDR video stream (1002-2) and the reconstructed SDR stream (1016).

12. The method of claim 11 wherein the first encoder is an AVC encoder and the second encoder is an HEVC encoder.

13. The method of claim 11 wherein the 3D SDR video stream has an HD resolution and the 3D EDR video stream has a 4K resolution.

14. The method of claim 11 wherein the 3D SDR stream and 3D EDR stream represent the same scene but at a different dynamic range.

18. An encoder performing the method as recited in claim 11.