Apparatus and methods for encoding, decoding and representing high dynamic range images

• US 9,412,156 B2
• Filed: 09/17/2014
• Issued: 08/09/2016
• Est. Priority Date: 04/23/2004
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Simmons, Maryann, Ward, Gregory John
• Primary Examiner(s): Itskovich, Mikhail
• Application Number:
  US14/488,488
• Publication Number:
  US 20150003537A1
• Time in Patent Office: 692 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G09G 5/10 Intensity circuits
• CPC Class Codes: H04N 1/3871 the composed originals being of different kinds, e.g. low- and high-resolution originals
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/463 by compressing encoding parameters before transmission
• CPC Class Codes: H04N 19/98 Adaptive-dynamic-range coding [ADRC]

Abstract

A data structure defining a high dynamic range image comprises a tone map having a reduced dynamic range and HDR information. The high dynamic range image can be reconstructed from the tone map and the HDR information. The data structure can be backwards compatible with legacy hardware or software viewers. The data structure may comprise a JFIF file having the tone map encoded as a JPEG image with the HDR information in an application extension or comment field of the JFIF file, or a MPEG file having the tone map encoded as a MPEG image with the HDR information in a video or audio channel of the MPEG file. Apparatus and methods for encoding or decoding the data structure may apply pre- or post correction to compensate for lossy encoding of the high dynamic range information.

Claims

1. A method for decoding a high dynamic range (HDR) image, comprising:

receiving a representation of the high dynamic range image in a compressed data structure, the representation of the high dynamic range image including a low dynamic range image and HDR image data, wherein the HDR image data include output data of a nonlinear function applied to ratios between values of a parameter as determined from the high dynamic range image and the low dynamic range image, wherein the low dynamic range image has a dynamic range that is lower than the high dynamic range image;

decompressing the compressed data structure to retrieve the low dynamic range image and the HDR image data, wherein the decompressing comprises dequantizing the compressed data structure;

after decompressing the compressed data structure, applying an inverse nonlinear function to the HDR image data to generate reconstructed HDR image data, wherein the reconstructed HDR image data include ratio data representing the ratios between the values of the parameter as determined from the high dynamic range image and the low dynamic range image; and

after applying the inverse nonlinear function to the HDR image data to generate the reconstructed HDR image data, computing, as a multiplicative product of the low dynamic range image and corresponding ratio data of the reconstructed HDR image data, a reconstructed high dynamic range image.

2. The method of claim 1, wherein decompressing the compressed data structure comprises decompressing the compressed data structure to generate a data structure, wherein the data structure is an MPEG file.

3. The method of claim 1, comprising decoding the HDR image data from the compressed data structure.

4. The method of claim 1, comprising downsampling the low dynamic range image.

5. The method of claim 1, comprising applying a postcorrection function to the HDR image data to generate corrected HDR image data.

6. The method of claim 5, comprising determining that a flag included in the compressed data structure indicates that precorrection was performed on the HDR image data, and

wherein applying the postcorrection function to the HDR image data to generate corrected HDR image data comprises applying the postcorrection function to the HDR image data to generate corrected HDR image data in response to determining that the flag included in the compressed data structure indicates that precorrection was performed on the HDR image data.

7. The method of claim 1, wherein the values of the parameter as determined from the low dynamic range image include luminance values of the low dynamic range image.

8. The method of claim 1, wherein the values of the parameter as determined from the low dynamic range image include chrominance values of the low dynamic range image.

9. The method of claim 1, wherein a spatial resolution of the low dynamic range image is higher than a spatial resolution of the HDR image data.

10. The method of claim 1, wherein the ratios include ratios between the values of the parameter as determined from the low dynamic range image for individual pixels of the low dynamic range image and luminance values for corresponding pixels of the high dynamic range image.

11. A decoder for decoding a high dynamic range (HDR) image, the decoder comprising:

one or more processors; and

a storage medium coupled to the one or more processors having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

receiving a representation of the high dynamic range image in a compressed data structure, the representation of the high dynamic range image including a low dynamic range image and HDR image data, wherein the HDR image data include output data of a nonlinear function applied to ratios between values of a parameter as determined from the high dynamic range image and the low dynamic range image, and wherein the low dynamic range image has a dynamic range that is lower than the high dynamic range image;

decompressing the compressed data structure to retrieve the low dynamic range image and the HDR image data, wherein the decompressing comprises dequantizing the compressed data structure;

after decompressing the compressed data structure, applying an inverse nonlinear function to the HDR image data to generate reconstructed HDR image data, wherein the reconstructed HDR image data include ratio data representing the ratios between the values of the parameter as determined from the high dynamic range image and the low dynamic range image; and

after applying the inverse nonlinear function to the HDR image data to generate the reconstructed HDR image data, computing, as a multiplicative product of the low dynamic range image and corresponding ratio data of the reconstructed HDR image data, a reconstructed high dynamic range image.

12. The decoder of claim 11, wherein decompressing the compressed data structure comprises decompressing the compressed data structure to generate a data structure, wherein the data structure is an MPEG file.

13. The decoder of claim 11, wherein the storage medium further stores instructions when executed by the one or more processors, cause the one or more processors to perform operations comprising decoding the HDR image data from the compressed data structure.

14. The decoder of claim 11, wherein the storage medium further stores instructions when executed by the one or more processors, cause the one or more processors to perform operations comprising downsampling the low dynamic range image.

15. The decoder of claim 11, wherein the values of the parameter as determined from the low dynamic range image include luminance values of the low dynamic range image.

16. The decoder of claim 11, wherein the values of the parameter as determined from the low dynamic range image include chrominance values of the low dynamic range image.

17. A non-transitory computer storage medium having instructions stored thereon which, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving a representation of the high dynamic range image in a compressed data structure, the representation of the high dynamic range image including a low dynamic range image and HDR image data, wherein the HDR image data include output data of a nonlinear function applied to ratios between values of a parameter as determined from the high dynamic range image and the low dynamic range image, and wherein the low dynamic range image has a dynamic range that is lower than the high dynamic range image;

decompressing the compressed data structure to retrieve the low dynamic range image and the HDR image data, wherein the decompressing comprises dequantizing the compressed data structure;

after decompressing the compressed data structure, applying an inverse nonlinear function to the HDR image data to generate reconstructed HDR image data, wherein the reconstructed HDR image data include ratio data representing the ratios between the values of the parameter as determined from the high dynamic range image and the low dynamic range image; and

after applying the inverse nonlinear function to the HDR image data to generate the reconstructed HDR image data, computing, as a multiplicative product of the low dynamic range image and corresponding ratio data of the reconstructed HDR image data, a reconstructed high dynamic range image.

18. The computer storage medium of claim 17, wherein decompressing the compressed data structure comprises decompressing the compressed data structure to generate a data structure, wherein the data structure is an MPEG file.

19. The computer storage medium of claim 17, wherein the computer storage medium further stores instructions when executed by the one or more processors, cause the one or more processors to perform operations comprising decoding the HDR image data from the compressed data structure.

20. The computer storage medium of claim 17, wherein the values of the parameter as determined from the low dynamic range image include luminance values of the low dynamic range image.