Encoding, decoding, and representing high dynamic range images

• US 8,334,911 B2
• Filed: 07/19/2012
• Issued: 12/18/2012
• Est. Priority Date: 04/15/2011
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Jia, Wenhui, Ninan, Ajit, Ten, Arkady, Ward, Gregory John
• Primary Examiner(s): Tran, Nhan T
• Application Number:
  US13/553,440
• Publication Number:
  US 20120281009A1
• Time in Patent Office: 152 Days
• Field of Search: 348/223.1, 348/222.1, 348/234, 348/235, 348/557, 348/663, 348/712, 348/273, 348/280, 382/254, 382/274, 382/162, 345/589, 345/590, 375/240.25, 375/240.26
• US Class Current: 348/223.1
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/169 characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/50 using predictive coding H04N19/61 takes precedence
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 9/67 for matrixing camera processing pipelines for matrixing of colour signals H04N23/85
• CPC Class Codes: H04N 9/77 Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase circuits for matrixing H04N9/67

Abstract

Techniques are provided to encode and decode image data comprising a tone mapped (TM) image with HDR reconstruction data in the form of luminance ratios and color residual values. In an example embodiment, luminance ratio values and residual values in color channels of a color space are generated on an individual pixel basis based on a high dynamic range (HDR) image and a derivative tone-mapped (TM) image that comprises one or more color alterations that would not be recoverable from the TM image with a luminance ratio image. The TM image with HDR reconstruction data derived from the luminance ratio values and the color-channel residual values may be outputted in an image file to a downstream device, for example, for decoding, rendering, and/or storing. The image file may be decoded to generate a restored HDR image free of the color alterations.

Claims

1. A device, comprising:

an input interface configured to access a high dynamic range (HDR) image;

a processor configured to execute instructions to;

generate a luminance ratio image based on luminance pixel values of the HDR image and a tone-mapped (TM) image, the TM image generated based on the HDR image;

generate a re-mapped HDR image based on the luminance ratio image and the HDR image;

generate residual values in color channels of a color space based on the re-mapped image and the TM image; and

an output interface configured to output the TM image with HDR reconstruction data, the HDR reconstruction data being derived from the luminance ratio image and the residual values.

2. The device of claim 1, wherein the processor is further configured to quantize the residual values to generate quantized residual values.

3. The device of claim 2, wherein the processor is further configured to:

determine logarithms of one or more pixels of the luminance ratio image to generate a log ratio image; and

to

quantize one or more pixel values of the log ratio image to generate a quantized ratio image.

4. The device of claim 3, wherein the processor is further configured to:

store the quantized ratio image and the quantized residual values into a single picture container, wherein the quantized ratio image comprises the luminance values of the single picture container and the quantized residual values comprise the chroma values of the single picture container.

5. The device of claim 1 wherein the processor generates a pixel of the luminance ratio image by dividing the luminance value of a corresponding pixel in the HDR image with the luminance value of a corresponding pixel in the TM image.

6. The device of claim 1, wherein the processor is further configured to compress the TM image to generate a compressed TM image.

7. The device of claim 6, wherein the TM image is compressed using a JPEG compression algorithm.

8. The device of claim 1, wherein the TM image and the HDR reconstruction data are stored together in an image file.

9. The device of claim 8, wherein the image file is a JPEG image file, the JPEG image file comprising a JPEG image based on the TM image and application data based on the HDR reconstruction data, the application data separated by the JPEG image by a JPEG application marker.

10. The device of claim 1, wherein the input interface is further configured to access the tone-mapped image.

11. A decoder, comprising:

an input interface configured to access an image file comprising a tone-mapped (TM) base image and high dynamic range (HDR) reconstruction data, the HDR reconstruction data comprising a luminance ratio image and residual values in color channels of a color space;

a processor configured to generate a first HDR image based on the tone-mapped base image and the luminance ratio image; and

an output interface configured to output an output HDR image, the output HDR image reconstructed by the processor using the first HDR image and the residual values in the color channels of the color space.

12. The decoder of claim 11, wherein reconstructing the output HDR image further comprises adding chroma pixel data based on the residual values in the color channels to chroma pixel values based on the first HDR image.

13. The decoder of claim 11, wherein the processor is further configured to generate a decoded baseline image by decoding the TM base image using JPEG decoding.

14. The decoder of claim 11, wherein the processor is further configured to:

dequantize the ratio image and the residual values to generate a dequantized ratio image and dequantized residual values, and

determine inverse logarithms of pixel values in the dequantized ratio image.

15. The decoder of claim 11, wherein generating the first HDR image with the processor further comprises multiplying, on individual pixel basis, pixel values based on the TM image with corresponding pixel values in the luminance ratio image.