Encoding, decoding and representing high dynamic range images

• US 8,218,625 B2
• Filed: 11/24/2004
• Issued: 07/10/2012
• Est. Priority Date: 04/23/2004
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Ward, Gregory John, Simmons, Maryann
• Primary Examiner(s): Ustaris, Joseph
• Assistant Examiner(s): Itskovich, Mikhail
• Application Number:
  US11/568,030
• Publication Number:
  US 20080192819A1
• Time in Patent Office: 2,785 Days
• Field of Search: None
• US Class Current: 375/240.03
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G09G 5/10 Intensity circuits
• CPC Class Codes: H04N 1/3871 the composed originals being of different kinds, e.g. low- and high-resolution originals
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/463 by compressing encoding parameters before transmission
• CPC Class Codes: H04N 19/98 Adaptive-dynamic-range coding [ADRC]

Abstract

A data structure defining a high dynamic range image comprises a tone map having a reduced dynamic range and HDR information. The high dynamic range image can be reconstructed from the tone map and the HDR information. The data structure can be backwards compatible with legacy hardware or software viewers. The data structure may comprise a JFIF file having the tone map encoded as a JPEG image with the HDR information in an application extension or comment field of the JFIF file, or a MPEG file having the tone map encoded as a MPEG image with the HDR information in a video or audio channel of the MPEG file. Apparatus and methods for encoding or decoding the data structure may apply pre- or post correction to compensate for lossy encoding of the high dynamic range information.

Claims

1. A method for encoding high dynamic range image data having an initial dynamic range, the method comprising:

obtaining tone map data corresponding to the high dynamic range image data, the tone map data having a reduced dynamic range lower than the initial dynamic range;

processing the high dynamic range image data and the tone map data by a machine to yield ratio data, the ratio data comprising ratios of values in the high dynamic range image data and corresponding values in the tone map data;

generating high dynamic range information based on the ratio data;

generating tone map information based on the tone map data; and

,

storing the high dynamic range information and the tone map information in a data structure;

wherein generating the high dynamic range information comprises applying to the ratio data a correction function, the correction function based on the tone map data;

the correction function is of the form;

2. A method according to claim 1 wherein the data structure comprises an image field and an application extension field and storing the high dynamic range information and the tone map information comprises storing the high dynamic range information in the application extension field and storing the tone map information in the image field.

3. A method according to claim 1 wherein generating the tone map information comprises JPEG encoding the tone map data.

4. A method according to claim 3 wherein the data structure comprises a JFIF data structure.

5. A method according to claim 4 comprising storing the high dynamic range information in an application extension of the JFIF data structure.

6. A method according to claim 4 comprising storing the high dynamic range information in a comment field of the JFIF data structure.

7. A method according to claim 6 comprising encoding the high dynamic range information as text before storing the high dynamic range information in the comment field.

8. A method according to claim 4 wherein generating the high dynamic range information comprises compressing the ratio data.

9. A method according to claim 8 wherein compressing the ratio data comprises downsampling the ratio data.

10. A method according to claim 8 wherein compressing the ratio data comprises JPEG encoding the ratio data.

11. A method according to claim 1 wherein generating the tone map information comprises MPEG encoding the tone map data.

12. A method according to claim 11 wherein the data structure comprises a MPEG data structure.

13. A method according to claim 12 comprising storing the high dynamic range information in an application extension of the MPEG data structure.

14. A method according to claim 12 comprising storing the high dynamic range information in a comment field of the MPEG data structure.

15. A method according to claim 13 comprising storing the high dynamic range information for each frame of a MPEG video.

16. A method according to claim 13 comprising storing the high dynamic range information only for keyframes of a MPEG video for which conventional MPEG keyframe interpolation techniques are used to create inter-keyframe frames.

17. A method according to claim 11 wherein generating the high dynamic range information comprises compressing the ratio data.

18. A method according to claim 17 wherein compressing the ratio data comprises downsampling the ratio data.

19. A method according to claim 17 wherein compressing the ratio data comprises MPEG encoding the ratio data.

20. A method according to claim 1 wherein the tone map data is encoded by lossy encoding and the method comprises decoding the tone map data and subsequently computing the ratio data from the decoded tone map data.

21. A method according to claim 20 comprising encoding the tone map data by the lossy encoding method prior to decoding the tone map data.

22. A method according to claim 20 wherein storing the tone map information comprises storing the encoded tone map data.

23. A method according to claim 22 wherein generating the high dynamic range information comprises compressing the ratio data to yield compressed ratio data.

24. A method according to claim 23 wherein compressing the ratio data comprises performing a lossy compression of the ratio data.

25. A method according to claim 24 wherein compressing the ratio data comprises downsampling the ratio data.

26. A method according to claim 24 wherein compressing the ratio data comprises JPEG encoding the ratio data.

27. A method according to claim 24 wherein compressing the ratio data comprises MPEG encoding the ratio data.

28. A method according to claim 23 wherein generating the tone map information comprises reconstructing the ratio data from the compressed ratio data and dividing the high dynamic range image data by the reconstructed ratio data.

30. A method according to claim 1 wherein generating the high dynamic range information comprises computing a mathematical function of the ratio data.

31. A method according to claim 30 wherein the mathematical function comprises a logarithm computation.

32. A method according to claim 1 wherein obtaining the tone map data comprises generating the tone map data with a tone mapper without chopping colour or luminance values.

33. A method according to claim 1 wherein obtaining the tone map data comprises generating the tone map data with a tone mapper which maintains colour and luminance ratios for each pixel of the tome map data.

35. A method according to claim 14 comprising storing the high dynamic range information for each frame of a MPEG video.

36. A method according to claim 14 comprising storing the high dynamic range information only for keyframes of a MPEG video for which conventional MPEG keyframe interpolation techniques are used to create inter-keyframe frames.

29. A method for encoding high dynamic range image data having an initial dynamic range, the method comprising:

obtaining tone map data corresponding to the high dynamic range image data, the tone map data having a reduced dynamic range lower than the initial dynamic range;

processing the high dynamic range image data and the tone map data by a machine to yield ratio data, the ratio data comprising ratios of values in the high dynamic range image data and corresponding values in the tone map data;

generating high dynamic range information based on the ratio data;

generating tone map information based on the tone map data; and

,

storing the high dynamic range information and the tone map information in a data structure;

wherein;

the tone map data is encoded by lossy encoding and the method comprises decoding the tone map data and subsequently computing the ratio data from the decoded tone map data;

storing the tone map information comprises storing the encoded tone map data;

generating the high dynamic range information comprises compressing the ratio data to yield compressed ratio data;

generating the high dynamic range information comprises applying to the ratio data a correction function, the correction function based on the tone map data;

the correction function is of the form;

34. Apparatus for encoding high dynamic range image data having an initial dynamic range, the apparatus comprising:

means for computing ratio data, the ratio data comprising ratios of values in the high dynamic range image data to corresponding values in tone map data corresponding to the high dynamic range image data, the tone map data having a reduced dynamic range lower than the initial dynamic range;

means for generating high dynamic range information based on the ratio data;

means for generating tone map information based on the tone map data; and

,

means for storing the high dynamic range information and the tone map information in a data structure;

wherein generating the high dynamic range information comprises applying to the ratio data a correction function, the correction function based on the tone map data;

the correction function is of the form;