Local multiscale tone-mapping operator

• US 9,299,317 B2
• Filed: 03/01/2012
• Issued: 03/29/2016
• Est. Priority Date: 01/07/2008
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Ward, Gregory John
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US13/984,524
• Publication Number:
  US 20130335438A1
• Time in Patent Office: 1,489 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G09G 5/02 characterised by the way in which colour is displayed details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607

Abstract

In a method to generate a tone-mapped image from a high-dynamic range image (HDR), an input HDR image is converted into a logarithmic domain and a global tone-mapping operator generates a high-resolution gray scale ratio image from the input HDR image. Based at least in part on the high-resolution gray scale ratio image, at least two different gray scale ratio images are generated and are merged together to generate a local multiscale gray scale ratio image that represents a weighted combination of the at least two different gray scale ratio images, each being of a different spatial resolution level. An output tone-mapped image is generated based on the high-resolution gray scale image and the local multiscale gray scale ratio image.

Claims

1. A method comprising:

converting luminance values in an input high dynamic range (HDR) image into a logarithmic domain;

applying a global ratio operator to the luminance values in the logarithmic domain derived from the input high dynamic range (HDR) image to generate a high resolution gray scale ratio image in the logarithmic domain comprising luminance ratio values;

generating, based at least in part on the high resolution gray scale ratio image in the logarithmic domain, at least two different gray scale ratio images, merging the at least two different gray scale ratio images to generate a local multiscale gray scale ratio image that comprises a weighted combination of the at least two different gray scale ratio images, each being of a different spatial resolution level;

wherein the local multiscale gray scale ratio image is obtained using recursive processing;

using ratio values in the local multiscale gray scale ratio image with the luminance values in the logarithmic domain derived from the input HDR image to generate a tone-mapped gray scale image;

wherein the tone-mapped gray scale image comprises a fixed-point grayscale image with the same spatial dimensions as the spatial dimensions of the input HDR image; and

computing a reference maximum over the tone-mapped gray scale image such that a percentage of pixels that lie outside a color gamut are not more than a threshold.

2. The method of claim 1, wherein the global ratio operator comprises a global luminance histogram adjustment operator in connection with a global tone mapping operator.

3. The method of claim 1, wherein the at least two different gray scale ratio images include the high resolution gray scale ratio image.

4. The method of claim 1, wherein the high resolution gray scale ratio image is a gray scale ratio image with a same spatial resolution level as that of the HDR image.

5. The method of claim 1, wherein the at least two different gray scale ratio images include at least one gray scale ratio image determined based on the high resolution gray scale ratio image by downsampling luminance ratio values in the high resolution gray scale ratio image.

6. The method of claim 1, wherein tone-mapped luminance values in the tone-map gray scale image comprise a reduced dynamic range lower than that of the luminance values in the input HDR image.

7. The method of claim 1, wherein the input HDR image is one of a floating-point RGB image or a fixed-point RGB image.

8. The method of claim 1, wherein tone-mapped luminance values in the tone-mapped gray scale image are in a gamma-encoding scale.

9. The method of claim 1, further comprising:

computing, based on the luminance values derived from the input HDR image, a histogram adjustment curve; and

applying the histogram adjustment curve to the luminance values derived from the input HDR image to generate the high resolution gray scale ratio image.

10. The method of claim 1, wherein, in a non-logarithmic domain, the luminance ratio values in the high resolution gray scale ratio image comprise a plurality of ratio values each of which is a globally tone-mapped luminance value of a global tone-mapped image as divided by one of the luminance values derived from the input HDR image.

11. The method of claim 1, further comprising assigning at least two weighting factors to the at least two different gray scale ratio images.

12. The method of claim 1, further comprising:

assigning values of a plurality of weighting factors to a plurality of gray scale ratio images that are used to generate the local multiscale gray scale ratio image, wherein the plurality of gray scale ratio images includes the at least two gray scale ratio images, and wherein the values of the plurality of weighting factors are normalized to a preconfigured value.

13. The method of claim 12, further comprising assigning at least two weighting factors to the at least two different gray scale ratio images.

14. The method of claim 1, further comprising:

assigning a first value to a first weight factor for a first gray scale ratio image in a plurality of gray scale ratio images that are used to generate the local multiscale gray scale ratio image; and

assigning a second value to a second weight factor for a second gray scale ratio image in the plurality of gray scale ratio images;

wherein the first value is higher than the second value; and

wherein the first gray scale ratio image has a higher spatial resolution level than the second gray scale ratio image.

15. The method of claim 1, wherein the tone-mapped gray scale image derived from the local multiscale gray scale ratio image and the input HDR image is used as a component image to derive a tone-mapped color image.

16. The method of claim 15, wherein the tone-mapped color image without the local multiscale gray scale ratio image is used for rendering on an image rendering device.

17. The method of claim 15, wherein the tone-mapped color image with the local multiscale gray scale ratio image is provided to an image processing system for rendering on an image rendering device.

18. The method of claim 17, wherein the tone-mapped color image and the local multiscale gray scale ratio image are used by the image processing system to generate an output HDR image.

19. An apparatus comprising a processor and configured to perform any one of the methods recited in claim 1.

20. A non-transitory computer readable storage medium, comprising software instructions, which when executed by one or more processors cause performance of any one of the methods recited in claim 1.

21. A method comprising:

applying a global tone mapping operator to luminance values derived from a high dynamic range (HDR) image to generate a high resolution gray scale ratio image comprising luminance ratio values;

determining a number of spatial resolution levels in between a highest resolution level and a lowest resolution level;

for each determined spatial resolution level from the highest resolution level to a spatial resolution level immediately above the lowest resolution level, downsampling a ratio image corresponding to the determined spatial resolution level to generate a succeeding ratio image corresponding to a spatial resolution level immediately lower than the determined spatial resolution level;

for each determined spatial resolution level from the lowest resolution level to a spatial resolution level immediately below the highest resolution level, upsampling a ratio image corresponding to the determined spatial resolution level to generate an upsample ratio image corresponding to the each spatial resolution level;

determining a weight factor value for each different ratio image of a number of ratio images each corresponding to a spatial resolution level of the number of determined spatial resolution levels, thereby giving rise to a number of weight factors;

determining a local multiscale ratio image as a weighted combination of the number of ratio images as weighted by the number of weight factors.

22. The method of claim 21, wherein the number of spatial resolution levels is determined without performing image analysis over content of the input HDR image.

23. The method of claim 21, wherein the number of spatial resolution levels is determined based at least in part on a result of image analysis content of the input HDR image.