Enhancing dynamic ranges of images

• US 8,135,230 B2
• Filed: 07/29/2008
• Issued: 03/13/2012
• Est. Priority Date: 07/30/2007
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Heidrich, Wolfgang, Seetzen, Helge, Ward, Gregory John, Whitehead, Lorne A., Rempel, Allan G.
• Primary Examiner(s): Patel, Kanjibhai
• Application Number:
  US12/182,121
• Publication Number:
  US 20090034867A1
• Time in Patent Office: 1,323 Days
• Field of Search: 382/169, 382/173, 382/232, 382260-270, 382274-275, 382/283, 382/299, 382/254, 345/63, 345/611, 345/626, 345/660, 345/687, 345/690, 348/235, 348/241, 348/252, 348/533, 353/31, 375/240.19, 375/254
• US Class Current: 382/264
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/30 Erosion or dilatation, e.g. thinning
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G09G 2320/0646 Modulation of illumination source brightness and image signal correlated to each other
• CPC Class Codes: G09G 2320/066 for control of contrast
• CPC Class Codes: G09G 3/3426 the different display panel areas being distributed in two dimensions, e.g. matrix
• CPC Class Codes: G09G 5/10 Intensity circuits

Abstract

Methods and apparatus according to various aspects take as input image data in a lower-dynamic-range (LDR) format and produce as output enhanced image data having a dynamic range greater than that of the input image data (i.e. higher-dynamic range (HDR) image data). In some embodiments, the methods are applied to video data and are performed in real-time (i.e. processing of video frames to enhance the dynamic range of the video frames is completed at least on average at the frame rate of the video signal).

Claims

1. A method for enhancing dynamic range of a first image, the method comprising:

identifying one or more enhancement regions within the first image for which luminance values satisfy an enhancement criterion;

applying a smoothly-varying brightness enhancement function to the first image to obtain a second image having a dynamic range greater than that of the first image, the brightness enhancement function increasing luminance values of the second image over the first image in the enhancement regions and in at least some areas adjacent to the enhancement regions, the effect of the brightness enhancement function dropping off smoothly with increasing distance from the enhancement regions.

2. A method according to claim 1 comprising generating the brightness enhancement function from the first image, wherein generating the brightness enhancement function comprises:

generating a mask based on the first image and blurring the mask.

3. A method according to claim 2 wherein blurring the mask comprises applying a Gaussian blur filter to the mask.

4. A method according to claim 1 wherein applying the brightness enhancement function to the first image comprises, for each pixel in the first image, multiplying a corresponding first image pixel value by a corresponding brightness enhancement factor.

5. A method according to claim 3 wherein the brightness enhancement function has values corresponding to pixels of the image that are in the range of 1 to a where a is in the range of 3 to 12.

6. A method according to claim 2 wherein a spatial spectrum of the brightness enhancement function comprises primarily angular frequencies of 1 cycle per degree or less referenced to an intended viewing location.

7. A method according to claim 2 comprising displaying the second image on a display and wherein a standard deviation of the brightness enhancement function is at least about 0.025 of a horizontal resolution of the display.

8. A method according to claim 1 wherein applying the brightness enhancement function to the first image comprises:

identifying one or more high-gradient edges corresponding to the enhancement regions;

increasing luminance values of the second image over the first image in the areas adjacent to the enhancement regions and closer to their respective enhancement regions than high-gradient edges; and

discontinuing increasing luminance values of the second image over the first image at areas further from their respective enhancement regions than the high-gradient edges.

9. A method according to claim 2 wherein generating the brightness enhancement function comprises generating an edge-stopping component of the brightness enhancement function based at least in part on a gradient image of the first image, the edge-stopping component having pixel values which indicate whether or not application of the brightness enhancement function should increase the luminance of corresponding pixels of the first image.

10. A method according to claim 1 comprising generating the brightness enhancement function, wherein generating the brightness enhancement function comprises:

generating a binarized mask which identifies pixels inside and outside of the one or more enhancement regions;

down-sampling the binarized mask to obtain a down-sampled mask;

for each iteration in a loop comprising N iterations;

applying a blur filter to the down-sampled mask in an initial iteration and to a result of the previous iteration in other iterations; and

up-sampling a result of the blur filter by an up-sampling step;

wherein a result of the N^th iteration of the loop comprises grey-scale image data having the same resolution as the first image data, the grey-scale image data convertable into at least a portion of the brightness enhancement function by suitable mapping.

11. A method according to claim 10 comprising mapping the grey-scale image data to provide a smooth component of the brightness enhancement function, the smooth component having a range of 1-a, where a is a brightness amplification parameter greater than 1.

12. A method according to claim 10 wherein generating the brightness enhancement function comprises generating an edge-stopping component of the brightness enhancement function based at least in part on a gradient image of the first image, the edge-stopping component having pixel values which indicate whether or not application of the brightness enhancement function should increase the luminance of corresponding pixels of the first image.

13. A method according to claim 12 wherein generating the edge-stopping component comprises:

down-sampling the gradient image of the first image in a series of N down-sampling steps to obtain a set of N down-sampled gradient images;

for each iteration in a loop comprising N iterations;

performing a DILATION operation to the down-sampled mask in an initial iteration and to a result of the previous iteration in other iterations; and

up-sampling a result of the DILATION operation by an up-sampling step.

14. A method according to claim 1 comprising generating the brightness enhancement function, wherein generating the brightness enhancement function comprises:

down-sampling the first image to obtain intermediate-resolution down-sampled image data;

further down-sampling the intermediate-resolution down-sampled image data to obtain lower-resolution down-sampled image data;

generating a lower-resolution mask from the lower-resolution down-sampled image data, the lower-resolution mask identifying pixels inside and outside of the one or more enhancement regions; and

for each iteration in a loop comprising N iterations;

applying a blur filter to the lower-resolution mask in an initial iteration and to a result of the previous iteration in other iterations; and

up-sampling a result of the blur filter by an up-sampling step;

wherein a result of the N^th iteration of the loop comprises grey-scale image data having the same resolution as the first image, the grey-scale image data convertable into at least a component of the brightness enhancement function.

15. A method according to claim 14 wherein generating the edge-stopping component comprises:

down-sampling the gradient image of the first image in a series of N down-sampling steps to obtain a set of N down-sampled gradient images;

for each iteration in a first loop comprising M iterations, where M<

N;

performing a DILATION operation to the lower-resolution mask in an initial iteration and to a result of the previous iteration in other iterations; and

up-sampling a result of the DILATION operation by an up-sampling step;

wherein a result of the first loop comprises intermediate-resolution edge-stopping data;

for each iteration in a second loop comprising N-M iterations;

performing the DILATION operation to the intermediate-resolution edge-stopping data in an initial iteration and to a result of the previous iteration in other iterations; and

up-sampling a result of the DILATION operation by an up-sampling step;

wherein a result of the second loop comprises full-resolution edge-stopping data having a resolution corresponding to that of the first image.

16. A method according to claim 14 comprising determining first modulator driving values for a first modulator of a dual modulator display used to display the second image based at least in part on the intermediate-resolution down-sampled image data.

17. A method according to claim 15 comprising determining first modulator driving values for a first modulator of a dual modulator display used to display the second image based at least in part on the intermediate-resolution down-sampled image data and based at least in part on the intermediate-resolution edge-stopping data.

18. A non-transitory computer-readable medium comprising a series of instructions which, when executed by a suitably configured processor, cause the processor to perform the method of claim 1.

19. A method for displaying a first image having a relatively high dynamic range obtained from a second image having relatively low dynamic range, the method comprising:

identifying at least one enhancement region within the second image;

applying a brightness enhancement function to the second image to generate the first image, wherein compared to the second image, the first image comprises increased luminance values of enhancement-region pixels in the enhancement region and increased luminance values of one or more boundary-region pixels outside of the enhancement region, the luminance value increase of the boundary-region pixels decreasing with distance from the enhancement region; and

determining driving signals for at least one modulator of a dual-modulator display based at least in part on intermediate data obtained in the course of application of the brightness enhancement function to the second image.

20. A method according to claim 19 wherein the intermediate data comprises down-sampled data obtained by down-sampling the second image.