Enhancing dynamic ranges of images

• US 8,582,913 B2
• Filed: 06/04/2012
• Issued: 11/12/2013
• Est. Priority Date: 07/30/2007
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Rempel, Allan G., Heidrich, Wolfgang, Seetzen, Helge, Whitehead, Lorne A., Ward, Gregory John
• Primary Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US13/488,228
• Publication Number:
  US 20120237136A1
• Time in Patent Office: 526 Days
• Field of Search: 382/169, 382/173, 382/232, 382/254, 382260-270, 382274-275, 382/283, 382/299, 382/305, 382/312, 345/611, 345/626, 345/660, 345/687, 345/690, 348/235, 348/241, 348/252, 348/533, 375/240.19, 375/254
• US Class Current: 382/263
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G09G 2320/0646 Modulation of illumination source brightness and image signal correlated to each other
• CPC Class Codes: G09G 2320/066 for control of contrast
• CPC Class Codes: G09G 3/3426 the different display panel areas being distributed in two dimensions, e.g. matrix
• CPC Class Codes: G09G 5/10 Intensity circuits
• CPC Class Codes: H04N 5/20 Circuitry for controlling amplitude response
• CPC Class Codes: H04N 9/3126 for spatial light modulators in series

Abstract

Methods and apparatus according to various aspects take as input image data in a lower-dynamic-range (LDR) format and produce as output enhanced image data having a dynamic range greater than that of the input image data (i.e. higher-dynamic range (HDR) image data). In some embodiments, the methods are applied to video data and are performed in real-time (i.e. processing of video frames to enhance the dynamic range of the video frames is completed at least on average at the frame rate of the video signal).

Claims

1. A controller for enhancing lower-dynamic range (LDR) image data representing an image, the controller configured to:

identify at least one enhancement region within the image;

apply a brightness enhancement function to the LDR image data to generate higher-dynamic range (HDR) image data, wherein compared to the LDR image data, the HDR image data comprises increased luminance values of enhancement-region pixels in the enhancement region and increased luminance values of one or more boundary-region pixels outside of the enhancement region, the luminance value increase of the boundary-region pixels decreasing with distance from the enhancement region.

2. A controller according to claim 1 wherein the controller is configured to identify the at least one enhancement region within the image by evaluating luminance values of pixels in the LDR image data using an enhancement criterion.

3. A controller according to claim 2 wherein the enhancement criterion comprises a luminance value threshold above which pixels are identified to be in the enhancement region and below which pixels are identified to be outside of the enhancement region.

4. A controller according to claim 1 wherein the controller is configured to generate the brightness enhancement function, the controller configured to generate the brightness enhancement function by:

generating a binarized mask which identifies pixels inside and outside of the enhancement region; and

applying a blur filter to the binarized mask to obtain grey-scale image data.

5. A controller according to claim 4 wherein the blur filter comprises a smoothly varying function.

6. A controller according to claim 5 wherein the smoothly varying function comprises at least one of:

a Gaussian shape; and

an approximately Gaussian shape.

7. A controller according to claim 4 wherein a spatial angular frequency of the blur filter is less than or equal to 1 cycle per degree.

8. A controller according to claim 4 wherein a standard deviation of the blur filter is at least about 0.025 of a horizontal resolution of a display on which the HDR image data is to be displayed.

9. A controller according to claim 4 wherein the controller is configured to map the grey-scale image data to provide a smooth component of the brightness enhancement function, the smooth component having a range of 1-α

, where α

is a brightness amplification parameter greater than 1.

10. A controller according to claim 9 wherein the controller is configured to generate the brightness enhancement function by generating an edge-stopping component of the brightness enhancement function based at least in part on a gradient image of the LDR image data, the edge-stopping component having pixel values which indicate whether or not application of the brightness enhancement function should increase the luminance of corresponding pixels of the LDR image data.

11. A controller according to claim 10 wherein the controller is configured to apply the brightness enhancement function to the LDR image data by applying the smooth component to the LDR image data only for pixels where the edge-stopping component indicates that application of the brightness enhancement function should increase the luminance of the pixels.

12. A controller according to claim 11 wherein the controller is configured to apply the smooth component to the LDR image data only for pixels where the edge-stopping component indicates that application of the brightness enhancement function should increase the luminance of the pixels comprises pixel-wise multiplication of the smooth component with the LDR image data.

13. A controller according to claim 1 wherein the controller is configured to generate the brightness enhancement function, the controller configured to generate the brightness enhancement function by:

generating a binarized mask which identifies pixels inside and outside of the enhancement region;

down-sampling the binarized mask to obtain a down-sampled mask;

for each iteration in a loop comprising N iterations;

applying a blur filter to the down-sampled mask in an initial iteration and to a result of the previous iteration in other iterations; and

up-sampling a result of the blur filter by an up-sampling step;

wherein a result of the Nth iteration of the loop comprises grey-scale image data having the same resolution as the LDR image data.

14. A controller according to claim 13 wherein the controller is configured to map the grey-scale image data to provide a smooth component of the brightness enhancement function, the smooth component having a range of 1-α

, where α

is a brightness amplification parameter greater than 1.

15. A controller according to claim 14 wherein the controller is configured to apply the brightness enhancement function to the LDR image data by performing pixel-wise multiplication of the smooth component with the LDR image data.

16. A controller according to claim 13 wherein the controller is configured to generate the brightness enhancement function by generating an edge-stopping component of the brightness enhancement function based at least in part on a gradient image of the LDR image data, the edge-stopping component having pixel values which indicate whether or not application of the brightness enhancement function should increase the luminance of corresponding pixels of the LDR image data.

17. A controller according to claim 14 wherein the controller is configured to generate the brightness enhancement function by generating an edge-stopping component of the brightness enhancement function based at least in part on a gradient image of the LDR image data, the edge-stopping component having pixel values which indicate whether or not application of the brightness enhancement function should increase the luminance of corresponding pixels of the LDR image data.

18. A controller for enhancing dynamic range of a first image, the controller configured to:

identify one or more enhancement regions within the first image for which luminance values satisfy an enhancement criterion; and

apply a smoothly-varying brightness enhancement function to the first image to obtain a second image having a dynamic range greater than that of the first image, the brightness enhancement function increasing luminance values of the second image over the first image in the enhancement regions and in at least some areas adjacent to the enhancement regions, the effect of the brightness enhancement function dropping off smoothly with increasing distance from the enhancement regions.

19. A controller according to claim 18, wherein the controller is configured to generate the brightness enhancement function from the first image, the controller configured to generate the brightness enhancement function by:

generating a mask based on the first image and blurring the mask.

20. A controller according to claim 18 wherein the controller is configured to apply the brightness enhancement function to the first image by:

identifying one or more high-gradient edges corresponding to the enhancement regions;

increasing luminance values of the second image over the first image in the areas adjacent to the enhancement regions and closer to their respective enhancement regions than high-gradient edges; and

discontinuing increasing luminance values of the second image over the first image at areas further from their respective enhancement regions than the high-gradient edges.

21. A controller according to claim 18 wherein the controller is configured to generate the brightness enhancement function, the controller configured to generate the brightness enhancement function by:

generating a binarized mask which identifies pixels inside and outside of the one or more enhancement regions;

down-sampling the binarized mask to obtain a down-sampled mask;

for each iteration in a loop comprising N iterations;

applying a blur filter to the down-sampled mask in an initial iteration and to a result of the previous iteration in other iterations; and

up-sampling a result of the blur filter by an up-sampling step;

wherein a result of the Nth iteration of the loop comprises grey-scale image data having the same resolution as the first image data, the grey-scale image data convertible into at least a portion of the brightness enhancement function by suitable mapping.

22. A controller according to claim 21 wherein the controller is configured to generate the brightness enhancement function by generating an edge-stopping component of the brightness enhancement function based at least in part on a gradient image of the first image, the edge-stopping component having pixel values which indicate whether or not application of the brightness enhancement function should increase the luminance of corresponding pixels of the first image.