ENHANCING DYNAMIC RANGES OF IMAGES

• US 20140168249A1
• Filed: 02/20/2014
• Published: 06/19/2014
• Est. Priority Date: 07/30/2007
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: REMPEL, Allan G., HEIDRICH, Wolfgang, WARD, Gregory John, WHITEHEAD, Lorne A., SEETZEN, Helge
• Granted Patent: US 8,948,537 B2
• Time in Patent Office: Days
• US Class Current: 345/589
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/007 Dynamic range modification applied in cameras using an electronic image sensor H04N23/741, H04N23/743
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G09G 2320/0646 Modulation of illumination source brightness and image signal correlated to each other
• CPC Class Codes: G09G 2320/066 for control of contrast
• CPC Class Codes: G09G 3/3426 the different display panel areas being distributed in two dimensions, e.g. matrix
• CPC Class Codes: G09G 5/10 Intensity circuits
• CPC Class Codes: H04N 5/20 Circuitry for controlling amplitude response
• CPC Class Codes: H04N 9/3126 for spatial light modulators in series

Abstract

Methods and apparatus according to various aspects take as input image data in a lower-dynamic-range (LDR) format and produce as output enhanced image data having a dynamic range greater than that of the input image data (i.e. higher-dynamic range (HDR) image data). In some embodiments, the methods are applied to video data and are performed in real-time (i.e. processing of video frames to enhance the dynamic range of the video frames is completed at least on average at the frame rate of the video signal).

Claims

1. A controller for enhancing dynamic range of a first image, the controller configured to:

identify one or more enhancement regions within the first image for which luminance values satisfy an enhancement criterion;

apply a smoothly-varying brightness enhancement function to the first image to obtain a second image having a dynamic range greater than that of the first image, the brightness enhancement function increasing luminance values of the second image over the first image in the enhancement regions and in at least some areas adjacent to the enhancement regions, the effect of the brightness enhancement function dropping off smoothly with increasing distance from the enhancement regions, wherein the controller is configured to apply the brightness enhancement function to the first image for pixels in the first image by multiplying a corresponding first image pixel value by a corresponding brightness enhancement factor.

2. A controller according to claim 1, wherein the controller is configured to generate the brightness enhancement function from the first image, the controller configured to generate the brightness enhancement function by generating a mask based on the first image and blurring the mask.

3. A controller according to claim 2, wherein the controller is configured to blur the mask by applying a Gaussian blur filter to the mask.

4. A controller according to claim 2, wherein the brightness enhancement function has values corresponding to pixels of the image that are in the range of 1 to a where a is in the range of 3 to 12.

5. A controller according to claim 1, wherein a spatial spectrum of the brightness enhancement function comprises primarily angular frequencies of 1 cycle per degree or less referenced to an intended viewing location.

6. A controller according to claim 1, wherein the controller is configured to display the second image on a display and wherein a standard deviation of the brightness enhancement function is at least about 0.025 of a horizontal resolution of the display.

7. A controller according to claim 1, wherein the controller is configured to generate the brightness enhancement function by generating an edge-stopping component of the brightness enhancement function based at least in part on a gradient image of the first image, the edge-stopping component having pixel values which indicate whether or not application of the brightness enhancement function should increase the luminance of corresponding pixels of the first image.

8. A controller according to claim 1, wherein the controller is configured to generate the brightness enhancement function, the controller configured to generate the brightness enhancement function by:

generating a binarized mask which identifies pixels inside and outside of the one or more enhancements regions;

down-sampling the binarized mask to obtain a down-sampled mask;

for each iteration in a loop comprising N iterations;

applying a blur filter to the down-sampled mask in an initial iteration and to a result of the previous iteration in other iterations; and

up-sampling a result of the blur filter by an up-sampling step;

wherein a result a result of the Nth iteration of the loop comprises grey-scale image data having the same resolution as the first image data, the grey-scale image data convertable into at least a portion of the brightness enhancement function by suitable mapping.

9. A controller according to claim 8, wherein the controller is configured to map the grey-scale image data to provide a smooth component of the brightness enhancement function, the smooth component having a range 1−

a, where a is a brightness amplification parameter greater than 1.

10. A controller according to claim 8, wherein the controller is configured to:

generate the brightness enhancement function by generating an edge-stopping component of the brightness enhancement function based at least in part on a gradient image of the first image, the edge-stopping component having pixel values which indicate whether or not application of the brightness enhancement function should increase the luminance of corresponding pixels of the first image; and

to generate the edge-stopping component by;

down-sampling the gradient image of the first image in a series of N down-sampling steps to obtain a set of N down-sampled gradient images;

for each iteration in a loop comprising N iterations;

performing a DILATION operation to the down-sampled mask in an initial iteration and to a result of the previous iteration in other iterations; and

up-sampling a result of the DILATION operation by an up-sampling step.

11. A controller according to claim 1, wherein the controller is configured to generate the brightness enhancement function, the controller configured to generate the brightness enhancement function by:

down-sampling the first image to obtain intermediate-resolution down-sampled image data;

further down-sampling the intermediate-resolution down-sampled image data to obtain lower-resolution down-sampled image data;

generating a lower-resolution mask from the lower-resolution down-sampled image data, the lower-resolution mask identifying pixels inside and outside of the one or more enhancement regions; and

for each iteration in a loop comprising N iterations;

applying a blur filter to the lower-resolution mask in an initial iteration and to a result of the previous iteration in other iterations; and

up-sampling a result of the blur filter by an up-sampling step;

wherein a result of the Nth iteration of the loop comprises grey-scale image data having the same resolution as the first image, the grey-scale image data convertable into at least a component of the brightness enhancement function.

12. A controller according to claim 11, wherein the controller is configured to generate the edge-stopping component by:

down-sampling the gradient image of the first image in a series of N down-sampling steps to obtain a set of N down-sampled gradient images;

for each iteration in a first loop comprising M iterations, where M<

N;

performing a DILATION operation to the lower-resolution mask in an initial iteration and to a result of the previous iteration in other iterations; and

up-sampling a result of the DILATION operation by an up-sampling step;

wherein a result of the first loop comprises intermediate-resolution edge-stopping data;

for each iteration in a second loop comprising N-M iterations;

performing the DILATION operation to the intermediate-resolution edge-stopping data in an initial iteration and to a result of the previous iteration in other iterations; and

up-sampling a result of the DILATION operation by an up-sampling step;

wherein a result of the second loop comprises full-resolution edge-stopping data having a resolution corresponding to that of the first image.

13. A controller according to claim 11, wherein the controller is configured to determine first modulator driving values for a first modulator of a dual modulator display used to display the second image based at least in part on the intermediate-resolution down-sampled image data.

14. A controller according to claim 12, wherein the controller is configured to determine first modulator driving values for a first modulator of a dual modulator display used to display the second image based at least in part on the intermediate-resolution down-sampled image data and based at least in part on the intermediate-resolution edge-stopping data.

15. A controller for use in displaying a first image having a relatively high dynamic range obtained from a second image having a relatively low dynamic range, the controller configured to:

identify at least one enhancement region within the second region;

apply a brightness enhancement function to the second image to generate the first image, wherein compared to the second image, the first image comprises increased luminance values of enhancement-region pixels in the enhancement region and increased luminance values of one or more boundary-region pixels outside of the enhancement region, the luminance value increase of the boundary-region pixels decreasing with distance from the enhancement region; and

determine driving signals for at least one modulator of a dual-modulator display based at least in part on intermediate data obtained in the course of application of the brightness enhancement function to the second image.

16. A controller according to claim 15, wherein the intermediate data comprises down-sampled data obtained by down-sampling the first image.

17. A controller for enhancing lower-dynamic range (LDR) image data representing an image, the controller configured to identify at least one enhancement region within the image;

generate a brightness enhancement function, wherein the controller is configured to generate the brightness enhancement function by;

generating a mask which identifies pixels inside and outside of the enhancement region;

applying a blur filter to the mask to obtain grey-scale image data;

applying the brightness enhancement function to the LDR image data to generate higher-dynamic range (HDR) image data, wherein compared to the LDR image data, the HDR image data comprises increased luminance values of enhancement-region pixels in the enhancement region and increased luminance values of one or more boundary-region pixels outside of the enhancement region, the luminance value increase of the boundary-region pixels decreasing with distance from the enhancement region.

18. A controller according to claim 17, wherein the controller is configured to map the grey-scale image data to provide a smooth component of the brightness enhancement function, the smooth component having a range of 1−

a where a is a brightness amplification parameter in a range of 2 to 10.

19. A controller according to claim 17, wherein the controller is configured to map the grey-scale image data to provide a smooth component of the brightness enhancement function, the smooth component having a range of 1−

a where a is a brightness amplification parameter greater than 1; and

apply the brightness enhancement function to the LDR image data by pixel-wise multiplication of the smooth component with the LDR image data.

20. A controller according to claim 17, wherein the controller is configured to generate the brightness enhancement function by generating an edge-stopping component of the brightness enhancement function based at least in part on a gradient image of the LDR image data, the edge-stopping component having pixel values which indicate whether or not application of the brightness enhancement function should increase the luminance of corresponding pixels of the LDR image data.