Image enhancement method using local illumination correction

• US 7,590,303 B2
• Filed: 09/29/2005
• Issued: 09/15/2009
• Est. Priority Date: 09/29/2005
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Yeong-Taeg, Lee, Sangkeun
• Primary Examiner(s): Bali; Vikkram
• Assistant Examiner(s): ENTEZARI, MICHELLE M
• Application Number:
  US11/240,306
• Publication Number:
  US 20070071350A1
• Time in Patent Office: 1,447 Days
• Field of Search: 382/260
• US Class Current: 382/274
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement

Abstract

An image processing system for implementing a variant of Retinex algorithm based on local illumination correction is provided to improve an intensity value of each position of an input image. The image processing system provides the ability to distinguish details when compressing dynamic range of a given image. Specifically, first the illumination of the image is estimated by performing a weighted low pass filter (WLPF) operation, and the original image is separated into the illumination and the reflectance images. Next, the estimated illumination is modified by using a non-linear mapping function based on local illumination at each pixel location. Then, the modified illumination image is combined with the reflectance image. Finally, an enhanced image is obtained after a contrast correction procedure.

Claims

1. A method for processing an input image, comprising:

employing a processor for

providing digital input image pixels indexed to represent positions on a display, each pixel being indicative of an intensity value for each position; and

determining pixel illumination correction for a pixel based on control parameters computed based on content and the local brightness of said input image, and adjusting the intensity value for the pixel based on the illumination correction to generate an enhanced value, wherein determining pixel illumination correction further includes;

estimating the local illumination of the input image by performing a weighted low pass filter (WLPF) operation on the image pixels;

separating the filtered image into illumination and reflectance images;

adjusting the pixel intensity value of the illumination images using a non-linear mapping function as illumination correction, based on the estimated local illumination at each pixel location, wherein adjusting said intensity value using said illumination correction further includes;

separating said input image Y_IN(x,y) into illumination image L(x,y) and reflectance image R(x,y) as Y_IN(x,y)=L(x,y)·

R(x,y),

correcting the illumination image L(x,y) by calculating

$Y_{\mathrm{IN}}^{\mathrm{\prime <br><br>}}<br><br>\left(x,y\right)=L_{\max }·<br><br>{\left(\frac{L<br><br>\left(x,y\right)}{L_{\max }}\right)}^{1<br><br>\text{/}<br><br>\mathrm{\gamma <br><br>}}·<br><br>R<br><br>\left(x,y\right),$

where Y′

_IN(x,y) is the corrected illumination image, L_max is the maximum value of the input image, and γ

is the correction value; and

combining the adjusted illumination image with the reflectance image to generate an output image,

wherein said illumination image is estimated by using a weighted low pass filter of the size m×

n as;

$L<br><br>\left(x,y\right)=\frac{\underset{i=1}{\overset{m}{\sum <br><br>}}<br><br>\underset{j=1}{\overset{n}{\sum <br><br>}}<br><br>g<br><br>\left(<br><br>Y_{\mathrm{IN}}<br><br>\left(i,j\right)-Y_{\mathrm{IN}}<br><br>\left(x,y\right)<br><br>\right)·<br><br>Y_{\mathrm{IN}}<br><br>\left(i,j\right)}{\underset{i=1}{\overset{m}{\sum <br><br>}}<br><br>\underset{j=1}{\overset{n}{\sum <br><br>}}<br><br>g<br><br>\left(<br><br>Y_{\mathrm{IN}}<br><br>\left(i,j\right)-Y_{\mathrm{IN}}<br><br>\left(x,y\right)<br><br>\right)},$

wherein the weighting function g(X) is determined as;

$g<br><br>\left(X\right)=\{\begin{array}{c}0,\mathrm{if}<br><br><br><br>X><br><br>\mathrm{\sigma <br><br>}·<br><br>{\mathrm{\tau <br><br>}}_2,\\ -\frac{1}{\left({\mathrm{\tau <br><br>}}_2-{\mathrm{\tau <br><br>}}_1\right)·<br><br>\mathrm{\sigma <br><br>}}·<br><br>\left(X-\mathrm{\sigma <br><br>}·<br><br>{\mathrm{\tau <br><br>}}_2\right),\mathrm{else}<br><br><br><br>\mathrm{if}<br><br><br><br>X><br><br>\mathrm{\sigma <br><br>}·<br><br>{\mathrm{\tau <br><br>}}_1,\\ 1,\mathrm{otherwise}\end{array}$

wherein X is an input intensity difference value, σ

is an predefined variance, and τ

₁ and τ

₂ are first and second thresholds, respectively.

2. The method of claim 1 further comprising performing contrast enhancement on the output image to generate an enhanced image.

3. The method of claim 2 further including correcting the contrast of the image by calculating

$Y_{\mathrm{OUT}}\; <li></li>\left(x,y\right)=\frac{{\mathrm{\sigma <br><br>}}_{Y_{\mathrm{IN}}}}{{\mathrm{\sigma <br><br>}}_{Y_{\mathrm{IN}}^{\mathrm{\prime <br><br>}}}}·<br><br>\left\{Y_{\mathrm{IN}}^{\mathrm{\prime <br><br>}}<br><br>\left(x,y\right)-Y_{\mathrm{mean}}^{\mathrm{\prime <br><br>}}<br><br>\left(x,y\right)\right\}+Y_{\mathrm{mean}}<br><br>\left(x,y\right)$

where, Y_OUT(x,y) represents the contrast corrected output image, σ

_z represents a standard deviation of an image Z, and Z_mean is a mean value of Z at the location (x,y).

4. The method of claim 3 wherein said correction value γ

is determined by calculating γ

=1+(γ

₀−

1)·

LPF(γ

₁·

γ

₂), where γ

₀ is a user input greater than 1, LPF(x) indicates low pass filtering, γ

₁ and γ

₂ are first and second control parameters computed based on the content and the local brightness of said input image, respectively.

5. The method of claim 4 further including the steps of calculating wherein the first control parameter γ

₁ as;

γ

₁=1−

g(ε

_d), where ε

_d represents average value of difference values d(x,y) at each pixel location (x,y) between two windows over the entire image, with different window sizes, m×

n, and M×

N, respectively.

6. The method of claim 5 further including the step of calculating the average value ε

_d as;

${\mathrm{\varepsilon <br><br>}}_d=\frac{1}{<br><br>S<br><br>}·<br><br>\underset{x,y\in <br><br>S}{\sum <br><br>}<br><br><br><br>d<br><br>\left(x,y\right)<br><br>,$

wherein |S| represents total size of the input image.

7. The method of claim 4 wherein the second control parameter γ

₂ is computed as;

γ

₂=g(B), wherein B(x,y) represents brightness ratio at a pixel location (x,y).

8. The method of claim 7 further including the steps of determining the brightness ratio B(x,y) as:

$B<br><br>\left(x,y\right)=\frac{L<br><br>\left(x,y\right)}{G+L<br><br>\left(x,y\right)},$

wherein G is an average brightness of the illumination image L(x,y).

16. The method of claim 1, wherein the content comprises high frequency components and low frequency components of the input image.

9. An image enhancement system for enhancing a digital input image comprising pixels, the system comprising:

means for receiving digital input image pixels indexed to represent positions on a display, each pixel being indicative of an intensity value for each position; and

an illumination correction module that performs local illumination correction by determining pixel illumination correction value for a pixel, and correcting the intensity value for the pixel based on control parameters computed based on content and the local brightness of said input image the illumination correction value to generate an enhanced value, the illumination correction module further comprises;

an illumination estimator that estimates the local illumination of the input image by performing a weighted low pass filter (WLPF) operation on the image pixels to generate the illumination correction value;

a separation module that separates the filtered image into illumination and reflectance images,

an adjustment module that corrects said intensity value by adjusting said pixel intensity value of the illumination image using a non-linear mapping function as illumination correction value based on the estimated local illumination at each pixel location; and

a combining module that combines the adjusted illumination image with the reflectance image to generate an output image; and

a contrast enhancement module that performs contrast enhancement on the output image to generate an enhanced output image,

wherein;

the separation module separates the input image Y_IN(x,y) into illumination image L(x,y) and reflectance image R(x,y) as Y_IN(x,y)=L(x,y)·

R(x,y),

the adjustment module adjusts said intensity value using said illumination correction value further by adjusting the illumination image L(x,y) by calculating

$Y_{I<br><br><br><br>N}^{\mathrm{\prime <br><br>}}<br><br>\left(x,y\right)=L_{\max }·<br><br>{\left(\frac{L<br><br>\left(x,y\right)}{L_{\max }}\right)}^{1/\mathrm{\gamma <br><br>}}·<br><br>R<br><br>\left(x,y\right),$

where Y′

_IN(x,y) is the corrected illumination image, L_max is the maximum value of the input image, and γ

is the correction value, and the separation module estimates the illumination image L(x,y) further by using a weighted low pass filter of the size m×

n as;

$L<br><br>\left(x,y\right)=\frac{\underset{i=1}{\overset{m}{\sum <br><br>}}<br><br>\underset{j=1}{\overset{n}{\sum <br><br>}}<br><br>g<br><br>\left(<br><br>Y_{\mathrm{IN}}<br><br>\left(i,j\right)-Y_{\mathrm{IN}}<br><br>\left(x,y\right)<br><br>\right)·<br><br>Y_{\mathrm{IN}}<br><br>\left(i,j\right)}{\underset{i=1}{\overset{m}{\sum <br><br>}}<br><br>\underset{j=1}{\overset{n}{\sum <br><br>}}<br><br>g<br><br>\left(<br><br>Y_{\mathrm{IN}}<br><br>\left(i,j\right)-Y_{\mathrm{IN}}<br><br>\left(x,y\right)<br><br>\right)},$

wherein the weighting function g(X) is determined as;

$g<br><br>\left(X\right)=\{\begin{array}{c}0,\mathrm{if}<br><br><br><br>X><br><br>\mathrm{\sigma <br><br>}·<br><br>{\mathrm{\tau <br><br>}}_2,\\ -\frac{1}{\left({\mathrm{\tau <br><br>}}_2-{\mathrm{\tau <br><br>}}_1\right)·<br><br>\mathrm{\sigma <br><br>}}·<br><br>\left(X-\mathrm{\sigma <br><br>}·<br><br>{\mathrm{\tau <br><br>}}_2\right),\mathrm{else}<br><br><br><br>\mathrm{if}<br><br><br><br>X><br><br>\mathrm{\sigma <br><br>}·<br><br>{\mathrm{\tau <br><br>}}_1,\\ 1,\mathrm{otherwise}\end{array}$

wherein X is an input intensity difference value, σ

is a predefined variance, and τ

₁ and τ

₂ are first and second thresholds, respectively.

10. The system of claim 9 wherein the contrast enhancement module corrects the contrast of the image by calculating

$Y_{\mathrm{OUT}}\; <li></li>\left(x,y\right)=\frac{{\mathrm{\sigma <br><br>}}_{Y_{I<br><br><br><br>N}}}{{\mathrm{\sigma <br><br>}}_{Y_{I<br><br><br><br>N}^{\mathrm{\prime <br><br>}}}}·<br><br>\left\{Y_{I<br><br><br><br>N}^{\mathrm{\prime <br><br>}}<br><br>\left(x,y\right)-Y_{\mathrm{mean}}^{\mathrm{\prime <br><br>}}<br><br>\left(x,y\right)\right\}+Y_{\mathrm{mean}}<br><br>\left(x,y\right)$

where, Y_OUT(x,y) represents the contrast corrected output image, σ

_Z represents a standard deviation of an image Z, and Z_mean is a mean value of Z at the location (x,y).

11. The system of claim 10 wherein the adjustment module further determines an adjustment value γ

for adjusting said intensity value, by calculating γ

=1+(γ

₀−

1)·

LPF(γ

₁·

γ

₂), where γ

₀ is a user input greater than 1, LPF(x) indicates low pass filtering, γ

₁ and γ

₂ are first and second control parameters computed based on the content and the local brightness of said input image, respectively.

12. The system of claim 11 wherein the adjustment module calculates the first control parameter γ

₁ as;

γ

₁=1−

g(ε

_d), where ε

_d represents average value of difference values d(x,y) at each pixel location (x,y) between two windows over the entire image, with different window sizes, m×

n, and M×

N, respectively.

13. The system of claim 12 wherein the adjustment module further calculates the average value ε

_d as;

${\mathrm{\varepsilon <br><br>}}_d=\frac{1}{<br><br>S<br><br>}·<br><br>\underset{x,y\in <br><br>S}{\sum <br><br>}<br><br><br><br>d<br><br>\left(x,y\right)<br><br>,$

wherein |S| represents total size of the input image.

14. The system of claim 11 wherein the adjustment module calculates the second control parameter γ

₂ as;

γ

₂=g(B), wherein B(x,y) represents brightness ratio at a pixel location (x,y).

15. The system of claim 14 wherein adjustment determining the brightness ratio B(x,y) as:

$B<br><br>\left(x,y\right)=\frac{L<br><br>\left(x,y\right)}{G+L<br><br>\left(x,y\right)},$

wherein G is a average brightness of the illumination image L(x,y).