Image enhancement method using local illumination correction

• US 20070071350A1
• Filed: 09/29/2005
• Published: 03/29/2007
• Est. Priority Date: 09/29/2005
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Yeong-Taeg, Lee, Sangkeun
• Granted Patent: US 7,590,303 B2
• Time in Patent Office: Days
• US Class Current: 382/260
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement

Abstract

An image processing system for implementing a variant of Retinex algorithm based on local illumination correction is provided to improve an intensity value of each position of an input image. The image processing system provides the ability to distinguish details when compressing dynamic range of a given image. Specifically, first the illumination of the image is estimated by performing a weighted low pass filter (WLPF) operation, and the original image is separated into the illumination and the reflectance images. Next, the estimated illumination is modified by using a non-linear mapping function based on local illumination at each pixel location. Then, the modified illumination image is combined with the reflectance image. Finally, an enhanced image is obtained after a contrast correction procedure.

Claims

1. A method for processing an input image, comprising the steps of:

(a) providing digital input image pixels indexed to represent positions on a display, each pixel being indicative of an intensity value for each position; and

(b) determining pixel illumination correction for a pixel, and adjusting the intensity value for the pixel based on the illumination correction to generate an enhanced value.

2. The method of claim 1 wherein step (b) further includes the steps of:

estimating the local illumination of the input image by performing a weighted low pass filter (WLPF) operation on the image pixels;

separating the filtered image into illumination and reflectance images;

adjusting the pixel intensity value of the illumination image using a non-linear mapping function as illumination correction, based on the estimated local illumination at each pixel location; and

combining the adjusted illumination image with the reflectance image to generate an output image.

3. The method of claim 2 further comprising the steps of performing contrast enhancement on the output image to generate an enhanced image.

4. The method of claim 2, wherein the step of adjusting said intensity value using said illumination correction further includes the steps of:

separating said input image Y_IN(x,y) into illumination image L(x,y) and reflectance image R(x,y) as Y_IN (x, y)=L(x, y)·

R(x, y),

correcting the illumination image L(x,y) by calculating

$Y_{\mathrm{IN}}^{\mathrm{\prime <br><br>}}<br><br>\left(x,y\right)=L_{\max }·<br><br>{\left(\frac{L<br><br>\left(x,y\right)}{L_{\max }}\right)}^{1<br><br>\text{/}<br><br>\mathrm{\gamma <br><br>}}·<br><br>R<br><br>\left(x,y\right),$

where Y′

_IN(x,y) is the corrected illumination image, L_max is the maximum value of the input image, and γ

is the correction value.

5. The method of claim 4 further including the steps of correcting the contrast of the image by calculating

$Y_{\mathrm{OUT}}$

( x , y ) = σ

Y IN σ

Y IN ′

·

{ Y IN ′

⁡

( x , y ) - Y mean ′

⁡

( x , y ) } + Y mean ⁡

( x , y ) .

where, Y_OUT(x,y) represents the contrast corrected output image, σ

_z represents a standard deviation of an image Z, and Z_mean is a mean value of Z at the location (x,y).

6. The method of claim 4 wherein said illumination image is estimated by using a weighted low pass filter of the size m×

n as;

$L<br><br>\left(x,y\right)=\frac{\underset{i=1}{\overset{m}{\sum <br><br>}}<br><br>\underset{j=1}{\overset{n}{\sum <br><br>}}<br><br>g<br><br>\left(<br><br>Y_{\mathrm{IN}}<br><br>\left(i,j\right)-Y_{\mathrm{IN}}<br><br>\left(x,y\right)<br><br>\right)·<br><br>Y_{\mathrm{IN}}<br><br>\left(i,j\right)}{\underset{i=1}{\overset{m}{\sum <br><br>}}<br><br>\underset{j=1}{\overset{n}{\sum <br><br>}}<br><br>g<br><br>\left(<br><br>Y_{\mathrm{IN}}<br><br>\left(i,j\right)-Y_{\mathrm{IN}}<br><br>\left(x,y\right)<br><br>\right)},$

wherein the weighting function g(X) is determined as;

$g<br><br>\left(X\right)=\{\begin{array}{c}0,\mathrm{if}<br><br>\text{\hspace{1em}<br><br>}<br><br>X><br><br>\mathrm{\sigma <br><br>}·<br><br>{\mathrm{\tau <br><br>}}_2,\\ -\frac{1}{\left({\mathrm{\tau <br><br>}}_2-{\mathrm{\tau <br><br>}}_1\right)·<br><br>\mathrm{\sigma <br><br>}}·<br><br>\left(X-\mathrm{\sigma <br><br>}·<br><br>{\mathrm{\tau <br><br>}}_2\right),\mathrm{else}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{if}<br><br>\text{\hspace{1em}<br><br>}<br><br>X><br><br>\mathrm{\sigma <br><br>}·<br><br>{\mathrm{\tau <br><br>}}_1,\\ 1,\mathrm{otherwise}.\end{array}$

wherein X is an input intensity difference value, σ

is an predefined variance, and τ

₁ and τ

₂ are first and second thresholds, respectively.

7. The method of claim 4 wherein said correction value y is determined by calculating γ

=1+(γ

₀−

1)·

LPF(γ

₁·

γ

₂), where γ

₀ is a user input greater than 1, LPF(X) indicates low pass filtering, γ

₁ and γ

₂ are first and second control parameters computed based on the content and the local brightness of said input image, respectively.

8. The method of claim 7 further including the steps of calculating wherein the first control parameter γ

₁ as;

γ

₁=1−

g(ε

_d), where ε

_d represents average value of difference values d(x,y) at each pixel location (x,y) between two windows over the entire image, with different window sizes, m×

n, and M×

N, respectively.

9. The method of claim 8 further including the step of calculating the average value ε

_d as;

${\mathrm{\varepsilon <br><br>}}_d=\frac{1}{<br><br>S<br><br>}·<br><br>\underset{x,y\in <br><br>S}{\sum <br><br>}<br><br><br><br>d<br><br>\left(x,y\right)<br><br>,$

wherein |S| represents total size of the input image.

10. The method of claim 7 wherein the second control parameter γ

₂ is computed as;

γ

₂=g(B), wherein B(x,y) represents brightness ratio at a pixel location (x,y).

11. The method of claim 10 further including the steps of determining the brightness ration B(x,y) as:

$B<br><br>\left(x,y\right)=\frac{L<br><br>\left(x,y\right)}{G+L<br><br>\left(x,y\right)},$

wherein G is an average brightness of the illumination image L(x,y).

12. An image enhancement system for enhancing a digital input image comprising pixels, the system comprising:

means for receiving digital input image pixels indexed to represent positions on a display, each pixel being indicative of an intensity value for each position; and

an illumination correction module that performs local illumination correction by determining pixel illumination correction value for a pixel, and correcting the intensity value for the pixel based on the illumination correction value to generate an enhanced value.

13. The system of claim 12 wherein the illumination correction module further comprises an illumination estimator that estimates the local illumination of the input image by performing a weighted low pass filter (WLPF) operation on the image pixels to generate the illumination correction value.

14. The system of claim 13 wherein the illumination correction module further comprises:

a separation module that separates the filtered image into illumination and reflectance images,

an adjustment module that corrects said intensity value by adjusting said pixel intensity value of the illumination image using a non-linear mapping function as illumination correction value based on the estimated local illumination at each pixel location; and

a combining module that combines the adjusted illumination image with the reflectance image to generate an output image.

15. The system of claim 14 further comprising a contrast enhancement module that performs contrast enhancement on the output image to generate an enhanced output image.

16. The system of claim 15 wherein:

the separation module separates the input image Y_IN(x,y) into illumination image L(x,y) and reflectance image R(x,y) as Y_IN (x,y)=L(x, y)·

R(x, y),

the adjustment module adjusts said intensity value using said illumination correction value further by adjusting the illumination image L(x,y) by calculating

$Y_{I<br><br>\text{\hspace{1em}<br><br>}<br><br>N}^{\mathrm{\prime <br><br>}}<br><br>\left(x,y\right)=L_{\max }·<br><br>{\left(\frac{L<br><br>\left(x,y\right)}{L_{\max }}\right)}^{1/\mathrm{\gamma <br><br>}}·<br><br>R<br><br>\left(x,y\right),$

where Y′

_IN (x,y) is the corrected illumination image, L_max is the maximum value of the input image, and γ

is the correction value.

17. The system of claim 16 wherein the contrast enhancement module corrects the contrast of the image by calculating

$Y_{\mathrm{OUT}}$

( x , y ) = σ

Y I ⁢

 

⁢

N σ

Y I ⁢

 

⁢

N ′

·

{ Y I ⁢

 

⁢

N ′

⁡

( x , y ) - Y mean ′

⁡

( x , y ) } + Y mean ⁡

( x , y ) .

where, Y_OUT(x,y) represents the contrast corrected output image, σ

_Z represents a standard deviation of an image Z, and Z_mean is a mean value of Z at the location (x,y).

18. The system of claim 16 wherein the separation module estimates the illumination image L(x,y) further by using a weighted low pass filter of the size m×

n as;

$L<br><br>\left(x,y\right)=\frac{\underset{i=1}{\overset{m}{\sum <br><br>}}<br><br>\underset{j=1}{\overset{n}{\sum <br><br>}}<br><br>g<br><br>\left(<br><br>Y_{\mathrm{IN}}<br><br>\left(i,j\right)-Y_{\mathrm{IN}}<br><br>\left(x,y\right)<br><br>\right)·<br><br>Y_{\mathrm{IN}}<br><br>\left(i,j\right)}{\underset{i=1}{\overset{m}{\sum <br><br>}}<br><br>\underset{j=1}{\overset{n}{\sum <br><br>}}<br><br>g<br><br>\left(<br><br>Y_{\mathrm{IN}}<br><br>\left(i,j\right)-Y_{\mathrm{IN}}<br><br>\left(x,y\right)<br><br>\right)},$

wherein the weighting function g(X) is determined as;

$g<br><br>\left(X\right)=\{\begin{array}{c}0,\mathrm{if}<br><br>\text{\hspace{1em}<br><br>}<br><br>X><br><br>\mathrm{\sigma <br><br>}·<br><br>{\mathrm{\tau <br><br>}}_2,\\ -\frac{1}{\left({\mathrm{\tau <br><br>}}_2-{\mathrm{\tau <br><br>}}_1\right)·<br><br>\mathrm{\sigma <br><br>}}·<br><br>\left(X-\mathrm{\sigma <br><br>}·<br><br>{\mathrm{\tau <br><br>}}_2\right),\mathrm{else}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{if}<br><br>\text{\hspace{1em}<br><br>}<br><br>X><br><br>\mathrm{\sigma <br><br>}·<br><br>{\mathrm{\tau <br><br>}}_1,\\ 1,\mathrm{otherwise}\end{array}$

wherein X is an input intensity difference value, σ

is a predefined variance, and τ

₁ and τ

₂ are first and second thresholds, respectively.

19. The system of claim 16 wherein the adjustment module further determines an adjustment value γ

for adjusting said intensity value, by calculating γ

=1+(γ

₀−

1)·

LPF(γ

₁·

γ

₂), where γ

₀ is a user input greater than 1, LPF(X) indicates low pass filtering, γ

₁ and γ

₂ are first and second control parameters computed based on the content and the local brightness of said input image, respectively.

20. The method of claim 19 wherein the adjustment module calculates the first control parameter γ

₁ as;

γ

₁=1−

g(ε

_d), where ε

_d represents average value of difference values d(x,y) at each pixel location (x,y) between two windows over the entire image, with different window sizes, m×

n, and M×

N, respectively.

21. The system of claim 20 wherein the adjustment module further calculates the average value ε

_d as;

${\mathrm{\varepsilon <br><br>}}_d=\frac{1}{<br><br>S<br><br>}·<br><br>\underset{x,y\in <br><br>S}{\sum <br><br>}<br><br><br><br>d<br><br>\left(x,y\right)<br><br>,$

wherein |S| represents total size of the input image.

22. The system of claim 19 wherein the adjustment module calculates the second control parameter γ

₂ as;

γ

₂=g(B), wherein B(x,y) represents brightness ratio at a pixel location (x,y).

23. The system of claim 22 wherein adjustment determining the brightness ration B(x,y) as:

$B<br><br>\left(x,y\right)=\frac{L<br><br>\left(x,y\right)}{G+L<br><br>\left(x,y\right)},$

wherein G is a average brightness of the illumination image L(x,y).