Low-illumination image processing method and device

• US 10,424,054 B2
• Filed: 06/26/2015
• Issued: 09/24/2019
• Est. Priority Date: 06/26/2015
• Status: Active Grant

• Current Assignee: Peking University Shenzhen Graduate School
• Original Assignee: Peking University Shenzhen Graduate School
• Inventors: Li, Lin, Wang, Ronggang, Tang, Chengzhou, Wang, Zhenyu, Gao, Wen
• Primary Examiner(s): Lu, Zhiyu
• Application Number:
  US15/736,403
• Publication Number:
  US 20180182074A1
• Time in Patent Office: 1,551 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20182 Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06T 7/90 Determination of colour characteristics

Abstract

A low-illumination image processing method and device address the problem of noise amplification in existing contrast enhancement techniques when applied to original low-illumination image. A noise suppression filter is additionally arranged before an operation of contrast enhancement, and smoothing processing is performed on an inverse color image of a low-illumination image by adopting a first filtering coefficient and a second filtering coefficient, so that image contrast is enhanced while random noise is suppressed. Texture and noise level parameter of an image are calculated according to a local characteristic inside block of the image. Weighted averaging is performed on a first smoothing image and a second smoothing image after smoothing processing according to the texture and noise level parameters. The texture image is obtained by performing texture structure extraction on a gradient image of an inverse color image, and the texture image is combined with a weighted image to sharpen the weighted image, to have an effect of enhancing image details. Therefore, the contrast of low-illumination image can be effectively enhanced, various types of noise can be filtered, and the image color and details can be retained at the same time to obtain a clear and vivid restored image.

Claims

1. A low-illumination image processing method, comprising:

inputting a low-illumination image;

dividing the low-illumination image into different texture regions, calculating a local standard deviation and a gradient sum of the gray scale of pixel points in various texture regions, and calculating a ratio of the local standard deviation and the gradient sum as a texture and noise level parameter of the image;

performing inverse color processing on the low-illumination image to obtain an inverse color image;

determining a first filtering coefficient and a second filtering coefficient according to the mean of the standard deviations of the gray scale of pixel points in various texture regions, and performing smoothing processing on an inverse color image by adopting the first filtering coefficient and the second filtering coefficient to obtain a first smoothing image and a second smoothing image respectively;

performing weighted averaging on the first smoothing image and the second smoothing image according to the texture and noise level parameter to obtain a weighted image;

calculating a dark channel image of the weighted image, and estimating an environment illumination intensity according to the dark channel image;

calculating a contrast enhancement coefficient according to the dark channel image and environment illumination intensity;

calculating a gradient image of the inverse color image, and extracting texture structure from the gradient image of the inverse color image to obtain a texture image;

combining the texture image with the weighted image to obtain a sharpened image; and

performing contrast enhancement on the sharpened image according to the environment illumination intensity and the contrast enhancement coefficient to obtain an enhanced image; and

performing inverse color processing on the enhanced image to obtain an output image.

2. A method according to claim 1, wherein the low-illumination image is divided into different texture regions by using superpixel segmentation.

3. The method of claim 1, wherein smoothing processing is performed by using a Three-dimensional Block Matching (BM3D) filter on an inverse color image by adopting the first filtering coefficient and the second filtering coefficient to obtain the first smoothing image and the second smoothing image respectively, wherein the first filtering coefficient is greater than the mean of the standard deviations of the gray scale of pixel points in various texture regions, wherein the second filtering coefficient is less than the mean of the standard deviations of the gray scale of pixel points in various texture regions.

4. The method of claim 1, further comprising:

correcting the contrast enhancement coefficient by reducing the contrast enhancement coefficient that is less than the pre-set value.

5. A low-illumination image processing device, comprising:

an input module configured to input a low-illumination image;

an image segmentation module configured to divide the low-illumination image into different texture regions;

a first calculation module configured to calculate a local standard deviation and a gradient sum of the gray scale of pixel points in various texture regions, and to calculate a ratio of the standard deviation and the gradient sum as a texture and noise level parameter of the image;

a first inverse color module configured to perform inverse color processing on the low-illumination image to obtain an inverse color image;

a smoothing filtering module configured to determine a first filtering coefficient and a second filtering coefficient according to a mean of the standard deviations of the gray scale of pixel points in various texture regions, and to perform smoothing processing on an inverse color image by adopting a first filtering coefficient and a second filtering coefficient to obtain a first smoothing image and a second smoothing image respectively;

a weighting module configured to perform weighted averaging on the first smoothing image and the second smoothing image according to the texture and noise level parameter to obtain a weighted image;

a second calculation module configured to calculate a dark channel image on the weighted image, and to estimate an environment illumination intensity according to the dark channel image; and

to calculate a contrast enhancement coefficient according to the dark channel image and environment illumination intensity;

a third calculation module configured to calculate a gradient image on the inverse color image, and performing texture structure extraction on the gradient image of the inverse color image to obtain a texture image;

a sharpening module configured to combine the texture image with the weighted image to obtain a sharpened image;

a contrast enhancement module configured to perform contrast enhancement on the sharpened image according to the environment illumination intensity and the contrast enhancement coefficient to obtain an enhanced image; and

a second inverse color module configured to perform inverse color processing on the enhanced image to obtain an output image.

6. The device of claim 5, wherein an image segmentation module is configured to divide the low-illumination image into different texture regions by using superpixel segmentation.

7. The device of claim 5, wherein the smoothing filtering module is configured to perform smoothing processing by using a Three-dimensional Block Matching (BM3D) filter on an inverse color image by adopting the first filtering coefficient and the second filtering coefficient to obtain the first smoothing image and the second smoothing image respectively, wherein the first filtering coefficient is greater than the mean of the standard deviations of the gray scale of pixel points in various texture regions, wherein the second filtering coefficient is less than the mean of the standard deviations of the gray scale of pixel points in various texture regions.

8. The device of claim 7, wherein the weighting module is configured to perform weighted averaging on the first smoothing image and the second smoothing image according to the texture and noise level parameter to obtain a weighted image by using the following formula:

R_s=α

·

R_s^fine+(1−

α

)·

R_s^coarse,

wherein Rs is a weighted image, R_s^coarse is a first smoothing image, R_s^fine is a second smoothing image, and α

is the texture and noise level parameter.

9. The device of claim 8, wherein the sharpening module is configured to combine the texture image with the weighted image to obtain a sharpened image by using the following formula:

R_sharp=R_s+α

*R_ds, wherein R_sharp is a sharpened image, and R_ds is a texture image.

10. The device of claim 5, wherein the second calculation module is configured to calculate a contrast enhancement coefficient according to dark channel image and environment illumination intensity by using the following formula:

11. The device of claim 10, wherein the second calculation module is further configured to adjust the weighted correction coefficient adaptively according to the illumination of the three RGB channels of the pixel by the following formula:

12. The device of claim 5, wherein the second calculation module is further configured to correct the contrast enhancement coefficient by reducing the contrast enhancement coefficient that is less than the pre-set value.

13. The device of claim 12, wherein the second calculation module is further configured to correct the contrast enhancement coefficient by using the following formula: