System and method for image enhancement, dynamic range compensation and illumination correction

• US 20030012448A1
• Filed: 04/30/2001
• Published: 01/16/2003
• Est. Priority Date: 04/30/2001
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Shaked, Doron, Keshet, Renato, Elad, Michael, Sobel, Irwin, Kimmel, Ronny
• Granted Patent: US 6,941,028 B2
• Time in Patent Office: Days
• US Class Current: 382/274
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model

Abstract

A system and a method solve the estimation problem of finding reflectance R and illumination L. The system and method to solve a functional of the unknown illumination L such that a minimum of the functional is assumed to yield a good estimate of the illumination L. Having a good estimate of the illumination L implies a good estimate of the reflectance R. The functional uses a variational framework to express requirements for the optimal solution. The requirements include: 1) that the illumination L is spatially smooth; 2) that the reflectance values are in the interval [0,1] —thus, when decomposing the image S, the solution should satisfy the constraint L>S; 3) that among all possible solutions, the estimate of the illumination L should be as close as possible to the image S, so that the contrast of the obtained R is maximal; and 4) that the reflectance R complies with typical natural image behavior (e.g., the reflectance is piece-wise smooth).

Claims

1. An image enhancement method, comprising:

capturing an image;

constructing a multi-resolution structure comprising one or more resolution layers;

processing each resolution layer using an iterative algorithm having at least one iteration;

projecting each processed resolution layer to a subsequent resolution layer;

up-calling each projected resolution layer to the subsequent resolution layer; and

using the projected resolution layers to estimate an illumination image.

2. The method of claim 1, further comprising, for each of one or more iterations:

calculating a gradient of a penalty functional; and

computing an optimal line-search step size.

3. The method of claim 2, wherein the penalty functional is given by:

4. The method of claim 2, wherein the penalty functional is given by:

F[l]=∫

_Ω

(w₁(∇

s)|∇

l|²+α

(l−

s)+β

w₂(∇

s) |∇

l−

∇

s|²)dxdy

where w₁ and w₂ are non-linear functions of the gradient.

5. The method of claim 1, wherein the iterative algorithm is a Projected Normalized Steepest Descent algorithm.

6. The method of claim 1, wherein the iterative algorithm is a Steepest Descent algorithm.

7. The method of claim 1, wherein a set of constraints comprise that the illumination is greater than the image intensity, L>

S.

8. The method of claim 1, further comprising applying penalty terms, the penalty terms, comprising:

that the illumination is spatially smooth;

that the reflectance is maximized;

that the reflectance is piece-wise smooth.

9. The method of claim 1, further comprising:

computing the reflectance image based on the captured image and the estimated illumination image;

computing a gamma correction factor;

applying the gamma correction factor to the estimated illumination image; and

multiplying the gamma-corrected illumination image and the reflectance image, thereby producing a corrected image.

10. A system, embodied in a computer-readable medium, for enhancing digital images, comprising:

a log module that receives an input digital image S and computes a logarithm s of the input digital image;

an illumination estimator module that produces an estimate l* of an illumination component L of the input digital image S, wherein the estimator module employs a construct comprising one or more resolution layers, and an iterative algorithm that processes each of the one or more resolution layers; and

a summing node that sums the logarithm s and a negative of the estimate l* to produce an estimate r* of a logarithm of a reflectance component R of the input digital image S, wherein a processed resolution layer is used to up-scale a subsequent resolution layer.

11. The system of claim 10, wherein the iterative algorithm, for each of one or more iterations:

calculates a gradient of a penalty functional; and

computes an optimal line-search step size.

12. The method of claim 11, wherein the penalty functional is given by:

13. The system of claim 10, wherein the penalty functional is given by:

F[l]=∫

_Ω

(w₁(∇

s)|∇

l|²30 α

(l−

s)+β

w₂(∇

s) |∇

l−

∇

s|²)dxdy

where w₁ and W₂ are non-linear functions of the gradient.

14. The system of claim 10, wherein the iterative algorithm is a Projected Normalized Steepest Descent algorithm.

15. The system of claim 10, wherein the iterative algorithm is a Steepest Descent algorithm.

16. The system of claim 10, wherein each of the one or more resolution layers is projected onto constraints, and wherein the constraints comprise that the illumination is greater than the image intensity, L>

S;

17. The system of claim 10, further comprising penalty terms, the penalty terms comprising:

that the illumination is spatially smooth

that the reflectance is maximized; and

that the reflectance is piece-wise smooth.

18. The system of claim 10, further comprising:

a module that computes reflectance and illumination images based on the input digital image and the estimated illumination image;

a gamma correction module that computes a gamma correction factor and applies the gamma correction factor to the estimated illumination image; and

a node that multiples the gamma-corrected illumination image and the reflectance image, thereby producing a corrected digital image.

20. The method of claim 19, wherein the image S is a RGB domain color image, the method further comprising:

mapping colors R, G, B of the image S into a luminance/chrominance color space;

applying a correction factor to a luminance layer; and

mapping the luminance/chrominance colors back to the RGB domain.

19. A method for enhancing an image S, the image S comprising a reflectance R and an illumination L, the method comprising:

constructing a multi-resolution image structure having one or more resolution layers;

processing the resolution layers using an iterative algorithm;

projecting the processed resolution layers onto a set of constraints, the set of constraints comprising boundary conditions and that L>

S; and

using the projected resolution layers to estimate an illumination image.