Method and system for dual-envelope image enhancement

• US 8,208,750 B2
• Filed: 10/26/2007
• Issued: 06/26/2012
• Est. Priority Date: 10/26/2007
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Fischer, Mani, Shaked, Doron, Staelin, Carl, Keshet, Renato
• Primary Examiner(s): Bhatnagar, Anand
• Assistant Examiner(s): BLOOM, NATHAN J
• Application Number:
  US11/977,969
• Publication Number:
  US 20090110323A1
• Time in Patent Office: 1,705 Days
• Field of Search: 382162-172, 382254-255, 382/260, 382/270, 382/274, 3482221-2241, 348/571, 348575-576, 348/582, 348671-675, 348/687, 348/708
• US Class Current: 382/274
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/007 Dynamic range modification applied in cameras using an electronic image sensor H04N23/741, H04N23/743
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof

Abstract

Various embodiments of the present invention are directed to methods and systems for processing signals, particularly signals encoding two-dimensional images, such as photographs, video frames, graphics, and other visually displayed information. In various method embodiments of the present invention, an input image signal is decomposed into an upper-envelope function, a lower-envelope function, and a details function. One or both of the upper-envelope and lower-envelope functions are then modified to produce one or more image-enhancing effects when the modified upper-envelope and lower-envelope functions are recombined with the details function to produce an enhanced image signal.

Claims

1. A signal-processing system comprising:

input means for receiving an input image signal S_in;

output means for outputting an enhance image signal; and

a processing component that

computes upper-envelope and lower-envelope functions U and L with respect to the input image signal S_in;

modifies the upper-envelope and lower-envelope functions U and L to produce a modified upper-envelope function U′ and

a modified lower-envelope function L′

; and

employs the modified upper-envelope function U′ and

the modified lower-envelope function L′

to generate an enhanced output image.

2. The signal-processing system of claim 1 wherein the processing component computes the upper-envelope and lower-envelope functions U and L using a Retinex-based method based on one or more of:

bilateral filters;

variational calculus;

robust recursive envelope operators;

smoothing filters and operators;

Poisson equation; and

infinite impulse response filters.

3. The signal-processing system of claim 2 wherein the processing component computes the upper-envelope and lower-envelope functions U and L based on one or more of the assumptions:

reflectance is everywhere greater than or equal to 0% and everywhere less than or equal to 100%; and

illumination and reflectance are at least piecewise spatially smooth.

4. The signal-processing system of claim 1 wherein the processing component computes the upper-envelope and lower-envelope functions U and L based on one or more of the assumptions:

U and L are at least piecewise spatially smooth;

large gradients in input image signal, S_in, are indicative of spatial discontinuity in U and L; and

small contiguous alternating shaded regions and non-shaded regions within an image are indicative of an absence of spatial discontinuities in U and L.

5. The signal-processing system of claim 1 wherein the processing component modifies the upper-envelope and lower-envelope functions U and L to produce a modified upper-envelope function U′

and a modified lower-envelope function L′

by;

determining envelope-modifying functions ƒ

₁ and ƒ

₂;

applying envelope-modifying function ƒ

₁ to one or both of upper-envelope function U and lower envelope function L to generate a modified upper-envelope function U′

; and

applying envelope-modifying function ƒ

₂ to one or both of lower-envelope function L and upper-envelope function U to generate a modified lower-envelope function L′

.

6. The signal-processing system of claim 5 wherein the processing component determines envelope-modifying functions ƒ

₁ and ƒ

₂ by one or more of;

computing functions ƒ

₁ and ƒ

₂ based on statistics computed for the input image signal S_in and on specified image enhancements for the enhanced output image;

selecting functions ƒ

₁ and ƒ

₂ from pre-computed functions;

selecting parameterized functions ƒ

₁ and ƒ

₂ from a set of functions and parameters based on specified image enhancements for the enhanced output image.

7. The signal-processing system of claim 5 wherein the processing component employs the modified upper-envelope function ƒ

₁(U) to generate an enhanced output image by;

computing an intermediate image Ŝ

as

8. The signal-processing system of claim 1 wherein the processing component employs the modified upper-envelope function U′

and the modified lower-envelope function L′

to generate an enhanced output image by;

computing a details function D from upper-envelope and lower-envelope functions U and L and input image signal S_in; and

combining the details function D, modified upper-envelope function U′

, and modified lower-envelope function L′

to produce the enhanced output signal.

9. The signal-processing system of claim 8

wherein the processing component computes the details function D from upper-envelope and lower-envelope functions U and L and input image signal S_in by

10. The signal-processing system of claim 8

wherein S is computed as

S=ƒ

<sub>U′

,L′</sub>

(D)

where ƒ

is a monotonic function of D, mapping a range [0, 1] of D values to an output range. [U′

,L′

].

11. The signal-processing system of claim 10 wherein ƒ

<sub>U′

,L′</sub>

(D) is one of;

a linear function ƒ

<sub>U′

,L′</sub>

;

an S-shaped tone mapping function, where the S-inflection is collocated with a Retinex-type average of an image, A.

12. The signal-processing system of claim 1 wherein the enhanced output image is enhanced with respect to one or more of:

local-contrast of details;

global contrast between shaded and unshaded regions of an image;

smoothness;

noise; and

intensity range.

20. The method of claim 8

wherein S is computed as

S=ƒ

<sub>U′

,L′</sub>

(D)

where ƒ

<sub>U′

,L′</sub>

is one of;

a monotonic function of D, mapping a range [0, 1] of D values to an output range [U′

, L′

];

a linear function; and

an S-shaped tone mapping function, where the S-inflection is collocated with a Retinex-type average of an image, A.

13. A method for enhancing an input image signal S_in, the method comprising:

computing upper-envelope and lower-envelope functions U and L with respect to the input image signal S_in;

modifying the upper-envelope and lower-envelope functions U and L to produce a modified upper-envelope function U′ and

a modified lower-envelope function L′

; and

employing the modified upper-envelope function U′ and

the modified lower-envelope function L′

to generate an enhanced output image.

14. The method of claim 13

wherein the upper-envelope and lower-envelope functions U and L are computed using a Retinex-based method based on one or more of:

bilateral filters,

variational calculus,

robust recursive envelope operators,

smoothing filters and operators,

Poisson equation, and

infinite impulse response filters; and

wherein the upper-envelope and lower-envelope functions U and L are computed based on one or more of the assumptions;

reflectance is everywhere greater than or equal to 0% and everywhere less than or equal to 100%; and

illumination and reflectance are at least piecewise spatially smooth.

15. The method of claim 13 wherein the upper-envelope and lower-envelope functions U and L are computed based on one or more of the assumptions:

U and L are at least piecewise spatially smooth;

large gradients in input image signal, S_in, are indicative of spatial discontinuity in U and L; and

small contiguous alternating shaded regions and non-shaded regions within an image are indicative of an absence of spatial discontinuities in U and L.

16. The method of claim 13 wherein the upper-envelope and lower-envelope functions U and L are modified to produce a modified upper-envelope function U′

and a modified lower-envelope function L′

by;

determining envelope-modifying functions ƒ

₁ and ƒ

₂;

applying envelope-modifying function ƒ

₁ to one or both of upper-envelope function U and lower envelope function L to generate a modified upper-envelope function U′

; and

applying envelope-modifying function ƒ

₂ to one or both of lower-envelope function L and upper-envelope function U to generate a modified lower-envelope function L′

; and

wherein the envelope-modifying functions ƒ

₁ and ƒ

₂ are determined by one or more of;

computing functions ƒ

₁ and ƒ

₂ based on statistics computed for the input image signal S_in and on specified image enhancements for the enhanced output image;

selecting functions ƒ

₁ and ƒ

₂ from pre-computed functions;

selecting parameterized functions ƒ

₁ and ƒ

₂ from a set of functions and parameters based on specified image enhancements for the enhanced output image.

17. The method of claim 13 wherein a modified upper-envelope function ƒ

₁(U) is used to generate an enhanced output image by;

computing an intermediate image Ŝ

as

18. The method of claim 13 wherein a modified upper-envelope function U′

and a modified lower-envelope function L′

are used to generate an enhanced output image by;

computing a details function D from upper-envelope and lower-envelope functions U and L and input image signal S_in; and

combining the details function D, modified upper-envelope function U′

, and modified lower-envelope function L′

to produce the enhanced output signal.

19. The method of claim 18

wherein the processing component computes the details function D from upper-envelope and lower-envelope functions U and L and input image signal S_in by