Generating compound images having increased sharpness and reduced noise

• US 7,925,113 B2
• Filed: 04/27/2007
• Issued: 04/12/2011
• Est. Priority Date: 04/27/2007
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Lim, Suk Hwan
• Primary Examiner(s): Cunningham; Gregory F
• Application Number:
  US11/740,976
• Publication Number:
  US 20080267530A1
• Time in Patent Office: 1,446 Days
• Field of Search: 345/615, 382/221, 382/264, 382/284, 702/17, 702/191, 702/194, 702/195
• US Class Current: 382/284
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

Systems and method of generating compound images having increased sharpness and reduced noise are described. In one aspect, a comparison image signal is determined. The comparison image signal includes pixels with values respectively indexing values of pixels of a first image signal in relation to values of spatially corresponding pixels of a second image signal. The comparison image signal is filtered with a spatial filter. A compound image signal is generated. The compound image signal has pixels with values composed of contributions from the pixel values of the first image signal and the pixel values of the second image signal. The contributions are functions of respective values of spatially corresponding pixels in the filtered comparison image signal in relation to values of noise variation in the filtered comparison image signal.

Claims

1. A method, comprising:

determining a comparison image signal comprising pixels with values respectively indexing values of pixels of a first image signal in relation to values of spatially corresponding pixels of a second image signal;

filtering the comparison image signal with a spatial filter; and

generating a compound image signal comprising pixels with values composed of contributions from the pixel values of the first image signal and the pixel values of the second image signal, wherein the contributions are functions of respective values of spatially corresponding pixels in the filtered comparison image signal in relation to values of noise variation in the filtered comparison image signal;

wherein the determining, the filtering. and the generating are performed by computing hardware.

2. The method of claim 1, wherein the determining comprises subtracting ones of the pixel values of the second image signal from the values of spatially corresponding ones of the pixels of the first image signal to produce respective pixel values of the comparison image signal.

3. The method of claim 1, wherein the filtering comprises convolving the comparison image signal with a kernel of a low-pass spatial filter.

4. The method of claim 1, wherein the generating comprises normalizing the filtered comparison image signal with respect to the values of noise variation in the filtered comparison image signal.

5. The method of claim 4, wherein:

the normalizing comprises squaring pixel values of the filtered comparison image signal and dividing the squared pixel values by values of noise variance in the filtered comparison image signal;

the spatial filter is a first low-pass filter defined by a set of coefficient values; and

the normalizing comprises estimating the values of noise variance in the filtered comparison image signal by determining values of noise variance in the first and second images, summing the determined values of noise variance at spatially corresponding pixels in the first and second image signals to produce estimated values of noise variance in the comparison image signal, and filtering the estimated values of noise variance in the comparison image signal with a second low-pass filter defined by a set of coefficients with respective values equal to the squares of the values of corresponding ones of the coefficients of the first low-pass filter.

6. The method of claim 4, further comprising mapping the normalized filtered comparison image signal to produce weights that at least in part determine the contributions of the pixel values of the first and second image signals to the pixel values of the compound image signal.

7. The method of claim 1, further comprising capturing the first image signal with a first exposure time parameter value, and capturing the second image signal with a second exposure time parameter value shorter than the first exposure parameter value.

8. The method of claim 1, wherein the generating comprises computing the values of respective ones of the pixels of the compound image signal from a weighted combination of the values of spatially corresponding ones of the pixels of the first and second image signals.

9. The method of claim 1, further comprising filtering the second image signal with a noise filter before determining the comparison image signal.

10. The method of claim 1, wherein the generating comprises calculating weighted combinations of the values of pixels in spatially corresponding regions of the first and second image signals to produce the compound image signal.

11. The method of claim 10, wherein the calculating comprises calculating the weighted combinations in accordance with a joint bilateral filter.

12. The method of claim 1, further comprising:

determining a second comparison image signal comprising pixels with values respectively indexing values of pixels of a third image signal in relation to values of spatially corresponding pixels of the second image signal; and

filtering the second comparison image signal with a spatial filter;

wherein the generating comprises generating the compound image signal comprising pixels with values composed of contributions from the pixel values of the first image signal, the second image signal, and the third image signal, wherein the contributions are functions of respective values of spatially corresponding pixels in the first and second filtered comparison image signal in relation to corresponding values of noise variation estimated in the first and second filtered comparison image signal.

13. A system, comprising:

a memory storing processor-readable instructions;

a processor coupled to the memory, operable to execute the instructions, and based at least in part on the execution of the instructions operable to perform operations comprising

determining a comparison image signal comprising pixels with values respectively indexing values of pixels of a first image signal in relation to values of spatially corresponding pixels of a second image signal;

filtering the comparison image signal with a spatial filter; and

generating a compound image signal comprising pixels with values composed of contributions from the pixel values of the first image signal and the pixel values of the second image signal, wherein the contributions are functions of respective values of spatially corresponding pixels in the filtered comparison image signal in relation to values of noise variation in the filtered comparison image signal.

14. The system of claim 13, wherein based at least in part on the execution of the instructions, the processor is operable to perform operations further comprising normalizing the filtered comparison image signal with respect to the values of noise variation in the filtered comparison image signal.

15. The system of claim 14, wherein in the normalizing the processing is operable to perform operations comprising squaring pixel values of the filtered comparison image signal and dividing the squared pixel values by estimated values of noise variance in the filtered comparison image signal, and the spatial filter is a first low-pass filter defined by a set of coefficient values;

wherein based at least in part on the execution of the instructions, the processor is operable to perform operations comprising estimating the values of noise variance in the filtered comparison image signal by determining values of noise variance in the first and second images and summing the determined values of noise variance at spatially corresponding pixels in the first and second image signals to produce estimated values of noise variance in the comparison image signal; and

wherein, based at least in part on the execution of the instructions, the processor is operable to perform operations comprising filtering the estimated values of noise variance in the comparison image signal with a second low-pass filter defined by a set of coefficients with respective values equal to the squares of the values of corresponding ones of the coefficients of the first low-pass filter.

16. The system of claim 14, wherein, based at least in part on the execution of the instructions, the processor is operable to perform operations comprising mapping the normalized filtered comparison image signal to produce weights that at least in part determine the contributions of the pixel values of the first and second image signals to the pixel values of the compound image signal.

17. The system of claim 14, wherein based at least in part on the execution of the instructions, the processor is operable to perform operations comprising calculating weighted combinations of the values of pixels in spatially corresponding regions of the first and second image signals to produce the compound image signal.

18. The system of claim 13, wherein, based at least in part on the execution of the instructions, the processor is operable to perform operations comprising:

determining a second comparison image signal comprising pixels with values respectively indexing values of pixels of a third image signal in relation to values of spatially corresponding pixels of the second image signal;

filtering the second comparison image signal with a spatial filter; and

generating the compound image signal comprising pixels with values composed of contributions from the pixel values of the first image signal, the second image signal, and the third image signal, wherein the contributions are functions of respective values of spatially corresponding pixels in the first and second filtered comparison image signal in relation to corresponding values of noise variation estimated in the first and second filtered comparison image signal.

19. The system of claim 13, further comprising an image sensing system operable to capture the first image signal with a first exposure time parameter value and operable to capture the second image signal with a second exposure time parameter value shorter than the first exposure parameter value.

20. A non-transitory machine readable medium storing machine-readable instructions causing a machine to perform operations comprising:

determining a comparison image signal comprising pixels with values respectively indexing values of pixels of a first image signal in relation to values of spatially corresponding pixels of a second image signal;

filtering the comparison image signal with a spatial filter; and

generating a compound image signal comprising pixels with values composed of contributions from the pixel values of the first image signal and the pixel values of the second image signal, wherein the contributions are functions of respective values of spatially corresponding pixels in the filtered comparison image signal in relation to values of noise variation in the filtered comparison image signal.