Salience Preserving Image Fusion

• US 20080080787A1
• Filed: 09/28/2006
• Published: 04/03/2008
• Est. Priority Date: 09/28/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Chao, Ye, Zhongfu, Tang, Xiaoou, Yang, Qiong
• Granted Patent: US 7,636,098 B2
• Time in Patent Office: Days
• US Class Current: 382/284
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction

Abstract

Salience-preserving image fusion is described. In one aspect, multi-channel images are fused into a single image. The fusing operations are based on importance-weighted gradients. The importance weighted gradients are measured using respective salience maps for each channel in the multi-channel images.

Claims

1. A method at least partially implemented by a computer, the method comprising:

fusing multi-channel images into a single image based on importance-weighted gradients, each multi-channel image comprising multiple channels; and

wherein the importance weighted gradients are measured using respective salience maps for each channel in the multi-channel images.

2. The method of claim 1, wherein fusing the multi-channel images further comprises:

for each channel in each image of the multi-channel images;

adjusting the channel to have a same mean gradient as a maximum mean gradient channel;

measuring local salience for respective pixels in the channel; and

assigning a normalized weight to each pixel of the pixels based on respective ones of the local salience measurements;

generating an importance weighted contrast form based on normalized weights associated with respective ones of the pixels in the channels;

constructing, using the importance weighted contrast form, a target gradient based on pixels in each of the channels; and

creating the single image from the target gradient.

3. The method of claim 2, wherein constructing the target gradient is accomplished using pixel-based Eigen decomposition operations in view of the importance weighted contrast form.

4. The method of claim 1, wherein the method further comprises removing halo effects associated with the target gradient.

5. The method of claim 4, wherein removing the halo effects further comprises implementing dynamic range compression operations on a pixel-by-pixel basis.

6. The method of claim 1, wherein fusing the multi-channel images removes color from the single image.

7. The method of claim 1, wherein the method further comprises presenting the single image to a viewer.

8. A computer-readable medium comprising computer-program instructions executable by a processor, the computer-program instructions, when executed, for performing operations comprising:

for each image in a set of source images, each image comprising multiple channels of pixel information, calculating a respective salience map of multiple salience maps for a gradient of each channel in the image;

weighting, based on respective ones of multiple salience maps, respective contributions of each channel of the multiple channels to global statistics; and

highlighting, based on weighted contribution of each channel to global statistics, gradients with high saliency in a target gradient to preserve salient features in a single image.

9. The computer-readable medium of claim 8, wherein before calculating the respective salience map, the method further comprises adjusting each channel of the multiple channels in each image of the source images to have a same mean gradient as a maximum mean gradient channel so that the channels can be compared and computed at a same gradient level.

10. The computer-readable medium of claim 8, wherein the gradients are represented in an importance weighted contrast form, and wherein highlighting the gradients with high saliency in the target gradient further comprises:

constructing, using the importance weighted contrast form, the target gradient based on pixels in each of the channels; and

fusing the single image from the target gradient.

11. The computer-readable medium of claim 8, wherein the computer-program instructions further comprise instructions for removing halo effects from the target gradient.

12. The computer-readable medium of claim 8, wherein the computer-program instructions further comprise instructions for implementing dynamic range compression operations to remove the halo effects.

13. The computer-readable medium of claim 8, wherein the computer-program instructions for calculating the salience maps, weighting the respective contributions, and highlighting the gradients with high saliency are implemented responsive to receiving a request to remove color from respective ones of the source images.

14. The computer-readable medium of claim 8, wherein the computer-program instructions further comprise instructions for presenting the single image to a viewer.

15. A computing device comprising:

a processor; and

a memory coupled to the processor, the memory comprising computer-program instructions executable by the processor for implementing operations comprising;

receiving multiple multi-channel images, each multi-channel image comprising multiple channels of pixel information; and

responsive to receiving the multi-channel images, fusing gradients from the multiple channels associated with each of the multi-channel images into a salience-preserving fused image, operations for fusing the gradients being based on local salience measurements for each pixel of the pixel information.

16. The computing device of claim 15, further comprising presenting the salience-preserving fused image to a viewer.

17. The computing device of claim 15, further comprising:

for each channel of the multiple channels;

adjusting the channel to have a same mean gradient as a maximum mean gradient channel; and

calculating the local salience measurements for respective pixels in each of the channels.

18. The computing device of claim 16, further comprising for each of the multi-channel images, assigning a normalized weight to each pixel in each channel of the multiple channels based on respective ones of the local salience measurements.

19. The computing device of claim 18, further comprising:

generating an importance weighted contrast form based on normalized weights associated with respective ones of the pixels in the channels; and

constructing, using the importance weighted contrast form, a target gradient based on pixels in each of the channels.

20. The computing device of claim 19 further comprising reducing, using dynamic range compression operations, halo(s) associated with the target gradient.