Method for fusing images and apparatus therefor

• US 5,488,674 A
• Filed: 05/12/1993
• Issued: 01/30/1996
• Est. Priority Date: 05/15/1992
• Status: Expired due to Term

• Current Assignee: Sarnoff Corporation (SRI International, Inc.)
• Original Assignee: David Sarnoff Research Center, Inc.
• Inventors: Hingorani, Rajesh, Burt, Peter J., van der Wal, Gooitzen S., Kolczynski, Raymond J.
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): KELLEY, CHRISTOPHER S
• Application Number:
  US08/059,610
• Time in Patent Office: 993 Days
• Field of Search: 382/22, 382/41, 382/17, 382/54, 382/162, 382/284, 382/276, 348/239, 348/389, 348/454, 348/584, 348/598, 395/125, 395/134
• US Class Current: 382/284
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A method for fusing two or more source images to form a composite image with extended information content which may be color augmented and apparatus for forming the composite image from the source images is disclosed. Each source image is decomposed into a number of source images of varying resolution. The decomposed source images are analyzed using directionally sensitive operators to generate a set of oriented basis functions characteristic of the information content of the original images. The oriented basis functions for the composite image are then selected from those of the different source images and the inverse of the decomposition performed to construct the composite image. Color augmentation provides information as to the relative weighting of the contribution of each source to the composite image.

Claims

1. In apparatus for forming a composite image from at least first and second monochrome source images comprising first means for decomposing each of the source images into a set of a plurality of component patterns and deriving a monochrome composite image in response to a weighted average of amplitudes of the component patterns of the source images;

the combination therewith of;

second means for assigning amplitudes of a first of opponent colors to first component patterns of a source image in accordance with their normalized weight contribution to said weighted average of the amplitudes of the component patterns of the source images; and

third means for assigning amplitudes of a second of said opponent colors to second component patterns of a source image in accordance with their normalized weight contributions to said weighted average of the amplitudes of the component patterns of the source images; and

fourth means responsive to outputs from said first, second and third means for augmenting said monochrome composite image with said first and second opponent colors thereby to provide a color composite image.

2. The apparatus of claim 1 wherein said color composite image is an NTSC video image in which said fourth means derives the Y luminance channel thereof from the output of said first means, the I chrominance channel from the output of said second means, and the Q chrominance channel from the output of said third means.

3. The apparatus of claim 1 wherein said first means includes a first means for deriving a first set of given different resolution component patterns of said first source image, a second means for deriving a corresponding second set of said given different resolution component patterns of said second source image, and a first reconstruction means responsive to a weighted average of each corresponding pair of component patterns of said first and second sets having the same resolution for deriving said monochrome composite image;

and wherein;

said second means includes a second reconstruction means responsive to the normalized weights of a first selected group of said given different resolution component patterns of said first and/or second source images for deriving a first weighting image for said first of said opponent colors; and

said third means includes a third reconstruction means responsive to the normalized weights of a second selected group of said given different resolution component patterns of said first and/or second source images for deriving a second weighting image for said second of said opponent colors.

4. The apparatus of claim 3 wherein said first and second means are Laplacian means and said first reconstruction means is an RE or FSD Laplacian pyramid;

and wherein

each of said second and third reconstruction means is a Gaussian reconstruction pyramid deriving respective normalized output amplitudes therefrom.

5. The apparatus of claim 4 wherein said first and second means are gradient pyramids and said first reconstruction means is a gradient pyramid;

and wherein

each of said second and third reconstruction means is a gradient reconstruction means deriving respective normalized output amplitudes therefrom.

6. A method for forming a composite image from N source images where N is greater than one comprising the steps of:

a) decomposing each source image I_n into a plurality L of sets of component patterns P_n (m, l), where n is indicative of the N^th source image and m is the number of patterns in each of the L sets;

(b) computing a match measure M_n1,n2 (m, l) for the source images, where n1 is the first and and n2 is the second of each pair of patterns being matched;

(c) computing a saliency measure S_n (m, l) for a component pattern P_n (m, l);

(d) selecting component patterns from the component pattern sets P_n (m, l) using the match measures M_n1,n2 (m, l) and the saliency measures S_n (m, l) to form a set of component patterns P_c (m, l) for the composite image, where c is indicative of the formed component patterns of the set being composite component patterns; and

(e) constructing the composite image from the set of component patterns P_c (m, l).

7. The method of claim 6 wherein

said component patterns P_n (m, l) into which said plurality L of sets of each source image I_n are decomposed by step (a) are oriented component patterns;

whereby oriented component patterns are selected by step (d) to form the set of component patterns P_c (m, l) from which the composite image is constructed by step (e).