Enhancing the resolution of multi-spectral image data with panchromatic image data using super resolution pan-sharpening

• US 5,949,914 A
• Filed: 03/17/1997
• Issued: 09/07/1999
• Est. Priority Date: 03/17/1997
• Status: Expired due to Term

• Current Assignee: DigitalGlobe, Inc. (Galileo Parent, Inc.)
• Original Assignee: Space Imaging L.P.
• Inventors: Yuen, Patrick W.
• Primary Examiner(s): Tadayon, Bijan
• Assistant Examiner(s): Novik, Dmitry A.
• Application Number:
  US08/819,046
• Time in Patent Office: 904 Days
• Field of Search: 382/254, 382/191, 382/266, 382/279, 382/299, 382/276, 348/606, 348/625, 348/30, 348/630, 358/447, 358/532
• US Class Current: 382/254
• CPC Class Codes: G01C 11/025 by scanning the object
• CPC Class Codes: G06T 2207/10036 Multispectral image; Hyperspectral image
• CPC Class Codes: G06T 2207/10041 Panchromatic image
• CPC Class Codes: G06T 3/4061 by injecting details from different spectral ranges
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening

Abstract

The present invention discloses a super resolution pan-sharpening technique that is used to increase the resolution of a multi-spectral signal using a panchromatic signal. Typically, multi-spectral signals, i.e., signals that contain a defined spectral band, have lower resolution than panchromatic signals since the multi-spectral signals normally contain fewer photons and require larger detectors for similar exposure periods. The technique of the present invention utilizes the panchromatic signal to increase the resolution of the multi-spectral signals to provide an increased resolution color output. The resolution of other spectral signals, such as infrared signals and UV signals, or any other desired signal, can also be increased in accordance with the techniques of the present invention.

Claims

1. A method of enhancing the resolution of multi-spectral image data having a plurality of spectral bands using panchromatic image data having a data record size and resolution that is greater than said resolution of said multi-spectral image data comprising the steps of:

using a modified Poisson Maximum A-Posteriori (MAP) super resolution technique for at least one iteration on each spectral band of said multi-spectral image;

generating pan-sharpened multi-spectral image data for each said spectral band based on said panchromatic image data;

combining said pan-sharpened multi-spectral image data for at least two of said spectral bands to produce pan-sharpened color image data that has a higher resolution than said multi-spectral image data; and

,

generating a pan-sharpened color image from said pan-sharpened color image data.

2. The method of claim 1 wherein said step of using said MAP super resolution algorithm comprises the steps of:

upsampling said multi-spectral image data for each spectral band to match said data record size of said panchromatic image data and produce upsampled multi-spectral image data;

generating error data by convolving said panchromatic image data and a normalized point spread function;

generating result data by dividing said upsampled image data by said error data and subtracting one;

generating correction data by convolving said result data with said normalized point spread function; and

, generating said pan-sharpened multi-spectral image data by exponentiating said panchromatic image data with said correction data for each spectral band.

3. The method of claim 1 further comprising the steps of:

transforming said pan-sharpened color image data from a RGB model, having red, green and blue components, to a HSI model having hue, saturation and intensity components;

modifying said saturation component of said HSI model to cover a dynamic range from values of approximately zero to approximately one;

spatially filtering said intensity component with a high boost filter for edge enhancement;

transforming said HSI model of said pan-sharpened color image data back to said RGB model.

4. A method of generating multi-spectral images from first image data having at least two spectral bands with a predetermined first range of resolution and a second image data having a third spectral band with a second range of resolution that is higher than said first range of resolution comprising the steps of:

detecting an image to generate said first and second image data;

upsampling said first image data to match the data record size of said second image data to produce upsampled first image data;

dividing said upsampled first image data by said second image data and subtracting one from the quotient to produce result data;

exponentiating said second image data with said result data to produce pan-sharpened image data; and

,

generating said multi-spectral images from said pan-sharpened image data.

5. A method of generating multi-spectral images from first image data having at least two spectral bands with a predetermined first range of resolution and second image data having a third spectral band with a second range of resolution that is higher than said first range of resolution comprising the steps of:

detecting an image to generate said first and second image data;

upsampling said first image data to match the data record size of said second image data to produce upsampled first image data;

generating error data by convolving said second image data with a point spread function that describes corrections for detector optics used to detect said image;

dividing said unsampled first image data by said error data and subtracting one from the quotient to produce result data;

generating correction data by convolving said result data with said point spread function;

exponentiating said second image data with said correction data to produce pan-sharpened image data; and

,

generating said multi-spectral images from said pan-sharpened image data.

6. A method of increasing the resolution of first image data having a first predetermined resolution using second image data having a second predetermined resolution that is higher than said first predetermined resolution comprising the steps of:

dividing said first image data by said second image data to produce a quotient;

subtracting one from said quotient to produce result data; and

,

exponentiating said second image data with said result data to produce third image data that is representative of said first image data having a resolution that is substantially equivalent to said second predetermined resolution.

7. The method of claim 6 further comprising the step of:

generating an image using said third image data.

8. A device for generating high resolution multi-spectral data from low resolution multi-spectral data and high resolution panchromatic data comprising:

a plurality of first detectors that generate said low resolution multi-spectral data;

at least one second detector that generates said high resolution panchromatic data;

a processor coupled to said plurality of first detectors and said second detector, said processor including;

a divider that divides said low resolution multi-spectral data by said high resolution panchromatic data to produce a quotient;

a subtractor that subtracts one from said quotient to produce result data; and

,

an exponentiator that exponentiates said high resolution panchromatic data with said result data to produce said high resolution multi-spectral data.

9. A device for generating high resolution multi-spectral data from low resolution multi-spectral data and high resolution panchromatic data comprising:

a plurality of first detectors that generate said low resolution multi-spectral data;

at least one second detector that generates said high resolution panchromatic data;

a processor coupled to said plurality of first detectors and said second detector, said processor including;

a first convolver that convolves said high resolution panchromatic data with a point spread function to produce error data;

a divider that divides said low resolution multi-spectral data by said error data to produce a quotient;

a subtractor that subtracts one from said quotient to produce result data;

a second convolver that convolves said result data and said point spread function to produce correction data; and

,

an exponentiator that exponentiates said high resolution panchromatic data with said correction data to produce said high resolution multi-spectral data.

10. The device of claim 9 wherein said processor further comprises:

an upsampler that upsamples said low resolution multi-spectral data to match the data record size of said high resolution panchromatic data prior to said divider dividing said low resolution multi-spectral data by said error data.

11. The device of claim 10 further comprising:

a first transformer that transforms said high resolution multispectral data from a RGB model, having red, green and blue components, to a HSI model having hue, saturation and intensity components;

a linear modifier that modifies said saturation component to cover a dynamic range of values from approximately zero to approximately one;

a high boost filter that spatially filters said intensity component for edge enhancement;

a second transformer that transforms said HSI mode of said high resolution multispectral data back to said RGB model.

12. A system for generating high resolution multi-spectral images comprising:

a multisensor detector that generates a plurality of low resolution multi-spectral data signals and at least one high resolution panchromatic data signal from an object that is scanned by said system;

a processor coupled to said multisensor detector that divides said low resolution multi-spectral data signals by said high resolution panchromatic data signal to produce a quotient, subtracts one from said quotient to produce result data and exponentiates said high resolution panchromatic data with said result data to produce pan-sharpened multi-spectral data; and

,

a display that generates said high resolution multi-spectral image from said pan-sharpened multi-spectral data.

13. The system of claim 12 wherein said system comprises a document scanner.

14. The system of claim 12 wherein said multisensor detector is space based.

15. The system of claim 12 wherein said multisensor detector is airborne.

16. The system of claim 12 wherein said display comprises printer.