Projective pan sharpening methods and apparatus

• US 6,097,835 A
• Filed: 07/23/1997
• Issued: 08/01/2000
• Est. Priority Date: 07/23/1997
• Status: Expired due to Term

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Lindgren, John E.
• Primary Examiner(s): Au, Amelia
• Assistant Examiner(s): COOPERRIDER, FRED E
• Application Number:
  US08/898,814
• Time in Patent Office: 1,105 Days
• Field of Search: 382/191, 382/294, 382/299, 382/300, 348/143, 348/144, 348/147, 348/606, 348/625, 356/402, 356/406, 356/416, 356/419
• US Class Current: 382/162
• CPC Class Codes: G06T 2207/10036 Multispectral image; Hyperspectral image
• CPC Class Codes: G06T 2207/10041 Panchromatic image
• CPC Class Codes: G06T 2207/30181 Earth observation
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening

Abstract

A projective panchromatic sharpening method and apparatus combines registered high spatial resolution panchromatic imagery and lower spatial resolution multispectral imagery to synthesize higher spatial resolution multispectral imagery. The degree of misregistration between bands and the correlation of intensity values are key factors in generating radiometrically accurate and visually crisp output images. The projective panchromatic sharpening method and apparatus comprises two components. The first component determines the linear radiometric relationship between the panchromatic and multispectral bands, and the second component processes pixels to produce the sharpened product.

Claims

1. A method of combining registered high spatial resolution panchromatic imagery and lower spatial resolution multispectral imagery existing in a plurality of spectral bands to synthesize higher spatial resolution of the multispectral imagery as an ensemble of the plurality of spectral bands, said method comprising,

collecting high spatial resolution panchromatic imagery,

collecting lower spatial resolution multispectral imagery existing in a plurality of spectral bands, and

initializing the collected imagery to produce a global model relating spatially overlapping multispectral and panchromatic intensities by determining the linear radiometric relationship between the panchromatic imagery and the multispectral imagery bands as an ensemble of the plurality of spectral bands,

said initializing compromising,

generating uniformly distributed registered panchromatic and multispectral image samples from the collected panchromatic imagery and the collected multispectral imagery,

determining color weights from the generated panchromatic and multispectral image samples in the form of a weighting vector which is normal to a panchromatic hyperplane containing multispectral imagery intensity vectors and modulated by panchromatic intensities,

and then pixel by pixel panchromatic sharpening processing the multispectral imagery in said ensemble by using the determined color weights and associated panchromatic intensity to produce a sharpened product,

said pixel by pixel processing comprising,

spatially resampling the collected multispectral image intensities to panchromatic imagery resolution,

projecting the resampled multispectral image intensities onto said hyperplane to adjust the intensities of the resampled multispectral imagery to the closest global model intensity contained in said hyperplane and to thereby produce a synthesized multispectral imagery which is sharpened to have substantially the spatial resolution of the panchromatic imagery, and

wherein sharpening the imagery in the plurality of spectral bands as an ensemble rather than sharpening the imagery in the plurality of spectral bands sequentially band by band reduces the amount of processing required and reduces the time required for said sharpening of the multispectral imagery.

2. The invention defined in claim 1 including clamping the adjusted intensities of the resampled multispectral imagery to a feasible output range.

3. The invention defined in claim 2 including outputting the clamped adjusted intensities as a sharpened multispectral imagery.

4. The invention defined in claim 1 wherein the spectral resolutions of the multispectral imagery are all within the range of spectral resolution of the panchromatic imagery.

5. The invention defined in claim 1 wherein the spatial resolution of the panchromatic imagery is one unit by one unit and the spatial resolution of the multispectral imagery is four units by four units.

6. The invention defined in claim 1 wherein the collected high spatial resolution panchromatic imagery and the collected lower spatial resolution multispectral imagery cover the same image location on the Earth.

7. The invention defined in claim 1 wherein there are n multispectral bands and wherein there is a panchromatic band having substantially complete spectral overlap with the multispectral bands and wherein there are spatially registered pixels with allowances for slight misregistration, and wherein there is a linear relationship between intensities in the panchromatic band and the multispectral bands and wherein a pan pixel intensity value is denoted as p and wherein there are spatially overlapping multispectral band intensity values), m_i where i=0, . . . , n-1, wherein the global model is mathematically expressed ##EQU6## and wherein m is a column vector of multispectral imagery intensities, b is a constant additive bias in panchromatic intensity values, and ε

is an error term, and wherein the vector w is normal to flee hyperplane containing multispectral imagery intensity vectors at a distance (p-b)/|w| from the origin.

8. The invention defined in claim 7 wherein accurate band sensitivities are known and including calculating the weighting vector, w, and the pan bias, b, directly from radiometry measurements.

9. The invention defined in claim 7 wherein the model is unknown and wherein linear, standard least squares methods are used to determine w and b.

10. The invention defined in claim 9 including coping with errors introduced by small band-to-band misregistration by averaging multispectral imagery samples over a square area to a common resolution before performing the least squares fit.

11. Apparatus for combining registered high spatial resolution panchromatic imagery and lower spatial resolution multispectral imagery existing in a plurality of spectral bands to synthesize higher spatial resolution of the multispectral imagery as an ensemble of the plurality of spectral bands, said apparatus comprising,

pan means for collecting high spatial resolution panchromatic imagery,

MSI means for collecting lower spatial resolution multispectral imagery exiting in a plurality of spectral bands, and

model means for initializing the collected imagery to produce a global model relating spatially overlapping multispectral and panchromatic intensities by determining the linear radiometric relationship between the panchromatic imagery and the multispectral imagery bands as an ensemble of the plurality of spectral bands,

said model means comprising,

generating means for generating uniformly distributed registered panchromatic and multispectral image samples from the collected panchromatic imagery and the collected multispectral imagery,

color weight determining means for determining color weights from the generated panchromatic and multispectral image samples in the form of a weighting vector which is normal to a panchromatic hyperplane containing multispectral imagery intensity vectors and modulated by panchromatic intensities,

and panchromatic sharpening means for then pixel by pixel panchromatic sharpening processing the multispectral imagery in said ensemble by using the determined color weights and associated panchromatic intensity to produce a sharpened product,

said panchromatic sharpening means comprising,

spatially resampling means for resampling the collected multispectral image intensities to panchromatic imagery resolution,

projecting means for projecting the resampled multispectral image intensities onto said hyperplane to adjust the intensities of the resampled multispectral imagery to the closest global model intensity contained in said hyperplane and to thereby produce a synthesized multispectral imagery which is sharpened to have substantially the spatial resolution of the panchromatic imagery, and

wherein sharpening the imagery in the plurality of spectral bands as an ensemble rather than sharpening the imagery in the plurality of spectral bands sequentially band by band reduces the amount of processing required and reduces the time required for said sharpening of the multispectral imagery.

12. The invention defined in claim 11 including clamping means for clamping the adjusted intensities of the resampled multispectral imagery to a feasible output range.

13. The invention defined in claim 12 including output means for outputting the clamped adjusted intensities as a sharpened multispectral imagery.

14. The invention defined in claim 11 wherein the spectral resolutions of the multispectral imagery are all within the range of spectral resolution of the panchromatic imagery.

15. The invention defined in claim 11 wherein the spatial resolution of the panchromatic imagery is one unit by one unit and the spatial resolution of the multispectral imagery is four units by four units.

16. The invention defined in claim 11 wherein the collected high spatial resolution panchromatic imagery and the collected lower spatial resolution multispectral imagery cover the same image location on the Earth.

17. The invention defined in claim 11 wherein there are n multispectral bands and wherein there is a panchromatic band having substantially complete spectral overlap with the multispectral bands and wherein there are spatially registered pixels with allowances for slight misregistration, and wherein there is a linear relationship between intensities in the panchromatic band and the multispectral bands and wherein a pan pixel intensity value is denoted as p and wherein there are spatially overlapping multispectral band intensity values, m₁ where i=0, . . . , n-1, wherein the global model is mathematically expressed ##EQU7## and wherein m is a column vector of multispectral imagery intensities, b is a constant additive bias in panchromatic intensity values, and ε

is an error term, and wherein the vector w is normal to the hyperplane containing multispectral imagery intensity vectors at a distance (p-b)/|w| from the origin.

18. The invention defined in claim 17 wherein accurate band sensitivities are known and including calculating means for calculating the weighting vector, w, and the pan bias, b, directly from radiometry measurements.

19. The invention defined in claim 17 wherein the model is unknown and wherein linear, standard least squares methods are used to determine w and b.

20. The invention defined in claim 19 including coping means for coping with errors introduced by small band-to-band misregistration by averaging multispectral imagery samples over a square area to a common resolution before performing the least squares fit.