Converting an image from a dual-band sensor to a visible color image

• US 9,544,562 B2
• Filed: 10/17/2013
• Issued: 01/10/2017
• Est. Priority Date: 10/17/2013
• Status: Active Grant

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Inventors: Schachter, Bruce J., Vanmaasdam, Peter J., Riddle, Jack Gordon
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): NOH, JAE NAM
• Application Number:
  US14/056,306
• Publication Number:
  US 20150288941A1
• Time in Patent Office: 1,181 Days
• Field of Search: 348/164
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20
• CPC Class Codes: H04N 9/77 Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase circuits for matrixing H04N9/67

Abstract

One embodiment describes a method for converting a dual-band image to a visible color image. The method includes converting the dual-band image to data in a placeholder image in a placeholder color space. The placeholder color space employs a wavelet-based image fusion technique while mimicking the oppositional color processing technique of the human visual system. The placeholder image is further mapped into visible color space, generating the visible color image.

Claims

1. An apparatus for displaying dual-band image sensor data as a visible color image, comprising:

a dual-band image sensor;

a non-transitory memory storing machine-readable instructions to at least;

fuse different bands of the dual-band image sensor data using wavelet-based image fusion to generate the L* component of an {L*, a*, b* } color space placeholder image;

generate the a* and b* components of the placeholder image by;

generating an {R, G, B } visible color space temporary image having red, green, and blue channels, wherein the red channel of the temporary image is created from one band of the dual-band image sensor, the green channel of the temporary image is created from the other band of the dual-band image sensor, and the blue channel of the temporary image is created from either the product of the two bands of the dual-band image sensor or the ratio of the two bands of the dual-band image sensor;

generating the a* component of the placeholder image as a function of the red and green channels of the temporary image; and

generating the b* component of the placeholder image as a function of the red, green, and blue channels of the temporary image; and

converting the placeholder image to an {R, G, B } visible color space output image having red, green, and blue channels, wherein the red channel of the output image is created as the sum of the L* component and the sum of 1/2 times the a* component and 1/3 times the b* component, the green channel of the output image is created as the difference between the L* component and the sum of 1/2 times the a* component and 1/3 times the b* component, and the blue channel of the output image is created as the difference between the L* component and 2/3 times the b* component; and

a processor, operably coupled to the memory and the dual-band image sensor, configured to execute the machine-readable instructions.

2. The apparatus of claim 1, wherein the dual-band image sensor is a dual-band forward-looking infrared sensor or at least two of a long-wave infrared sensor, a mid-wave infrared sensor, a short-wave infrared sensor, a monochrome visible sensor, a near infrared sensor, or an image intensified image sensor.

3. The apparatus of claim 1, wherein:

the image from the dual-band sensor comprises long-wave infrared data and mid-wave infrared data; and

the a* and b* components of the placeholder image are further generated by expanding the dynamic ranges of the a* and b* channels to reduce redundancy between the two bands of the dual-band image sensor, such that data within the placeholder image exhibits a decreased redundancy relative to a pairing of the long-wave infrared data and the mid-wave infrared data.

4. The apparatus of claim 1, wherein:

the image from the dual-band sensor comprises long-wave infrared data and mid-wave infrared data;

the red channel of the temporary image comprises the long-wave infrared data;

the green channel of the temporary image comprises the mid-wave infrared data; and

the blue channel of the temporary image comprises data that represents either the product or the ratio of the long-wave infrared data and the mid-wave infrared data.

5. The apparatus of claim 3, wherein the expansion of the dynamic ranges of the a* and b* channels is performed by stretching the histogram of a* and b* image values about their respective image mean values.

6. The apparatus of claim 1, wherein the L* component of the placeholder image is generated using only a single wavelet frequency.

7. The apparatus of claim 6, wherein the single wavelet frequency is a function of range in units of distance.

8. The apparatus of claim 1, wherein the processor further facilitates the execution of the machine-readable instructions to apply a mean filter, a median filter, a high-pass filter, a low-pass filter, or an unsharp-mask filter to at least one of the image from the dual-band sensor, the placeholder image, or the visible color image.

9. A non-transitory computer-readable storage medium storing instructions executable by an associated processor to perform a method for converting a dual-band image to a visible color image, the method comprising:

fusing different bands of the dual-band image using wavelet-based image fusion to generate the L* component of an {L*, a*, b*} color space placeholder image;

generating the a* and b* components of the placeholder image by;

generating an {R, G, B} visible color space temporary image having red, green, and blue channels, wherein the red channel of the temporary image is created from one band of the dual-band image sensor, the green channel of the temporary image is created from the other band of the dual-band image sensor, and the blue channel of the temporary image is created from either the product of the two bands of the dual-band image sensor or the ratio of the two bands of the dual-band image sensor;

generating the a* component of the placeholder image as a function of the red and green channels of the temporary image; and

generating the b* component of the placeholder image as a function of the red, green, and blue channels of the temporary image; and

converting the placeholder image to an {R, G, B } visible color space output image having red, green, and blue channels, wherein the red channel of the output image is created as the sum of the L* component and the sum of 1/2 times the a* component and 1/3 times the b* component, the green channel of the output image is created as the difference between the L* component and the sum of 1/2 times the a* component and 1/3 times the b* component, and the blue channel of the output image is created as the difference between the L* component and 2/3 times the b* component; and

emphasizing a feature of the output image based on the converting the placeholder image to the output image.

10. The non-transitory computer-readable storage medium of claim 9, wherein the first channel of the dual-band image comprises long-wave infrared data and the second channel of the dual-band image comprises mid-wave infrared data.

11. The non-transitory computer-readable storage medium of claim 10, wherein:

the red channel of the temporary image comprises the long-wave infrared data;

the green channel of the temporary image comprises the mid-wave infrared data; and

the blue channel of the temporary image comprises data that represents either the product or the ratio of the long-wave infrared data and the mid-wave infrared data.

12. The non-transitory computer-readable storage medium of claim 11, wherein the a* and b* components of the placeholder image are further generated by expanding the dynamic ranges of the a* and b* channels to reduce redundancy between the two bands of the dual-band image sensor, such that data within the placeholder image exhibits a decreased redundancy relative to a pairing of the long-wave infrared data and the mid-wave infrared data.

13. The non-transitory computer-readable storage medium of claim 9, wherein the L* component of the placeholder image is generated using only a single wavelet frequency, the frequency being a function of range in units of distance.

14. A non-transitory computer-readable device storing executable instructions executable by an associated processor to perform operations that convert a dual-band infrared image to a visible color image, the operations comprising:

reducing noise in the dual-band infrared image;

standardizing a dynamic range of a first band of the dual-band infrared image and a dynamic range of a second band of the dual-band infrared image;

converting the dual-band infrared image with reduced noise and standardized dynamic range of the first band and the second band to a placeholder image in an {L*, a*, b* } color space by;

fusing the first and second bands using wavelet-based image fusion to generate the L* component of the placeholder image;

generating the a* and b* components of the placeholder image by;

generating an {R, G, B] visible color space temporary image having red, green, and blue channels, wherein the red channel of the temporary image is created from the first band, the green channel of the temporary image is created from the second band, and the blue channel of the temporary image is created from either the product of or the ratio of the first and second bands;

generating the a* component of the placeholder image as a function of the red and green channels of the temporary image; and

generating the b* component of the placeholder image as a function of the red, green, and blue channels of the temporary image; and

converting the placeholder image to an {R, G, B} visible color space output image having red, green, and blue channels, wherein the red channel of the output image is created as the sum of the L* component and the sum of 1/2 times the a* component and 1/3 times the b* component, the green channel of the output image is created as the difference between the L* component and the sum of 1/2 times the a* component and 1/3 times the b* component, and the blue channel of the output image is created as the difference between the L* component and 2/3 times the b* component.

15. The non-transitory computer-readable device of claim 14, wherein the L* component of the placeholder image is generated using only a single wavelet frequency.

16. The non-transitory computer-readable device of claim 15, wherein the single wavelet frequency is a function of range in units of distance.

17. The non-transitory computer-readable device of claim 14, wherein the the first band comprises long-wave infrared data, the second band comprises mid-wave infrared data, the a* component of the placeholder image is generated by subtracting the green channel from the red channel, and the b* component of the placeholder image is generated by subtracting from the blue channel the sum of the red and green channels.

18. The non-transitory computer-readable device of claim 17, wherein the wavelet-based image fusion comprises generating the L* data based on fusing the long-wave infrared data and the short-wave infrared data according to a dual-tree complex wavelet transform process.

19. The non-transitory computer-readable device of claim 18, wherein the dual-tree complex wavelet transform process comprises:

converting the long-wave infrared data and the mid-wave infrared data to a transform domain according to a dual-tree complex wavelet transform process that increases a shift invariance and increases a directional sensitivity in the L* data; and

forming a transform image by selecting coefficients from the transformed long-wave band image and the transformed mid-wave infrared band image.

20. The non-transitory computer-readable device of claim 19, wherein the dual-tree complex wavelet transform process further comprises producing a fused grayscale image by taking an inverse transform of the transform image.