SYSTEMS AND METHODS FOR FUSING INFRARED IMAGE AND VISIBLE LIGHT IMAGE

• US 20190318463A1
• Filed: 06/27/2019
• Published: 10/17/2019
• Est. Priority Date: 12/27/2016
• Status: Active Grant

• Current Assignee: Zhejiang Dahua Technology Company Limited
• Original Assignee: Zhejiang Dahua Technology Company Limited
• Inventors: ZHANG, Dong, WANG, Song
• Granted Patent: US 11,030,731 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 2207/20182 Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/90 Determination of colour characteristics

Abstract

A system and a method for image fusion are provided. The method may comprise: obtaining a visible light image and an infrared image relating to a same scene; performing a first decomposition to the visible light image to obtain a first high-frequency component of the visible light image and a first low-frequency component of the visible light image; performing a first decomposition to the infrared image to obtain a first high-frequency component of the infrared image and a first low-frequency component of the infrared image; fusing the first high-frequency component of the visible light image and the first high-frequency component of the infrared image based on a first algorithm to generate a first fused high-frequency component; and performing reconstruction based on the first fused high-frequency component, the first low-frequency component of the visible light image, and the first low-frequency component of the infrared image to generate a fused image.

Claims

1. An image fusion system, comprising:

a processor coupled to a storage; and

the storage configured to store instructions, the instructions, when executed by the processor, causing the image fusion system to effectuate a method comprising;

obtaining a visible light image and an infrared image relating to a same scene;

performing a first decomposition to the visible light image to obtain a first high-frequency component of the visible light image and a first low-frequency component of the visible light image;

performing a first decomposition to the infrared image to obtain a first high-frequency component of the infrared image and a first low-frequency component of the infrared image;

fusing the first high-frequency component of the visible light image and the first high-frequency component of the infrared image based on a first algorithm to generate a first fused high-frequency component; and

performing reconstruction based on the first fused high-frequency component, the first low-frequency component of the visible light image, and the first low-frequency component of the infrared image to generate a fused image.

2. The system of claim 1, wherein the image fusion system is further caused to effectuate the method comprising:

preprocessing the visible light image and the infrared image, wherein the preprocessing comprises;

registering the visible light image and the infrared image;

performing haze removal on the visible light image; and

performing color filtering on the visible light image.

3. The system of claim 2, wherein the performing haze removal on the visible light image comprises:

determining a dark channel matrix;

determining an atmospheric light;

determining an initial transmittance based on the dark channel matrix and the atmospheric light;

determining a bright area to obtain an optimized transmittance based on the initial transmittance; and

performing the haze removal based on the optimized transmittance.

4. The system of claim 1, wherein the first decomposition is pyramid decomposition.

5. The system of claim 4, wherein the pyramid decomposition comprises:

performing low-pass filtering on a subject image to obtain a filtered image;

downsampling the filtered image to obtain a downsampled image;

upsampling the downsampled image to obtain an upsampled image; and

performing band-pass filtering on the upsampled image to obtain a decomposed subject image.

6. The system of claim 5, the subject image comprising the first high-frequency component of the visible light image, the first low-frequency component of the visible light image, the first high-frequency component of the infrared image, or the first low-frequency component of the infrared image.

7. The system of claim 5, wherein the image fusion system is further caused to effectuate the method comprising fusing the first low-frequency component of the visible light image and the first low-frequency component of the infrared image, wherein the fusing comprises:

determining a difference between the first low-frequency component of the visible light image and the first low-frequency component of the infrared image;

determining whether the difference exceeds a threshold;

designating, in response to the determination that the difference exceeds the threshold, the low-frequency component of the visible light image as a first fused low-frequency component; and

determining, in response to the determination that the difference is lower than the threshold, the first fused low-frequency component based on a second algorithm.

8. The system of claim 7, the fusing the first high-frequency component of the visible light image and the first high-frequency component of the infrared image comprises:

determining the first fused high-frequency component based on a third algorithm.

9. The system of claim 8, wherein the performing the reconstruction comprises:

performing an inverse pyramid decomposition based on the first fused low-frequency component and the first fused high-frequency component.

10. The system of claim 1, wherein the image fusion system is further caused to effectuate the method comprising:

filtering the first high-frequency component of the visible light image by performing one or both of mapping and median filtering.

11. The system of claim 10, wherein the image fusion system is further caused to effectuate the method comprising:

amplifying the first high-frequency component of the infrared image.

12. The system of claim 7, wherein the image fusion system is further caused to effectuate the method comprising:

performing a second decomposition to the first low-frequency component of the visible light image to obtain a second high-frequency component of the visible light image and a second low-frequency component of the visible light image;

performing a second decomposition to the first low-frequency component of the infrared image to obtain a second high-frequency component of the infrared image and a second low-frequency component of the infrared image; and

fusing the second high-frequency component of the visible light image and the second high-frequency component of the infrared image based on the first algorithm to generate a second fused high-frequency component.

13. The system of claim 12, the performing the reconstruction further including:

fusing the second low-frequency component of the visible light image and the second low-frequency component of the infrared image based on the second algorithm to generate a second fused low-frequency component; and

generating the fused image based on the first fused high-frequency component, the second fused high-frequency component, and the second fused low-frequency component.

14. An image fusion method, comprising:

obtaining a visible light image and an infrared image relating to a same scene;

performing a first decomposition to the visible light image to obtain a first high-frequency component of the visible light image and a first low-frequency component of the visible light image;

performing a first decomposition to the infrared image to obtain a first high-frequency component of the infrared image and a first low-frequency component of the infrared image;

fusing the first high-frequency component of the visible light image and the first high-frequency component of the infrared image based on a first algorithm to generate a first fused high-frequency component; and

performing reconstruction based on the first fused high-frequency component, the first low-frequency component of the visible light image, and the first low-frequency component of the infrared image to generate a fused image.

15. The method of claim 14, wherein the first decomposition is pyramid decomposition.

16. The method of claim 15, wherein the pyramid decomposition comprises

performing low-pass filtering on a subject image to obtain a filtered image;

downsampling the filtered image to obtain a downsampled image;

upsampling the downsampled image to obtain an upsampled image; and

performing band-pass filtering on the upsampled image to obtain a decomposed subject image.

17. The method of claim 14, the method further comprising:

filtering the first high-frequency component of the visible light image by performing mapping and median filtering.

18. The method of claim 14, the method further comprising:

amplifying the first high-frequency component of the infrared image.

19. The method of claim 14, the method further comprising:

performing a second decomposition to the first low-frequency component of the visible light image to obtain a second high-frequency component of the visible light image and a second low-frequency component of the visible light image;

performing a second decomposition to the first low-frequency component of the infrared image to obtain a second high-frequency component of the infrared image and a second low-frequency component of the infrared image; and

fusing the second high-frequency component of the visible light image and the second high-frequency component of the infrared image based on the first algorithm to generate a second fused high-frequency component.

20. A non-transitory computer readable medium storing instructions, the instructions, when executed by a computing device, causing the computing device to implement a method, comprising:

obtaining a visible light image and an infrared image relating to a same scene;

performing a first decomposition to the visible light image to obtain a first high-frequency component of the visible light image and a first low-frequency component of the visible light image;

performing a first decomposition to the infrared image to obtain a first high-frequency component of the infrared image and a first low-frequency component of the infrared image;

fusing the first high-frequency component of the visible light image and the first high-frequency component of the infrared image based on a first algorithm to generate a first fused high-frequency component; and

performing reconstruction based on the first fused high-frequency component, the first low-frequency component of the visible light image, and the first low-frequency component of the infrared image to generate a fused image.

21-24. -24. (canceled)