True color night vision (TCNV) fusion

• US 8,836,793 B1
• Filed: 08/12/2011
• Issued: 09/16/2014
• Est. Priority Date: 08/13/2010
• Status: Active Grant

• Current Assignee: OPTO-Knowledge Systems, Inc.
• Original Assignee: OPTO-Knowledge Systems, Inc.
• Inventors: Gat, Nahum, Kriesel, Jason
• Primary Examiner(s): NGUYEN, HUY THANH
• Application Number:
  US13/209,269
• Time in Patent Office: 1,131 Days
• Field of Search: 386/164
• US Class Current: 348/164
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths

Abstract

True color images are produced by combining data collected from one or more color cameras with data collected from one or more infrared cameras. The produced images are the result of combining portions of the visible light data with portions of the infrared light data that may have been captured at dark, at daytime or at low light level conditions. These images appear as normal color images, with infrared information highlighted in a non-distracting fashion. The true color images that are produced in this fashion can also facilitate identification and reproduction of various objects that may not be visible or readily identifiable in infrared or false color imagery.

Claims

1. A method of producing an image, comprising:

obtaining a first image data from a color imaging device;

obtaining a second image data from an infrared imaging device;

separating chroma data and luma data associated with the first image data;

processing luma data associated with the second image data to produce edge data;

combining the luma data associated with the first image data, the luma data associated with the second image and the edge data to produce a combined luma data; and

fusing the combined luma data with the chroma data to obtain a true color image.

2. The method of claim 1, wherein the edge data is produced by processing luma data associated with both the first and second image data.

3. The method of claim 1, wherein the combined luma data comprises a weighted combination of each of the luma data associated with the first image data, the luma data associated with the second image data, and the edge data.

4. The method of claim 3, wherein proportions of each of the luma data associated with the first image data, the luma data associated with the second image data and the edge data are user adjustable.

5. The method of claim 1, further comprising spatially and/or temporally smoothing the chroma data prior to the fusing.

6. The method of claim 1, further comprising adjusting a composition of the chroma data in accordance with a white balancing technique.

7. The method of claim 1, wherein Near Infrared (NIR) information is excluded from the first image data by placing a Near Infrared (NIR) blocking filter between incoming radiation from a scene and input aperture of the color imaging device.

8. The method of claim 7, wherein the Near Infrared (NIR) blocking filter is removed when intensity level of the incoming radiation drops below a particular level.

9. The method of claim 1, wherein the first image data comprises Visible to Near Infrared (VNIR) data.

10. The method of claim 9, wherein the Visible to Near Infrared (VNIR) data comprises data associated with a narrow portion of the Near Infrared (NIR) spectrum.

11. The method of claim 10, wherein the narrow portion of the Near Infrared (NIR) spectrum corresponds to a range of wavelengths used by a laser-pointing device.

12. The method of claim 1, wherein the fusing is carried out in accordance with real-time fusion parameters that can be adjusted by a user to optimize viewing quality of the true color image.

13. The method of claim 1, further comprising:

obtaining one or more additional image data from one or more additional imaging devices, wherein the combined luma data is obtained by combining the luma data associated with the one or more additional image data with the luma data associated with the first image data, the luma data associated the second image data and the edge data.

14. The method of claim 13, wherein the second imaging device and the one or more additional imaging devices can be any one of a Short Wave Infrared (SWIR) imaging device, a Mid Wave Infrared (MWIR) imaging device, a Long Wave Infrared (LWIR) device, and a radar imaging device.

15. The method of claim 1, further comprising

filtering the luma data associated with the second image data to obtain information associated with a narrow portion of the infrared spectrum, and

assessing the presence or absence of a particular substance within scenery associated with the second image data in accordance with the obtained information.

16. The method of claim 15, wherein the assessing facilitates a determination of mineral content, water content, and/or presence of explosives.

17. A system, comprising:

a color imaging device configured to produce a first image data;

an infrared imaging device configured to produce a second image data; and

a processor configured to

separate chroma data and luma data associated with the first image data,

process luma data associated with the second image data to produce edge data,

combine the luma data associated the first image data, the luma data associated with the second image data, and the edge data to produce a combined luma data, and

fuse the combined luma data with the chroma data to produce a true color image.

18. The system of claim 17, wherein the first imaging device and the second imaging device are configured to share an aperture for capturing incoming radiation from a scene.

19. The system of claim 17, wherein the first imaging device and the second imaging device are positioned in a side-by-side configuration to separately capture incoming radiation from a scene.

20. The system of claim 19, wherein the first imaging device, the second imaging device, and the processor are part of a binocular viewing apparatus.

21. The system of claim 17, wherein processor is configured to produce the edge data by processing luma data associated with both the first and the second image data.

22. The system of claim 17, wherein processor is configured to produce the combined luma data by combining a weighted combination of each of the luma data associated with the first image data, the luma data associated with the second image data, and the edge data.

23. The system of claim 22, wherein proportions of each of the luma data associated with the first image data, the luma data associated with the second image data, and edge data are user adjustable.

24. The system of claim 17, wherein the processor is further configured to spatially and/or temporally smooth the chroma data.

25. The system of claim 17, wherein the processor is configured to adjust a composition of the chroma data in accordance with a white balancing technique.

26. The system of claim 17, further comprising a Near Infrared (NIR) blocking filter positioned between incoming radiation from a scene and input aperture of the color imaging device.

27. The system of claim 26, wherein the processor is configured to detect an intensity level of the incoming radiation and provide an indication to trigger removal of the Near Infrared (NIR) blocking filter when the intensity level drops below a particular level.

28. The method of claim 26, wherein the Near Infrared (NIR) blocking filter is user removable.

29. The system of claim 17, wherein the first image data comprises Visible to Near Infrared (VNIR) data.

30. The system of claim 29, further comprising a Near Infrared (NIR) notch filter positioned between incoming radiation from a scene and input aperture of the color imaging device, wherein the Near Infrared (NIR) notch filter is configured to allow transmission of only a narrow portion of the Near Infrared (NIR) spectrum.

31. The system of claim 29, wherein the processor is configured to fuse the combined luma data with the chroma data in accordance with real-time fusion parameters that can be adjusted by a user to optimize viewing quality of the true color image.

32. The system of claim 17, further comprising:

one or more additional imaging devices, wherein the processor is configured to;

obtain one or more additional image data from the one or more additional imaging devices, and

produce the combined luma data by combining the luma data associated with the one or more additional image data with the luma data associated with the first image data, the luma data associated the second image data and the edge data.

33. The system of claim 17, wherein the second imaging device and the one or more additional imaging devices can be any one of a Short Wave Infrared (SWIR) imaging device, a Mid Wave Infrared (MWIR) imaging device, a Long Wave Infrared (LWIR) device and a radar imaging device.

34. The system of claim 17, wherein the processor is further configured to:

filter the luma data associated with the second image data to obtain information associated with a narrow portion of the infrared spectrum, and

assess the presence or absence of a particular substance within scenery associated with the second image data in accordance with the obtained information.

35. The system of claim 34, wherein processor is configured to facilitate a determination of mineral content, water content, and/or presence of explosives.

36. A computer program product, embodied on a non-transitory computer readable medium, comprising:

program code for obtaining a first image data from a color imaging device;

program code for obtaining a second image data from an infrared imaging device;

program code for separating chroma data and luma data associated with the first image data;

program code for processing luma data associated with the second image data to produce edge data;

program code for combining the luma data associated with the first image data, the luma data associated with the second image data, and edge data to produce a combined luma data; and

program code for fusing the combined luma data with the chroma data to obtain a true color image.