VIDEO IMAGE PROCESSING AND FUSION

• US 20090196457A1
• Filed: 01/31/2008
• Published: 08/06/2009
• Est. Priority Date: 01/31/2008
• Status: Active Grant

• Current Assignee: Peco Limited (Astronics Corporation)
• Original Assignee: Max-Viz, Inc. (Astronics Corporation)
• Inventors: Zuro, Gregory
• Granted Patent: US 8,144,927 B2
• Time in Patent Office: Days
• US Class Current: 382/100
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Processing of image data derived from radiation emanating from a scene and acquired by a multi-channel enhanced vision system renders an image of the scene for display. Detected first and second wavelength bands of radiation produce respective first and second sets of image data that include representations of relatively low contrast, high spatial frequency detail of features of the scene. Nonlinear intensity transformation of data derived from the first and second sets of image data produces, respectively, first and second sets of low dynamic range image data representing, respectively, first and second sets of intensity values. Different pairs of associated intensity values of the first and second sets correspond to different pixels forming an image. The associated intensity values of the different pairs are combined to form fused image data representing brightness levels of the pixels forming a displayed image that exhibits with high brightness and in great detail the features of the scene.

Claims

1. A method of processing image data derived from radiation emanating from a scene and acquired by a multi-channel enhanced vision system to render an image of the scene for display, the image exhibiting features of spatial regions of the scene with high brightness and in great detail, comprising:

detecting first and second wavelength bands of radiation emanating from a scene to produce respective first and second sets of image data that include representations of relatively low contrast, high spatial frequency detail of features of spatial regions of the scene;

performing nonlinear intensity transformation of data derived from the first set and second set of image data to produce, respectively, a first set of low dynamic range image data representing a first set of intensity values and a second set of low dynamic range image data representing a second set of intensity values, different pairs of associated intensity values of the first and second sets corresponding to different pixels forming an image of the scene; and

combining the associated intensity values of the different pairs to form fused image data representing brightness levels of the pixels forming an image that, when rendered on a display, exhibits with high brightness and in great detail the features of the spatial regions of the scene.

2. The method of claim 1, in which the combining to form fused image data representing brightness levels of the pixels comprises performing weighted addition of the intensity values of each of the different pairs of associated intensity values.

3. The method of claim 1, further comprising performing distortion correction to the first set of intensity values, in which the distortion correction to an intensity value of the first set corresponding to a first pixel comprises:

selecting a first set of predetermined offset values that identify the first set of intensity values corresponding to a first group of adjacent pixels;

selecting a first set of predetermined weight values for the adjacent pixels in the first group;

determining a first weighted average intensity value based upon the first set of weight values and the first set of intensity values corresponding to the first group of adjacent pixels; and

assigning the first weighted average intensity value as the intensity value of the first set corresponding to the first pixel.

4. The method of claim 3, in which the first set of image data represents the intensity values of the first set and the distortion correction is performed before the performance of nonlinear intensity transformation.

5. The method of claim 3, in which the first set of predetermined offset values and the first set of predetermined weight values are stored in and selected from a look-up table.

6. The method of claim 3, further comprising performing the distortion correction to the second set of intensity values, in which the distortion correction to an intensity value of the second set corresponding to a second pixel comprises:

selecting a second set of predetermined offset values that identify the second set of intensity values corresponding to a second group of adjacent pixels;

selecting a second set of predetermined weight values for the adjacent pixels in the second group;

determining a second weighted average intensity value based upon the second set of weight values and the second set of intensity values corresponding to the second group of adjacent pixels; and

assigning the second weighted average intensity value as the intensity value of the second set corresponding to the second pixel.

7. The method of claim 1, in which the first and second wavelength bands are at least partly non-overlapping.

8. The method of claim 7, in which a wavelength range of the first wavelength band is between about 0.7 μ

m and about 3 μ

m and a wavelength range of the second wavelength band is between about 3 μ

m and about 15 μ

m.

9. The method of claim 8, further comprising adjusting intensity values of the first set representing negative-going excursions to reduce perception of visible artifacts of the image of the scene.

10. The method of claim 1, further comprising:

detecting a third wavelength band of radiation emanating from the scene to produce a third set of image data that include representations of relatively low contrast, high spatial frequency detail of features of spatial regions of the scene;

performing nonlinear intensity transformation of data derived from the third set of image data to produce a third set of low dynamic range image data representing a third set of intensity values, different groups of associated intensity values of the first, second, and third sets corresponding to different pixels forming the image of the scene; and

combining the associated intensity values of the different groups to form the fused image data.

11. The method of claim 10, in which the first, second, and third wavelength bands are at least partly non-overlapping.

12. The method of claim 11, in which a wavelength range of the first wavelength band is between about 0.7 μ

m and about 3 μ

m, a wavelength range of the second wavelength band is between about 3 μ

m and about 15 μ

m, and a wavelength range of the third wavelength band is between about 0.4 μ

m and about 0.7 μ

m.

13. The method of claim 12, further comprising:

determining an average value of the intensity values of the first set; and

adjusting the intensity values of the third set according to the determined average value.

14. The method of claim 13, in which the intensity values of the third set are decreased corresponding to an increase of the average value of the first set.

15. The method of claim 12, further comprising adjusting intensity values of the first and third sets representing negative-going excursions to reduce perception of visible artifacts of the image of the scene.

16. The method of claim 1, further comprising:

identifying a subset of the image data of the first set;

determining a peak intensity level of the first wavelength band of radiation represented in the subset;

producing a control signal representing the peak intensity level; and

regulating an amount of radiation detected in the first wavelength band in response to the control signal to reduce saturation caused by image features of the scene represented in the first wavelength band of radiation.

17. The method of claim 16, in which a wavelength range of the first wavelength band is between about 0.7 μ

m and about 3 μ

m.