Imaging apparatus and image signal processing device

• US 20080111894A1
• Filed: 11/06/2007
• Published: 05/15/2008
• Est. Priority Date: 11/10/2006
• Status: Active Grant

• Current Assignee: Semiconductor Components Industries LLC (ON Semiconductor Corporation)
• Original Assignee: Sanyo Semiconductor Company Limited (ON Semiconductor Corporation), Sanyo Electric Company Limited (Panasonic Holdings Corporation)
• Inventors: Tanimoto, Takashi
• Granted Patent: US 7,956,325 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/131 including elements passing infrared wavelengths
• CPC Class Codes: H04N 25/135 based on four or more different wavelength filter elements
• CPC Class Codes: H04N 25/20 for transforming only infrared radiation into image signals
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Infrared light emitted by a fluorescent material is weak. When an infrared image obtained by this fluorescent material is superposed and displayed on a visible-light image, the infrared image is made clear. A signal <IR> of an IR pixel having selective sensitivity to infrared light is subtracted from RGB pixel signals <R>, <B>, and <G> of a solid-state image sensor by adders (20), and signals R₀, G₀, and B₀ that correspond to the R, G, and B components of incident light within <R>, <G>, and <B> are separated and extracted. <IR> is multiplied by each of the gains κ_R, κ_G, and κ_B to enhance the IR component by multipliers (22), and each is synthesized with R₀, G₀, and B₀ by adders (24). κ_R, κ_G, and κ_B are set so that the infrared image displayed on the visible-light image by the synthesized IR component is enhanced and displayed in a hue that can be distinguished from the color white or another visible-light image.

Claims

1. An imaging apparatus comprising:

a solid-state image sensor having, in a two dimensional array, infrared light-receiving pixels having selective sensitivity to infrared light, and a plurality of types of color light-receiving pixels having sensitivity to the infrared light and to visible light of mutually different specific colors; and

an image signal processing circuit for generating a processed image signal for displaying an infrared composite image in which an enhanced infrared image is synthesized with and displayed on a visible image on the basis of a raw image signal that is output by the solid-state image sensor and is composed of an infrared pixel signal that corresponds to the infrared light-receiving pixels and a plurality of types of color pixel signals that corresponds to each of the color light-receiving pixels;

wherein

the image signal processing circuit performs;

a component image signal generation process for generating an infrared component image signal that corresponds to an infrared light component, and generating a plurality of types of color component image signals that corresponds to visible-light components of the specific colors on the basis of the infrared pixel signal and the color pixel signals;

an infrared composite process for synthesizing, for each of the specific colors, the color component image signal and a signal obtained by multiplying the infrared component image signal by a gain of the specific color in accordance with the manner in which the infrared image is enhanced, and generating a converted color component image signal of each of the specific colors; and

a processed image signal generation process for generating the processed image signal on the basis of the plurality of converted color component image signals.

2. An imaging apparatus comprising:

a solid-state image sensor having, in a two dimensional array, infrared light-receiving pixels having selective sensitivity to infrared light, and a plurality of types of color light-receiving pixels having sensitivity to the infrared light and to visible light of mutually different specific colors; and

an image signal processing circuit for generating a processed image signal for displaying an infrared composite image in which an infrared image is synthesized with and displayed on a visible image on the basis of a raw image signal that is output by the solid-state image sensor and is composed of an infrared pixel signal that corresponds to the infrared light-receiving pixels and a plurality of types of color pixel signals that corresponds to each of the color light-receiving pixels;

wherein

the image signal processing circuit performs;

a component image signal generation process for generating an infrared component image signal that corresponds to an infrared light component, and generating a plurality of types of color component image signals that corresponds to visible-light components of the specific colors on the basis of the infrared pixel signal and the color pixel signals;

an intermediate image generation process for generating the intermediate image signal that corresponds to the visible-light image on the basis of the plurality of color component image signals; and

a processed image signal generation process for generating the processed image signal by varying luminance or hue of each pixel that expresses the intermediate image signal in accordance with signal level of the infrared component image signal.

3. An image signal processing device for generating a processed image signal for displaying an infrared composite image in which an enhanced infrared image is synthesized with and displayed on a visible image on the basis of a raw image signal that is output by an image sensor having, in a two dimensional array, infrared light-receiving pixels having selective sensitivity to infrared light, and a plurality of types of color light-receiving pixels having sensitivity to the infrared light and to visible light of mutually different specific colors, and that is composed of an infrared pixel signal that corresponds to the infrared light-receiving pixels and a plurality of types of color pixel signals that corresponds to each of the color light-receiving pixels;

the device performs;

a component image signal generation process for generating an infrared component image signal that corresponds to an infrared light component, and generating a plurality of types of color component image signals that corresponds to visible-light components of the specific colors on the basis of the infrared pixel signal and the color pixel signals;

an infrared composite process for synthesizing, for each of the specific colors, the color component image signal and a signal obtained by multiplying the infrared component image signal by a gain of the specific color in accordance with the manner in which the infrared image is enhanced, and generating a converted color component image signal of each of the specific colors; and

a processed image signal generation process for generating the processed image signal on the basis of the plurality of converted color component image signals.

4. The image signal processing device according to claim 3, wherein the infrared image is enhanced by varying luminance of the infrared composite image in accordance with the infrared component image signal.

5. The image signal processing device according to claim 3, wherein the infrared image is enhanced by varying hue of each pixel on the infrared composite image in accordance with the signal level of the infrared component image signal.

6. The image signal processing device according to claim 5, wherein, the hue is set to a predetermined enhanced hue when the signal level of the infrared component image signal is at a predetermined threshold or above.

7. The image signal processing device according to claim 6, wherein saturation of the enhanced hue is increased in conjunction with the signal level of the infrared component image signal at or above the threshold.

8. The image signal processing device according to claim 3, comprising performing the steps of:

a visible image suppression process for stopping the input of the color component image signals into the infrared composite process; and

a composition control process for controlling whether the visible image suppression process is performed or not and allowing switching to be performed between generation of the processed image signal for displaying the infrared composite image, and generation of the processed image signal for displaying an image composed of the infrared image alone.

9. An image signal processing device for generating a processed image signal for displaying an infrared composite image in which an infrared image is synthesized with and displayed on a visible image on the basis of a raw image signal that is output by an image sensor having, in a two dimensional array, infrared light-receiving pixels having selective sensitivity to infrared light, and a plurality of types of color light-receiving pixels having sensitivity to the infrared light and to visible light of mutually different specific colors, and that is composed of an infrared pixel signal that corresponds to the infrared light-receiving pixels and a plurality of types of color pixel signals that corresponds to each of the color light-receiving pixels;

the device performs;

a component image signal generation process for generating an infrared component image signal that corresponds to an infrared light component, and generating a plurality of types of color component image signals that corresponds to visible-light components of the specific colors on the basis of the infrared pixel signal and the color pixel signals;

an intermediate image generation process for generating the intermediate image signal that corresponds to the visible-light image on the basis of the plurality of color component image signals; and

a processed image signal generation process for generating the processed image signal by varying luminance or hue of the pixels for displaying the intermediate image signal, in accordance with signal level of the infrared component image signal.

10. The image signal processing device according to claim 9, wherein the processed image signal generation process changes the hue expressed by the intermediate image signal to an achromatic color for pixels in which the signal level of the infrared component image signal is less than a predetermined threshold.