Image signal processing apparatus, imaging apparatus, image signal processing method and computer program

• US 20070153335A1
• Filed: 12/18/2006
• Published: 07/05/2007
• Est. Priority Date: 12/22/2005
• Status: Active Grant

• Current Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Hosaka, Hajime
• Granted Patent: US 8,467,088 B2
• Time in Patent Office: Days
• US Class Current: 358/463
• CPC Class Codes: H04N 1/58 Edge or detail enhancement; Noise or error suppression, e.g. colour misregistration correction H04N1/62 takes precedence
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/131 including elements passing infrared wavelengths
• CPC Class Codes: H04N 25/135 based on four or more different wavelength filter elements
• CPC Class Codes: H04N 25/68 applied to defects

Abstract

There is provided an image signal processing apparatus, comprising a demosaic processing unit receiving input of mosaic image data of each of signals obtained by a single plate imaging device having an element array composed of visible light obtaining elements obtaining visible light signals, and invisible light obtaining elements obtaining signals including invisible light components, and generating a demosaic image of each of the obtained signals; and a noise reduction processing unit receiving input of the demosaic image to execute correction of pixel values of the demosaic image obtained by the visible light obtaining elements on the basis of edge information extracted from the demosaic image of the signals obtained by the invisible light obtaining elements.

Claims

1. An image signal processing apparatus, comprising:

a demosaic processing unit receiving input of mosaic image data of each of signals obtained by a single plate imaging device having an element array composed of visible light obtaining elements obtaining visible light signals, and invisible light obtaining elements obtaining signals including invisible light components, and generating a demosaic image of each of the obtained signals; and

a noise reduction processing unit receiving input of the demosaic image to execute correction of pixel values of the demosaic image obtained by the visible light obtaining elements on the basis of edge information extracted from the demosaic image of the signals obtained by the invisible light obtaining elements.

2. The image signal processing apparatus according to claim 1, wherein:

the visible light obtaining elements are RGB elements obtaining R, G and B color signals severally,

the invisible light obtaining elements are A elements obtaining A signals including R, G and B color signal components and infrared components,

the demosaic processing unit is configured to receive input of mosaic image data of each of the R, G, B and A signals obtained by the single plate imaging device having a color array composed of the RGB elements obtaining the R, G and B color signals severally, and the A elements obtaining the A signal including the R, G and B color signal components and the infrared components, and to generate the demosaic image of each of the R, G, B and A signals, and

the noise reduction processing unit is configured to receive input of the demosaic image of each of the R, G, B and A signals, and to execute the correction of the pixel values of the RGB demosaic image on the basis of the edge information extracted from the demosaic image of the A signal.

3. The image signal processing apparatus according to claim 2, wherein:

the noise reduction processing unit is configured to execute processing of correcting the pixel values of the pixels constituting the RGB demosaic image on the basis of pixel values of surrounding pixels according to edge distribution near the pixels.

4. The image signal processing apparatus according to claim 2, wherein:

the demosaic processing unit is configured to generate an edge evaluating image as the demosaic image of the A signal; and

the noise reduction processing unit is configured to execute the demosaic processing of the mosaic image of each of the R, G and B signals by collectively executing noise reduction and demosaic processing by executing the demosaic processing in consideration of edge positions on the basis of the edge evaluating image as the demosaic image of the A signal.

5. The image signal processing apparatus according to claim 4, wherein:

the noise reduction processing unit is configured to execute processing of calculating an output pixel value of each pixel position in the RGB mosaic image on the basis of the pixel values of the surrounding pixels by applying weight calculated by a product of space-dependent weight dependent upon a distance between a pixel of a correction object and a reference pixel and edge-dependent weight dependent upon strength of an edge of an image calculated in conformity to the edge evaluating image as the demosaic image of the A signal at a time of the demosaic processing of the mosaic images of each of the R, G and B signals.

6. The image signal processing apparatus according to claim 2, further comprising:

an A-noise reduction processing unit generating a noise-reduced A-demosaic image corresponding to the A signal on the basis of the edge information extracted from the demosaic image of the A signal; and

a matrix operation executing unit receiving input of the noise-reduced RGB-demosaic image generated by the noise reduction processing unit and the noise-reduced A-demosaic image, and converting pixel values of corresponding pixels of input four demosaic images of R, G, B and A by a matrix operation to generate the RGB image in which the infrared components are removed form the R, G and B signals.

7. The image signal processing apparatus according to claim 6, wherein:

the matrix operation executing unit is configured to generate and output an estimated infrared image composed of infrared components on the basis of the noise-reduced RGB-demosaic image generated by the noise reduction processing unit and the noise-reduced A-demosaic image.

8. An imaging apparatus, comprising:

an imaging device having a color array composed of RGB elements obtaining R, G and B color signals severally and A elements obtaining A signals including R, G and B color signal components and infrared components;

a demosaic processing unit receiving input of mosaic image data of each of the R, G, B and A signals obtained by the single plate imaging device to generate a demosaic image of the R, G, B and A signals; and

a noise reduction processing unit receiving the input of the demosaic image of the R, G, B and A signals to correct pixel values of the RGB-demosaic image on the basis of edge information extracted from the demosaic image of the A signal.

9. The imaging apparatus according to claim 8, wherein:

the imaging device is configured to have a checkered-pattern arrangement of the A elements obtaining the A signals.

10. An image signal processing method, comprising:

a demosaic processing step of receiving input of mosaic image data of each of signals obtained by a single plate imaging device having an element array composed of visible light obtaining elements obtaining visible light signals and invisible light obtaining elements obtaining signals including invisible light components to generate a demosaic image of each of the obtained signals; and

a noise reduction processing step of receiving input of the demosaic image to execute a correction of pixel values of the demosaic image obtained by the visible light obtaining elements on the basis of edge information extracted from the demosaic image of the signals obtained by the invisible light obtaining elements.

11. The image signal processing method according to claim 10, wherein:

the visible light obtaining elements are RGB elements obtaining R, G and B color signals severally,

the invisible light obtaining elements are A elements obtaining A signals including R, G and B color signal components and infrared components,

the first demosaic processing step is a step of receiving the input of the mosaic image data of each of the R, G, B and A signals obtained by the single plate imaging device having a color array composed of the RGB elements obtaining the R, G and B color signals severally, and the A elements obtaining the A signals including the R, G and B color signal components and the infrared components, and generating the demosaic image of each of the R, G, B and A signals, and

the noise reduction processing step is a step of receiving the input of the demosaic image of each of the R, G, B and A signals, and executing the correction of the pixel values of the RGB demosaic image on the basis of the edge information extracted from the demosaic image of the A signals.

12. The image signal processing method according to claim 11, wherein:

the step of receiving the input of the demosaic image is a step of executing processing of correcting the pixel values of the pixels constituting the RGB demosaic image on the basis of pixel values of surrounding pixels according to edge distribution near the pixels on the basis of the edge information detected from the demosaic image of the A signals.

13. The image signal processing method according to claim 11, wherein:

the demosaic processing step is a step of generating an edge evaluating image as the demosaic image of the A signals; and

the noise reduction processing step is a step of executing the demosaic processing of the mosaic image of each of the R, G and B signals by collectively executing noise reduction and demosaic processing by executing the demosaic processing in consideration of edge positions on the basis of the edge evaluating image as the demosaic image of the A signal.

14. The image signal processing method according to claim 11, wherein:

the noise reduction processing step is a step of executing processing of calculating an output pixel value of each pixel position in the RGB demosaic image on the basis of the pixel values of the surrounding pixels by applying weight calculated by a product of space-dependent weight dependent upon a distance between a pixel of a correction object and a reference pixel and edge-dependent weight dependent upon strength of an edge of an image calculated in conformity to the edge evaluating image as the demosaic image of the A signals at a time of the demosaic processing of the mosaic images of each of the R, G and B signals.

15. The image signal processing method according to claim 11, further comprising:

an A-noise reduction processing step of generating a noise-reduced A-demosaic image corresponding to the A signals on the basis of the edge information extracted from the demosaic image of the A signals; and

a matrix operation executing step of receiving input of the noise-reduced RGB demosaic image generated at the noise reduction processing step and the noise-reduced A-demosaic image, and converting pixel values of corresponding pixels of input four demosaic images of R, G, B and A by a matrix operation to generate the RGB image in which the infrared components are removed from the R, G and B signals.

16. The image signal processing method according to claim 15, wherein:

the matrix operation executing step is a step of generating and outputting an estimated infrared image composed of the infrared components on the basis of the noise-reduced RGB-demosaic image generated at the noise reduction processing step and the noise-reduced A-demosaic image.

17. A computer program for making an imaging apparatus execute image signal processing, the program making the apparatus execute:

a demosaic processing step of receiving input of mosaic image data of each of signals obtained by a single plate imaging device having an element array composed of visible light obtaining elements obtaining visible light signals and invisible light obtaining elements obtaining signals including invisible light components to generate a demosaic image of each of the obtained signals; and

a noise reduction processing step of receiving input of the demosaic image to execute a correction of pixel values of the demosaic image obtained by the visible light obtaining elements based on edge information extracted from the demosaic image obtained by the invisible light obtaining elements.