IMAGE PROCESSING DEVICE, SOLID-STATE IMAGING DEVICE, AND CAMERA MODULE

• US 20110043665A1
• Filed: 06/18/2010
• Published: 02/24/2011
• Est. Priority Date: 08/19/2009
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: OGASAHARA, Takayuki
• Granted Patent: US 8,339,483 B2
• Time in Patent Office: Days
• US Class Current: 348/241
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/611 Correction of chromatic aberration

Abstract

According to one embodiment, an image processing device includes a shading correcting unit, a distortion correcting unit, a lens-characteristic estimating unit, and a resolution restoring unit. The shading correcting unit, the distortion correcting unit, the lens-characteristic estimating unit, the resolution restoring unit carry out signal processing for each of image data obtained by a plurality of sub-camera modules.

Claims

1. An image processing device comprising:

a shading correcting unit that carries out shading correction for subject images picked up by imaging elements;

a distortion correcting unit that corrects distortion of the subject images;

a lens-characteristic estimating unit that estimates lens characteristics of imaging lenses that capture light made incident on the imaging elements;

a resolution restoring unit that carries out, based on the estimated lens characteristics, resolution restoration processing for the subject images;

a block matching unit that carries out block matching processing for alignment of the subject images; and

a demosaicing unit that generates a color image by performing demosaicing processing for image data obtained by the block matching processing, wherein

the shading correcting unit, the distortion correcting unit, the lens-characteristic estimating unit, and the resolution restoring unit carry out, on a plurality of sub-camera modules including the imaging elements and the imaging lenses, signal processing for each of image data obtained by the sub-camera modules.

2. The image processing device according to claim 1, wherein

one of the sub-camera modules is set as a reference sub-camera module, and

the shading correcting unit, the distortion correcting unit, the lens-characteristic estimating unit, and the resolution restoring unit carry out the signal processing on the image data obtained by shifting focusing positions of the subject images by the sub-camera modules other than the reference sub-camera module with respect to a focusing position of the subject image by the reference sub-camera module.

3. The image processing device according to claim 2, wherein the reference sub-camera module is a sub-camera module for green that picks up a green component of the subject image.

4. The image processing device according to claim 1, wherein

one of the sub-camera modules is set as a reference sub-camera module, and

the resolution restoring unit carries out the resolution restoration processing based on the lens characteristics added with position information for shifting the subject images picked up by the sub-camera modules other than the reference sub-camera module with respect to the subject image picked up by the reference sub-camera module.

5. The image processing device according to claim 1, wherein

one of the sub-camera modules is set as a reference sub-camera module, and

the image processing device further comprises a sampling unit that carries out, with the subject image picked up by the reference sub-camera module set as a reference, sampling of a signal value concerning an image, parallax of which due to the sub-camera modules other than the reference sub-camera module is corrected.

6. The image processing device according to claim 5, further comprising a weighting processing unit that calculates a parallax amount of the subject images obtained by the sub-camera modules and carries out weighting processing corresponding to the parallax amount, wherein

the sampling unit carries out the sampling based on an image generated by the weighting processing unit.

7. The image processing device according to claim 5, further comprising a coordinate conversion processing unit that carries out coordinate conversion processing for reducing a parallax amount of the subject images obtained by the sub-camera modules, wherein

the sampling unit carries out the sampling based on an image generated by the coordinate conversion processing unit.

8. The image processing device according to claim 1, wherein the block matching unit carries out the block matching processing for obtaining a predetermined total number of pixels.

9. The image processing device according to claim 1, wherein the demosaicing unit carries out up-sampling for obtaining a predetermined total number of pixels.

10. A solid-state imaging device comprising a plurality of imaging elements that pick up subject images, wherein

one of a plurality of sub-camera modules including the imaging elements and imaging lenses, which capture light made incident on the imaging elements, is set as a reference sub-camera module, and

the imaging elements of the sub-camera modules other than the reference sub-camera module are arranged with focusing positions of the subject images shifted with respect to the imaging element of the reference sub-camera module.

11. The solid-state imaging device according to claim 10, wherein the imaging element of the reference sub-camera module is arranged in a center of the imaging elements.

12. The solid-state imaging device according to claim 10, further comprising:

a shading correcting unit that carries out shading correction for the subject images;

a distortion correcting unit that corrects distortion of the subject images;

a lens-characteristic estimating unit that estimates lens characteristics of the imaging lenses that capture light made incident on the imaging elements; and

a resolution restoring unit that carries out, based on the estimated lens characteristics, resolution restoration processing for the subject images, wherein

the shading correcting unit, the distortion correcting unit, the lens-characteristic estimating unit, and the resolution restoring unit carry out signal processing for each of image data obtained by the sub-camera modules.

13. The solid-state imaging device according to claim 12, further comprising:

a block matching unit that carries out, on image data subjected to the processing by the shading correcting unit, the distortion correcting unit, the lens-characteristic estimating unit, and the resolution restoring unit, block matching processing for alignment of the subject images; and

a demosaicing unit that generates a color image by performing demosaicing processing for an image obtained by the block matching processing.

14. A camera module comprising:

a plurality of imaging elements that pick up subject images;

a plurality of imaging lenses that capture light made incident on the imaging elements;

a housing that houses the imaging elements and the imaging lenses;

a shading correcting unit that carries out shading correction for the subject images;

a distortion correcting unit that corrects distortion of the subject images;

a lens-characteristic estimating unit that estimates lens characteristics of the imaging lenses;

a resolution restoring unit that carries out, based on the estimated lens characteristics, resolution restoration processing for the subject images;

a block matching unit that carries out block matching processing for alignment of the subject images; and

a demosaicing unit that generates a color image by performing demosaicing processing for an image obtained by the block matching processing, wherein

the shading correcting unit, the distortion correcting unit, the lens-characteristic estimating unit, and the resolution restoring unit carry out, on a plurality of sub-camera modules including the imaging elements and the imaging lenses, signal processing for each of image data obtained by the sub-camera modules.

15. The camera module according to claim 14, wherein

one of the sub-camera modules is set as a reference sub-camera module, and

the imaging elements of the sub-camera modules other than the reference sub-camera module are arranged with focusing positions of the subject images shifted with respect to the imaging element of the reference sub-camera module.

16. The camera module according to claim 14, wherein

one of the sub-camera modules is set as a reference sub-camera module, and

the resolution restoring unit carries out the resolution restoration processing based on the lens characteristics added with position information for shifting the subject images picked up by the sub-camera modules other than the reference sub-camera module with respect to the subject image picked up by the reference sub-camera module.

17. The camera module according to claim 14, wherein

one of the sub-camera modules is set as a reference sub-camera module, and

the camera module further comprises a sampling unit that carries out, with the subject image picked up by the reference sub-camera module set as a reference, sampling of a signal value concerning an image, parallax of which due to the sub-camera modules other than the reference sub-camera module is corrected.

18. The camera module according to claim 17, further comprising a weighting processing unit that calculates a parallax amount of the subject images obtained by the sub-camera modules and carries out weighting processing corresponding to the parallax amount, wherein

the sampling unit carries out the sampling based on an image generated by the weighting processing unit.

19. The camera module according to claim 17, further comprising a coordinate conversion processing unit that carries out coordinate conversion processing for reducing a parallax amount of the subject images obtained by the sub-camera modules, wherein

the sampling unit carries out the sampling based on an image generated by the coordinate conversion processing unit.

20. The camera module according to claim 14, further comprising a parameter storing unit that stores individual information of the camera module as parameters, wherein

at least one of the shading correcting unit, the distortion correcting unit, and the lens-characteristic estimating unit refers to the parameters stored in the parameter storing unit.