MULTOCULAR IMAGE PICKUP APPARATUS AND MULTOCULAR IMAGE PICKUP METHOD

• US 20120262607A1
• Filed: 12/22/2010
• Published: 10/18/2012
• Est. Priority Date: 12/24/2009
• Status: Active Grant

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Shimura, Tomoya, Tanaka, Seiichi
• Granted Patent: US 8,885,067 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 2013/0088 Synthesising a monoscopic image signal from stereoscopic images, e.g. synthesising a panoramic or high resolution monoscopic image
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 5/265 Mixing

Abstract

A multocular image pickup apparatus includes: a distance calculation unit that calculates information regarding the distance to a captured object from an output video of a first image pickup unit, which is to be the reference unit of a plurality of image pickup units that pick up images, and from an output video of an image pickup unit that is different from the first image pickup unit; a multocular video synthesizing unit that generates synthesized video from the output video of the plurality of image pickup units based on the distance information for regions where the distance information could be calculated; and a monocular video synthesizing unit that generates synthesized video from the output video of the first image pickup unit for regions where the distance information could not be calculated. The distance calculation unit calculates a first distance information that is the distance to the captured object from the output video of the first image pickup unit and the output video of a second image pickup unit that is different from the first image pickup unit, and in case that there is a region where the first distance information could not be calculated, for the region where the first distance information could not be calculated, the distance calculation unit recalculates information regarding the distance to the captured object from the output video of an image pickup unit that was not used for calculating the distance among the plurality of image pickup units, and from the output video of the first image pickup unit.

Claims

1. A multocular image pickup apparatus comprising:

a plurality of image pickup units that pick up images;

a distance calculation unit that calculates information regarding the distance to a captured object from an output video of a first image pickup unit, which is to be the reference unit of the image pickup units, and from an output video of at least one image pickup unit among the plurality of image pickup units that is different from the first image pickup unit;

a multocular video synthesizing unit that generates synthesized video from the output video of the plurality of image pickup units based on the distance information for regions where the distance information could be calculated in the distance calculation unit; and

a monocular video synthesizing unit that generates synthesized video from the output video of the first image pickup unit for regions where the distance information could not be calculated in the distance calculation unit;

wherein the distance calculation unit calculates a first distance information that is the distance to the captured object from the output video of the first image pickup unit and the output video of a second image pickup unit that is different from the first image pickup unit, and in case that there is a region where the first distance information could not be calculated, for the region where the first distance information could not be calculated, the distance calculation unit recalculates, at least one time, information regarding the distance to the captured object from the output video of an image pickup unit that was not used for calculating the distance among the plurality of image pickup units, and from the output video of the first image pickup unit.

2. The multocular image pickup apparatus according to claim 1, wherein the monocular video synthesizing unit reduces, by a prescribed ratio, the luminance of the regions where the distance information could not be calculated in the distance calculation unit so as to generate the synthesized image.

3. The multocular image pickup apparatus according to claim 1, wherein the monocular video synthesizing unit lowers a luminance value of the regions, where the distance information could not be calculated in the distance calculation unit, to lower than that of foreground regions adjacent thereto, so as to generate the synthesized image.

4. The multocular image pickup apparatus according to claim 1, wherein the monocular video synthesizing unit causes the luminance value of the regions where the distance information could not be calculated in the distance calculation unit are made to coincide with that of a background region adjacent thereto, so as to generate the synthesized image.

5. The multocular image pickup apparatus according to claim 1, wherein the monocular video synthesizing unit reduces, by the prescribed ratio, the color saturation of the regions where the distance information could not be calculated in the distance calculation unit, so as to generate the synthesized image.

6. The multocular image pickup apparatus according to claim 1, wherein the monocular video synthesizing unit lowers color saturation of the regions where the distance information could not be calculated in the distance calculation unit to lower than that of foreground regions adjacent thereto, so as to generate the synthesized image.

7. The multocular image pickup apparatus according to claim 1, wherein the monocular video synthesizing unit causes the color saturation of the regions where the distance information could not be calculated in the distance calculation unit to conform with that of the background region adjacent thereto, so as to generate the synthesized image.

8. The multocular image pickup apparatus according to claim 1, wherein the monocular video synthesizing unit suppresses a change of luminance of the regions where the distance information could not be calculated in the distance calculation unit so as to generate the synthesized image.

9. A multocular image pickup method comprising:

calculating information regarding the distance to a captured object from an output video of a first image pickup unit, which is to be the reference unit of a plurality of image pickup units that pick up images, and from an output video of at least one image pickup unit among the plurality of image pickup units that is different from the first image pickup unit;

generating a synthesized video from the output video of the plurality of image pickup units based on the distance information for regions where the distance information could be calculated; and

generating a synthesized video from the output video of the first image pickup unit for regions where the distance information could not be calculated;

wherein

in case of calculating the distance information, calculating a first distance-information that is the distance to the captured object from the output video of the first image pickup unit and the output video of a second image pickup unit that is different from the first image pickup unit; and

in case that there is a region where the first distance information could not be calculated, for the region where the first distance information could not be calculated, recalculating, at least one time, information regarding the distance to the captured object form the output video of an image pickup unit that was not used for calculating the distance among the plurality of image pickup units, and the output video of the first image pickup unit.