Solid state imaging device, calculating device, and calculating program

• US 9,479,760 B2
• Filed: 08/05/2014
• Issued: 10/25/2016
• Est. Priority Date: 09/18/2013
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Suzuki, Kazuhiro, Ueno, Risako, Kobayashi, Mitsuyoshi, Kwon, Honam, Funaki, Hideyuki
• Primary Examiner(s): Bailey, Frederick
• Assistant Examiner(s): Nawaz, Talha M
• Application Number:
  US14/451,835
• Publication Number:
  US 20150077522A1
• Time in Patent Office: 812 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G02B 27/0075 with means for altering, e.g. increasing, the depth of field or depth of focus
• CPC Class Codes: G02B 3/0037 characterized by the distribution or form of lenses
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/557 from light fields, e.g. from plenoptic cameras
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

According to one embodiment, a solid state imaging device includes a first imaging device, a second imaging device, and a calculating unit. The first imaging device includes a first optical system, a first imaging unit, and a second optical system provided between the first optical system and the first imaging unit. The second imaging device includes a third optical system, a second imaging unit, and a fourth optical system provided between the third optical system and the second imaging unit. The calculating unit is configured to perform a first calculation and a second calculation. The first calculation includes deriving a first distance from stereo disparity. The second calculation includes deriving a second distance from a parallax image. The calculating unit is configured to estimate a target distance based on at least one selected from the first distance and the second distance.

Claims

1. A solid state imaging device, comprising:

a first imaging device including

a first optical system,

a first imaging unit provided between the first optical system and an imaging position of the first optical system, the first imaging unit including a plurality of imaging pixels, and

a second optical system provided between the first optical system and the first imaging unit, the second optical system including a first microlens array;

a second imaging device provided to be separated from the first imaging device, the second imaging device including

a third optical system,

a second imaging unit provided between the third optical system and an imaging position of the third optical system, the second imaging unit including a plurality of imaging pixels, and

a fourth optical system provided between the third optical system and the second imaging unit, the fourth optical system including a second microlens array; and

a calculating unit configured to perform a first calculation and a second calculation, the first calculation including deriving a first distance from stereo disparity using a first image signal and a second image signal, the first image signal being of an object as obtained by the first imaging device, the second image signal being of the object as obtained by the second imaging device, the second calculation including deriving a second distance from a parallax image obtained via at least one selected from the first microlens array and the second microlens array, the calculating unit being configured to estimate a target distance based on at least one selected from the first distance and the second distance, wherein

the calculating unit is configured to adopt the first distance as the target distance in response to the first distance exceeding a threshold, and adopt the second distance as the target distance in response to the first distance not being more than the threshold.

2. The device according to claim 1, wherein

the calculating unit performs the first calculation for an overlapping region where a first imaging region of the first imaging device and a second imaging region of the second imaging device overlap, and

the calculating unit performs the second calculation for one other region of the first imaging region and second imaging region other than the overlapping region.

3. The device according to claim 1, wherein

the calculating unit performs the first calculation and the second calculation for an overlapping region where a first imaging region of the first imaging device and a second imaging region of the second imaging device overlap, and

the calculating unit performs the second calculation for one other region of the first imaging region and second imaging region other than the overlapping region.

4. The device according to claim 1, wherein the calculating unit is configured to select one selected from the first distance and the second distance according to an instruction transmitted from an outside.

5. The device according to claim 1, wherein the calculating unit is configured to implement the second calculation and the first calculation in parallel.

6. The device according to claim 1, wherein an optical axis of the first optical system is non-parallel to an optical axis of the third optical system.

7. The device according to claim 1, further comprising a third imaging device disposed between the first imaging device and the second imaging device, the third imaging device including:

a fifth optical system;

a third imaging unit provided between the fifth optical system and an imaging position of the fifth optical system, the third imaging unit including a plurality of imaging pixels; and

a sixth optical system provided between the fifth optical system and the third imaging unit, the sixth optical system including a third microlens array,

the calculating unit being configured to further perform a third calculation including deriving a third distance from a parallax image obtained via the third microlens array, and configured to estimate the target distance based on the third distance.

8. The device according to claim 1, wherein a focal distance of the first optical system and a focal distance of the second optical system are different.

9. The device according to claim 1, further comprising a substrate configured to have the first imaging device, the second imaging device, and the calculating unit mounted on the substrate.

10. The device according to claim 1, wherein

the first calculation is performed in a first interval, and

the second calculation is performed in a second interval different from the first interval.

11. A calculating device comprising:

calculating circuitry configured to perform a first calculation and a second calculation,

the first calculation including deriving a first distance from stereo disparity using a first image signal and a second image signal, the first image signal being of an object obtained by a first imaging device, the second image signal being of the object obtained by a second imaging device, and

the second calculation including deriving a second distance from a parallax image obtained via at least one selected from a first microlens array and a second microlens array,

the calculating circuitry being configured to adopt the first distance as a target distance in response to the first distance exceeding a threshold, and adopt the second distance as the target distance in response to the first distance not being more than the threshold,

the first imaging device including;

a first optical system;

a first imaging unit, implemented by circuitry, provided between the first optical system and an imaging position of the first optical system, the first imaging unit including a plurality of imaging pixels; and

a second optical system provided between the first optical system and the first imaging unit, the second optical system including the first microlens array,

the second imaging device being provided to be separated from the first imaging device and including;

a third optical system;

a second imaging unit, implemented by the circuitry, provided between the third optical system and an imaging position of the third optical system, the second imaging unit including a plurality of imaging pixels; and

a fourth optical system provided between the third optical system and the second imaging unit, the fourth optical system including the second microlens array.

12. The calculating device according to claim 11, wherein the calculating circuitry is configured to perform

the first calculation for an overlapping region where a first imaging region of the first imaging device and a second imaging region of the second imaging device overlap, and

the second calculation for one other region of the first imaging region and second imaging region other than the overlapping region.

13. The calculating device according to claim 11, wherein the calculating circuitry is configured to perform

the first calculation and the second calculation for an overlapping region where a first imaging region of the first imaging device and a second imaging region of the second imaging device overlap, and

the second calculation for one other region of the first imaging region and second imaging region other than the overlapping region.

14. The calculating device according to claim 11, wherein the calculating circuitry is configured to select one selected from the first distance and the second distance according to an instruction transmitted from an outside.

15. The calculating device according to claim 11, wherein the calculating circuitry is configured to implement at least a portion of the second calculation and at least a portion of the first calculation in parallel.

16. The calculating device according to claim 11, wherein an optical axis of the first optical system is non-parallel to an optical axis of the third optical system.

17. A non-transitory computer-readable medium including computer executable instructions, wherein the instructions, when executed by a computer, cause the computer to perform a method, the method comprising:

performing a first calculation including deriving a first distance from stereo disparity using a first image signal and a second image signal, the first image signal being of an object obtained by a first imaging device, the second image signal being of the object obtained by a second imaging device;

performing a second calculation including deriving a second distance from a parallax image obtained via at least one selected from a first microlens array and a second microlens array; and

adopting the first distance as a target distance in response to the first distance exceeding a threshold, and adopting the second distance as the target distance in response to the first distance not being more than the threshold,

the first imaging device including;

a first optical system;

a first imaging unit, implemented by circuitry, provided between the first optical system and an imaging position of the first optical system, the first imaging unit including a plurality of imaging pixels; and

a second optical system provided between the first optical system and the first imaging unit, the second optical system including the first microlens array,

the second imaging device being provided to be separated from the first imaging device and including;

a third optical system;

a second imaging unit, implemented by the circuitry, provided between the third optical system and an imaging position of the third optical system, the second imaging unit including a plurality of imaging pixels; and

a fourth optical system provided between the third optical system and the second imaging unit, the fourth optical system including the second microlens array.