Systems And Methods For Active Depth Imager With Background Subtract

• US 20180124372A1
• Filed: 11/02/2016
• Published: 05/03/2018
• Est. Priority Date: 11/02/2016
• Status: Active Grant

• Current Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Original Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Inventors: Yang, Zheng, Funatsu, Eiichi, Manabe, Sohei, Mabuchi, Keiji, Yang, Dajiang, Mao, Duli, Zhang, Bowei
• Granted Patent: US 10,582,178 B2
• Time in Patent Office: Days
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10144 Varying exposure
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/586 from multiple light sources, e.g. photometric stereo
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H01L 27/146 Imager structures
• CPC Class Codes: H01L 27/14614 having a special gate structure
• CPC Class Codes: H01L 27/14643 Photodiode arrays; MOS imagers
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 2013/0092 Image segmentation from stereoscopic image signals
• CPC Class Codes: H04N 25/533 by using differing integration times for different sensor regions
• CPC Class Codes: H04N 25/75 Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 25/771 comprising storage means other than floating diffusion

Abstract

An active depth imaging system and method of operating the same captures illuminator-on and illuminator-off image data with each of a first and second imager. The illuminator-on image data includes information representing an imaged scene and light emitted from an illuminator and reflected off of objects within the imaged scene. The illuminator-off image data includes information representing the imaged scene without the light emitted from the illuminator. For each image set captured by the first and second imagers, illuminator-off image data is subtracted from the illuminator-on image data to identify the illuminated light within the scene. The depth of an object at which the light is incident on then is determined by the subtracted image data of the first and second imagers.

Claims

1. An active depth imaging system with background subtract comprising:

an illuminator for emitting a light ray to an object in a scene, the object reflecting the emitted light ray;

a first imager for imaging a first image set of the scene and a second imager for imaging a second image set of the scene,

the first and second image sets each including an illuminator-on image data representing the scene when the illuminator is in an on mode, and an illuminator-off image data representing the scene when the illuminator is in an off mode;

an image processor programmed to;

generating background-subtracted image data by, for each of the first and second image sets, subtracting the illuminator-off image data from the illuminator-on image data, and

analyzing the first and the second background-subtracted image data to determine a depth of the object reflecting the emitted light ray in the scene.

2. The active depth imaging system of claim 1, wherein the first and second imagers comprising an image array including:

a plurality of pixels, each pixel including;

a photodiode coupled with a first storage component and a second storage component, and

a global shutter switch for controlling charge accumulation within the photodiode,

signal generation circuitry for generating a plurality of signals to control each pixel for a first, second, third, and fourth time periods such that;

during the first time period, the photodiode captures the illuminator-on image data, and storing the illuminator-on image data in the first storage component,

during the second and fourth time periods, the photodiode does not accumulate any charge,

during the third time period, the photodiode captures the illuminator-off image data, and storing the illuminator-off image data in the second storage component.

3. The active depth imaging system of claim 2, each of the first and second imagers further comprising:

readout circuitry for obtaining the illuminator-on and illuminator-off image data from the first and second storage components;

the first storage component including;

a first transfer-storage gate for transferring and storing the illuminator-on image data from the photodiode, and a first output gate for outputting the illuminator-on image data to the readout circuitry; and

the second storage component including;

a second transfer-storage gate for transferring and storing the illuminator-off image data from the photodiode, and a second output gate for outputting the illuminator-off image data to the readout circuitry.

4. The active depth imaging system of claim 3, the first transfer-storage gate adapted to store multiple exposures of illuminator-on image data before the first output gate outputs the stored multiple exposures of illuminator-on image data.

5. The active depth imaging system of claim 4, the second transfer-storage gate adapted to store multiple exposures of illuminator-off image data before the second output gate outputs the stored multiple exposures of illuminator-off image data.

6. The active depth imaging system of claim 3, the plurality of signals including a global shutter signal (GS) for controlling the global shutter, a first transfer-storage signal (TSG1) for controlling the first transfer-storage gate, and a second transfer-storage signal (TSG2) for controlling the second transfer-storage gate.

7. The active depth imaging system of claim 6, a falling edge of the GS being synchronized with a rising edge of at least one of the TSG1 and TSG2, and a rising edge of the GS being synchronized with a falling edge of at least one of the TSG1 and TSG2.

8. The active depth imaging system of claim 6, a falling edge of the GS being before a rising edge of at least one of the TSG1 and TSG2, and a rising edge of the GS being before a falling edge of at least one of the TSG1 and TSG2.

9. The active depth imaging system of claim 6, a falling edge of the GS being after a rising edge of at least one of the TSG1 and TSG2, and a rising edge of the GS being after a falling edge of at least one of the TSG1 and TSG2.

10. The active depth imaging system of claim 6, a falling edge of the GS being before a rising edge of at least one of the TSG1 and TSG2, and a rising edge of the GS being after a falling edge of at least one of the TSG1 and TSG2.

11. The active depth imaging system of claim 6, a falling edge of the GS being after a rising edge of at least one of the TSG1 and TSG2, and a rising edge of the GS being before a falling edge of at least one of the TSG1 and TSG2.

12. The active depth imaging system of claim 1, the illuminator-on image data including data of multiple exposures of the illuminator.

13. A method for operating an active depth imaging system comprising:

a plurality of exposure cycles, each exposure cycle comprising;

for a first time period, configuring each of a first and second imager of the active depth imaging system to capture illuminator-on image data representing an imaged scene and light emitted by an illuminator;

for a second time period, configuring each of the first and second imager to not accumulate a charge;

for a third time period, configuring each of the first and second imager to capture illuminator-off image data representing the imaged scene without the light;

for a fourth time period, configuring each of the first and second imager to not accumulate a charge;

accumulating, for each of the respective first and second imagers, the illuminator-on image data captured in the plurality of exposure cycles and accumulating, for each of the respective first and second imagers, the illuminator-off image data capture in the plurality of exposure cycles;

subtracting, for each of the respective first and second imagers, the accumulated illuminator-off image data from the accumulated illuminator-on image data to identify the light emitted by the illuminator within the scene; and

,

identifying depth information of an object within the imaged scene at which the light is incident on.

14. The method of claim 13, further comprising during the first time period, configuring the illuminator to illuminate the imaged scene more than once.

15. The method of claim 13, further comprising during the first time period, storing the illuminator-on image data captured respectively by each of the first and second imagers in a first storage component of the each respective first and second imagers.

16. The method of claim 13, further comprising reading out the illuminator-on and illuminator-off image data after the plurality of exposure cycles.

17. The method of claim 13, further comprising, during the first time period, toggling the illuminator.

18. The method of claim 13, a duration of each of the first, second, third, and fourth time periods being equal.

19. The method of claim 13, a duration of the first time period being greater than a duration of each of the second, third, and fourth time periods, and a duration of the second time period being less than a duration of the third and fourth time periods.

20. The method of claim 13, a duration of the third time period being greater than a duration of each of the first, second, and fourth time periods, and a duration of the fourth time period being less than a duration of the first and second time periods.

21. The method of claim 13, a duration of each of the first and third time periods being equal to each other and greater than a duration of each of the second and fourth time periods.

22. The method of claim 13, the duration of the second and fourth time periods being equal to each other.