THREE-DIMENSIONAL IMAGE SENSOR, OPTICAL RADAR APPARATUS, AND PROCESSING APPARATUS

• US 20200137373A1
• Filed: 04/06/2018
• Published: 04/30/2020
• Est. Priority Date: 04/28/2017
• Status: Active Grant

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Inventors: IGUCHI, KATSUJI
• Granted Patent: US 10,873,740 B2
• Time in Patent Office: Days
• CPC Class Codes: G01C 3/06 Use of electric means to obtain final indication
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/4863 Detector arrays, e.g. charge-transfer gates
• CPC Class Codes: G01S 7/4865 Time delay measurement, e.g. time-of-flight measurement, time of arrival measurement or determining the exact position of a peak peak detection in noise, signal conditioning G01S7/487
• CPC Class Codes: G01S 7/487 Extracting wanted echo signals , e.g. pulse detection
• CPC Class Codes: H01L 27/14623 Optical shielding
• CPC Class Codes: H01L 27/14627 Microlenses
• CPC Class Codes: H01L 27/14643 Photodiode arrays; MOS imagers
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

An object is to realize a low-cost three-dimensional image sensor capable of providing a wide measurement range. The three-dimensional image sensor includes a light receiving unit including a plurality of pixels each including an SPAD (single-photon avalanche photodiode) configured to detect, in Geiger mode, a photon of incident light including reflected light from an object to be detected, a plurality of pixel storage elements provided for the respective pixels in a one-to-one manner, each pixel storage unit being configured to count a accumulated value of pulse signals output from a corresponding one of the pixels as a result of detecting photons for each of temporally successive time intervals, and a signal processing circuit (DS) configured to calculate distance information regarding a distance between a light receiving unit and an object for each pixel by using groups of accumulated values accumulated by the pixel storage elements.

Claims

1. A three-dimensional image sensor comprising:

a light receiving unit including a plurality of pixels each including an avalanche photodiode configured to detect, in Geiger mode, a photon of incident light including reflected light from an object to be detected;

a plurality of accumulating units provided for the respective pixels in a one-to-one manner, each accumulating unit being configured to accumulate pulse signals output from a corresponding one of the pixels as a result of detecting photons for each of temporally successive time intervals; and

a calculation unit configured to calculate distance information regarding a distance between the light receiving unit and the object for each pixel by using a time series of groups of accumulated values accumulated by the accumulating units, wherein

the accumulating units are disposed outside the light receiving unit.

2. The three-dimensional image sensor according to claim 1, wherein the calculation unit has a function of determining an accumulated value corresponding to a background light signal and a noise level of the groups of accumulated values corresponding to the background light signal based on the groups of accumulated values, a function of determining an accumulated value greater than a sum of the accumulated value corresponding to the background light signal and the noise level, further determining a temporally adjacent accumulated value, and extracting a pair of the resultant accumulated values, and a function of determining a time of flight of light from the extracted pair with an accuracy shorter than the time interval.

3. The three-dimensional image sensor according to claim 1, wherein the calculation unit performs reading of the groups of accumulated values from the accumulating units in parallel with accumulating pulse signals performed by the accumulating units.

4. The three-dimensional image sensor according to claim 1, wherein the light receiving unit, the accumulating unit, and the calculation unit are monolithically formed on a silicon substrate.

5. The three-dimensional image sensor according to claim 1, wherein the pixels each includes a plurality of avalanche photodiodes.

6. The three-dimensional image sensor according to claim 1, wherein

the accumulating units each includes

a plurality of binary counters for accumulating the pulse signals for each time interval.

7. The three-dimensional image sensor according to claim 6, wherein

the accumulating units each includes

a switching circuit configured to switch the accumulated values from the respective binary counters for each binary counter and output the accumulated values to the calculation unit,

and wherein the number of signal lines via Which the accumulated values are output from the switching circuit to the calculation unit is equal to or greater than the maximum number of output bits of each binary counter.

8. The three-dimensional image sensor according to claim 1, wherein the calculation unit reads the groups of accumulated values from the accumulating units in a prestored order.

9. The three-dimensional image sensor according to claim 1, wherein the calculation unit further calculates and outputs light intensity info oration representing an intensity of the incident light using the accumulated values.

10. An optical radar apparatus comprising:

a light emitting element configured to emit pulsed light to illuminate an object to be detected, and

the three-dimensional image sensor according to claim 1,

wherein the avalanche photodiode in the three-dimensional image sensor detects a photon of reflected light of the pulsed light reflected from the object.

11. The optical radar apparatus according to claim 10, wherein

the light emitting element emits pulsed light a plurality of times, and

the accumulating unit of the three-dimensional image sensor accumulates a time series of groups of accumulated values acquired by the three-dimensional image sensor each time a plurality of pulses of light are emitted.

12. The optical radar apparatus according to claim 10, wherein the three-dimensional image sensor outputs distance information regarding the distance to the object, first light intensity information representing an intensity of reflected light of the pulsed light radiated to the object and reflected from the object, and second light intensity information that does not include reflected light of the pulsed light from the object.

13. The optical radar apparatus according to claim 10, wherein a length of the time interval is equal to or less than a full width at half maximum the pulsed light.

14. An optical radar apparatus comprising:

a light emitting element configured to emit pulsed light to illuminate an object to be detected; and

the three-dimensional image sensor according to claim 6,

wherein the avalanche photodiode in the three-dimensional image sensor detects a photon of reflected light of the pulsed light reflected from the object, and part of the binary counters of each of the accumulating units accumulates a number of photons of the incident light before pulsed light is emitted to irradiate the object.

15. A processing apparatus configured to recognize the object using the second light intensity information output from the optical radar apparatus according to claim 12.

16. A processing apparatus configured to detect the object approaching or moving away by detecting a time-dependent change in the first light intensity information output from the optical radar apparatus according to claim 12.

17. A processing apparatus configured to:

using the distance information, the first light intensity information, and the second light intensity information, all output from the optical radar apparatus according to claim 12;

in case where information from another image sensor is used together, recognize the object which is common in both the information from the other image sensor and the second light intensity information by comparing the information from the other image sensor and the second light intensity information; and

when there is a large error which makes it difficult to determine the distance, detect a time-dependent change in the first light intensity information output.

18. An optical radar apparatus comprising:

a light emitting element configured to emit pulsed light to illuminate an object to be detected, and

the three-dimensional image sensor according to claim 3,

wherein the optical radar apparatus can output its measurement result in middle of a frame in addition to a measurement result at the end of the frame.

19. The optical radar apparatus according to claim 18, wherein the measurement result in middle of a frame is about the object close to the optical radar apparatus.

20. The optical radar apparatus according to claim 18, wherein the measurement result in middle of a frame is generated before:

overflow occur in the pixel storage elements of the three-dimensional image sensor.