CMOS IMAGE SENSOR FOR DEPTH MEASUREMENT USING TRIANGULATION WITH POINT SCAN

• US 20160307325A1
• Filed: 09/01/2015
• Published: 10/20/2016
• Est. Priority Date: 04/20/2015
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Dirk Smits, Ilia Ovsiannikov, Yibing Michelle Wang
• Inventors: SMITS, Dirk, WANG, Yibing Michelle, OVSIANNIKOV, Ilia
• Granted Patent: US 10,145,678 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01B 11/22 for measuring depth
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01B 11/2518 Projection by scanning of the object
• CPC Class Codes: G03B 15/02 Illuminating scene
• CPC Class Codes: G03B 17/54 with projector
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 2213/003 Aspects relating to the "2D+depth" image format
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 25/75 Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 25/77 Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components

Abstract

Using the same image sensor to capture both a two-dimensional (2D) image of a three-dimensional (3D) object and 3D depth measurements for the object. A laser point-scans the surface of the object with light spots, which are detected by a pixel array in the image sensor to generate the 3D depth profile of the object using triangulation. Each row of pixels in the pixel array forms an epipolar line of the corresponding laser scan line. Timestamping provides a correspondence between the pixel location of a captured light spot and the respective scan angle of the laser to remove any ambiguity in triangulation. An Analog-to-Digital Converter (ADC) in the image sensor generates a multi-bit output in the 2D mode and a binary output in the 3D mode to generate timestamps. Strong ambient light is rejected by switching the image sensor to a 3D logarithmic mode from a 3D linear mode.

Claims

1. A method comprising:

performing a one-dimensional (1D) point scan of a three-dimensional (3D) object along a scanning line using a light source, wherein the point scan projects a sequence of light spots on a surface of the 3D object;

selecting a row of pixels in an image sensor, wherein the image sensor has a plurality of pixels arranged in a two-dimensional (2D) array forming an image plane, and wherein the row of pixels forms an epipolar line of the scanning line on the image plane;

detecting each light spot using a corresponding pixel in the row of pixels;

generating a pixel-specific output in response to a pixel-specific detection of a corresponding light spot in the sequence of light spots; and

determining a distance to the corresponding light spot on the surface of the 3D object based at least on the pixel-specific output and on a scan angle used by the light source for projecting the corresponding light spot.

2. The method of claim 1, wherein each pixel in the row of pixels is one of the following:

a Four Transistor (4T) pixel;

a Three Transistor (3T) pixel;

a Two Transistor (2T) pixel;

a One Transistor (1T) pixel;

a shared-transistor pixel;

a 1×

2 2-shared pixel; and

a 1×

4 4-shared pixel.

3. The method of claim 1, wherein the light source is one of the following:

a laser light source;

a visible laser light source;

a point light source;

a near infrared laser light source; and

a monochromatic illumination source.

4. The method of claim 3, wherein the light source is an X-Y addressable light source.

5. The method of claim 1, wherein the 2D array is a Complementary Metal Oxide Semiconductor (CMOS) array.

6. The method of claim 1, wherein detecting each light spot includes:

receiving, at a pixel in the row of pixels, light reflected from projection of one or more light spots on the 3D object.

7. The method of claim 1, wherein determining the distance to the corresponding light spot includes:

performing a triangulation-based calculation using the following parameters to determine the distance to the corresponding light spot;

the scan angle of the light source;

an offset distance between the light source and a collection optics associated with the image sensor;

a distance between the collection optics and the 2D array; and

an offset distance between the collection optics and the pixel that detects the corresponding light spot.

8. The method of claim 1, wherein determining the distance to the corresponding light spot includes:

retrieving a value for the distance from a Look-Up Table (LUT) using the following parameters as inputs into the LUT;

the scan angle of the light source; and

an offset distance between the collection optics and the pixel that detects the corresponding light spot,

wherein the LUT is pre-populated with a plurality of distance values as a function of different scan angles of the light source and different offset distances between the collection optics and the pixels in the row of pixels.

9. The method of claim 8, further comprising:

generating the LUT with the plurality of distance values calculated using triangulation.

10. The method of claim 1, wherein generating the pixel-specific output includes:

generating a pre-determined binary value as an output of an Analog-to-Digital Converter (ADC) in the absence of the pixel-specific detection of the corresponding light spot;

sensing the pixel-specific detection of the corresponding light spot; and

in response to sensing the pixel-specific detection of the corresponding light spot, using the ADC as a binary counter to generate a timestamp value for the corresponding light spot as the pixel-specific output.

11. The method of claim 10, further comprising:

when a given pixel in the row of pixels detects a plurality of light spots in the sequence of light spots, using an ADC associated with the given pixel to generate a respective timestamp value for each light spot in the plurality of light spots detected by the given pixel;

comparing timestamp values for the light spots detected by the given pixel to determine an earliest-detected light spot and a latest-detected light spot in the plurality of light spots; and

providing only one of the following as the pixel-specific output for the given pixel;

the respective timestamp value for the earliest-detected light spot, or

the respective timestamp value for the latest-detected light spot.

12. The method of claim 10, further comprising:

using timestamp values for light spots detected by pixels in the row of pixels to establish a temporal correlation among time-stamped light spots; and

determining distances to time-stamped light spots in an order specified by the temporal correlation.

13. The method of claim 10, further comprising:

establishing a pre-determined threshold value;

adjusting the pre-determined threshold value to compensate for a pixel-specific dark current, thereby generating an adjusted threshold value; and

using the adjusted threshold value to sense the pixel-specific detection of the corresponding light spot.

14. The method of claim 10, further comprising:

using the timestamp value for the corresponding light spot to establish a correspondence between the following;

an offset distance between a collection optics associated with the image sensor and the pixel that detects the corresponding light spot, and

the scan angle of the light source; and

determining the distance to the corresponding light spot based on the offset distance and the scan angle.

15-29. -29. (canceled)