Apparatus, method, and program for generating range-image-data

• US 20050162638A1
• Filed: 12/09/2004
• Published: 07/28/2005
• Est. Priority Date: 01/28/2004
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Sano, Naoki, Suzuki, Yousuke, Yamada, Hitoshi, Sugawara, Ryoichi
• Granted Patent: US 7,230,685 B2
• Time in Patent Office: Days
• US Class Current: 356/4.40
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01S 17/04 Systems determining the presence of a target
• CPC Class Codes: G01S 17/88 Lidar systems specially adapted for specific applications
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/4863 Detector arrays, e.g. charge-transfer gates
• CPC Class Codes: G01S 7/4876 by removing unwanted signals G01S7/495 takes precedence
• CPC Class Codes: H01L 27/146 Imager structures

Abstract

First image data including a first pixel value according to the intensity of background light is obtained by means of only exposure. Second image data including a second pixel value according to the intensity of the light reflected or scattered by an object is obtained by so controlling light emission and exposure as to receive part of the reflected or scattered light over the whole exposure period. The part of the reflected or scattered light corresponds to the flat period of emitted pulsed light. Third image data including a third pixel value according to the distance to the object is obtained by so controlling light emission and exposure as to receive the part of the reflected or scattered light for a time according to the distance. The first pixel value is subtracted from each of the second and third pixel values so that the influence of the background light can be excluded. The ratio between the subtraction results is calculated so that the influences of factors that vary the intensity of the reflected or scattered light can be cancelled.

Claims

1. An apparatus for generating range-image-data, the apparatus comprising:

a light-emitting means for emitting pulsed light within a preset measuring range, the pulsed light having a rise period, a flat period, and a fall period;

an imaging means for photographing an object existing in the measuring range, the imaging means including a photosensor comprised of a two-dimensional array of charge integration type photoelectric conversion elements, the photosensor adapted to be exposed adjustably;

an image-data-obtaining means for obtaining image data by controlling the emission period or the emission-start timing for the light-emitting means and the exposure period or the exposure-start timing for the imaging means, the image data being such that a value of each pixel is the output from one of the photoelectric conversion elements;

a range-image-data-generating means for generating range-image-data based on the obtained image data, the range-image-data comprised of pixels, each having a pixel value according to the distance to the photographed object;

the image-data-obtaining means including a first image-data-obtaining means for obtaining first image data from the imaging means by causing the imaging means to make an exposure, without causing the light-emitting means to emit pulsed light;

the image-data-obtaining means further including a second image-data-obtaining means for obtaining second image data from the imaging means by so setting the emission period or the emission-start timing and the exposure period or the exposure-start timing in order to receive a part of the light reflected or scattered from the object over the whole exposure period, the part corresponding to the flat period of the pulsed light;

the image-data-obtaining means further including a third image-data-obtaining means for obtaining third image data from the imaging means by so setting the emission period or the emission-start timing and the exposure period or the exposure-start timing in order to receive the part of the reflected or scattered light for a time according to the round-trip time for the distance to the object;

the range-image-data-generating means including a first arithmetic means for finding reference image data by subtracting the first image data from the second image data;

the range-image-data-generating means further including a second arithmetic means for finding intensity image data by subtracting the first image data from the third image data;

the range-image-data-generating means further including a third arithmetic means for finding the range-image-data by calculating the ratio between the reference and intensity image data.

2. The apparatus of claim 1, wherein the second image-data-obtaining means sets the exposure period arbitrary and the emission period equal to or longer than the sum of the rise period of the pulsed light, the maximum round-trip time taken by the pulsed light to make a round trip over the distance to the boundary of the measuring range, and the exposure period, and wherein the emission from the light-emitting means starts before the exposure of the photosensor by a time equal to or longer than the sum of the rise period and the maximum round-trip time.

3. The apparatus of claim 1,

wherein the third image-data-obtaining means sets the emission period as the flat period is equal to the maximum round-trip time, and sets the exposure period equal to or longer than the flat period, and

wherein the emission from the light-emitting means starts before the exposure of the photosensor by the sum of the rise period and the maximum round-trip time.

4. The apparatus of claim 1, wherein the exposure periods set by the first to third image-data-obtaining means are equal.

5. The apparatus of claim 1,

wherein at least one of the first and second image-data-obtaining means sets the associated exposure period different from the exposure period set by the third image-data-obtaining means, and

wherein the first and second arithmetic means find the reference and intensity-image data by standardizing the pixel values of the first to third image data to values per preset unit time.

6. The apparatus of claim 1, wherein at least one of the exposure periods set by the first to third image-data-obtaining means is divided into a plurality of periods.

7. The apparatus of claim 1, wherein the first and second arithmetic means accumulate pixel values of the reference and intensity image data, and

wherein the third arithmetic means generates the range-image-data based on the reference and intensity image data having the accumulated pixel values.

8. The apparatus of claim 1, further comprising a distance-calculating means for finding the absolute distance to the object based on the range-image-data generated by the third arithmetic means.

9. The apparatus of claim 8, wherein the distance-calculating means finds the absolute distance to the object by multiplying the value of each pixel of the range-image-data by a preset constant.

10. The apparatus of claim 8, wherein the distance-calculating means finds the absolute distance to the object by using a conversion table showing the relation between the pixel value and the absolute distance.

11. The apparatus of claim 1, wherein at least one of the pixels of the photoelectric conversion elements is a reference pixel so set as to receive the light reflected or scattered from a reference object, and wherein the distance between the reference pixel and the reference object is already known;

the distance-calculating means adapted to correct the absolute distance between each pixel and the object in the measuring range based on the absolute distance calculated for the reference pixel.

12. The apparatus of claim 1, further comprising:

a saturation-time-measuring means for measuring the saturation time taken by the output from the photoelectric conversion elements to be saturated while the light-emitting means and the imaging means are making an emission and an exposure, respectively;

the second image-data-obtaining means is adapted to set the exposure period equal to or shorter than the saturation time.

13. The apparatus of claim 1, wherein each of the photoelectric conversion elements is a PIN type photodiode including a P-layer, an N-layer, and an I-layer interposed between the P-layer and the N-layer, the I-layer having low impurity concentration.

14. The apparatus of claim 1, wherein the charge generated by each of the photoelectric conversion elements is stored in a capacitor separate from the conversion elements.

15. A method for generating range-image-data based on image data such that the pixel values of pixels are the output from a photosensor comprised of a two-dimensional array of charge integration type photoelectric conversion elements, the method comprising:

a first step of obtaining first image data from the photosensor by exposing the photosensor without irradiating an object in a preset measuring range with pulsed light having a rise period, a flat period, and a fall period;

a second step of obtaining second image data from the photosensor by irradiating the object with the pulsed light and exposing the photosensor in order to receive a first part of the light reflected or scattered from the object, the first part corresponding to the flat period of the pulsed light;

a third step of obtaining third image data from the photosensor by irradiating the object with the pulsed light and exposing the photosensor in order to receive a second part of the reflected or scattered light for a time according to the round-trip time for the distance to the object, the second part corresponding to the flat and fall periods of the pulsed light;

a fourth step of finding reference image data by subtracting the first image data from the second image data;

a fifth step of finding intensity image data by subtracting the first image data from the third image data; and

a sixth step of finding the range-image-data by calculating the ratio between the reference and intensity image data.

16. The method of claim 15, wherein the method is executed by a computer program.

17. A method for generating range data comprising:

performing a background light exposure for a first predetermined time period so that a reference pixel value of each of a plurality of unit circuits varies with an intensity of background light;

emitting first pulsed light toward a distant object and performing an exposure of a first received reflection of the first irradiated light for a second predetermined time period so that a second pixel value of each of the plurality of unit circuits varies with an intensity of the reflected emitted light;

emitting second pulsed light towards a distant object and performing an exposure of a second received reflection of the second irradiated light for a third predetermined time period so that a third pixel value of each of the plurality of unit circuits varies with a distance to the distant object; and

determining the range data based upon the reference pixel value, second pixel value and the third pixel value, wherein the range data depends only upon the distance to the distant object and is substantially independent of the intensity of the background light,

wherein the second predetermined time period begins after a rise time and maximum round trip time of the emitted light, wherein the third predetermined time period is substantially equal to or greater than the maximum round trip time of the emitted light, and the third predetermined time period begins after the rise time and maximum round trip time of the irradiated light.

18. An apparatus for generating range-image-data, the apparatus comprising:

an image receiving unit for receiving a reflection of the emitted pulse light and background light, and for performing a first exposure to generate first image data corresponding to an intensity of background light, a second exposure to generate second image data corresponding to an intensity of the reflected emitted light, and a third exposure to generate third image data corresponding to a distance to the distant object, wherein the image receiving unit comprises a photosensor comprised of a two-dimensional array of charge integration type photoelectric conversion elements;

a control signal generating unit for controlling a light emitting unit to emit pulse light towards a distant object and for controlling the image receiving unit; and

an arithmetic unit for receiving the first, second and third image data from the image receiving unit and generating the range data, and for controlling the control signal generating unit so that the first exposure, second exposure, and third exposure are respectively performed at a first predetermined time period, second predetermined time period, and third predetermined time period, wherein during the first predetermined time period, no light is emitted from the light emitting unit, wherein the third predetermined time period is substantially equal to or greater than the maximum round trip time of the emitted light, and the third predetermined time period begins after the rise time and maximum round trip time of the irradiated light.

19. The apparatus according to claim 18, wherein the arithmetic unit determines reference image data by subtracting the first image data from the second image data, determines an intensity image data by subtracting the first image data from the third image data, and determines the range-image-data by calculating the ratio between the reference and intensity image data.