Method and apparatus for picking up a three-dimensional range image

• US 6,373,557 B1
• Filed: 06/09/2000
• Issued: 04/16/2002
• Est. Priority Date: 12/23/1997
• Status: Expired due to Term

• Current Assignee: Siemens AG
• Original Assignee: Siemens AG
• Inventors: Mengel, Peter, Doemens, Günter
• Primary Examiner(s): Tarcza, Thomas H.
• Assistant Examiner(s): Mull, Fred H.
• Application Number:
  US09/581,091
• Time in Patent Office: 676 Days
• Field of Search: 356/4.07, 356/5.05, 356/5.06, 356/5.07, 356/5.08, 356/5.03
• US Class Current: 356/4.07
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: G01S 17/04 Systems determining the presence of a target
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/487 Extracting wanted echo signals , e.g. pulse detection

Abstract

A method and apparatus for generating three-dimensional range images of spatial objects, wherein a short-time illumination of the object is performed, for instance using laser diodes. An image sensor is used that has a high light sensitivity, pixel resolution and can be read out randomly, this sensor also having an integration time that can be adjusted for each pixel. By evaluating the backscattered laser pulses in two integration windows with different integration times and by averaging over several laser pulses, three-dimensional range images can be picked up with a high reliability in, for example, 5 ms at the most.

Claims

1. A method for picking up a three-dimensional range image of spatial objects using an optoelectronic sensor with pixel resolution having electronic short-time integrators for each pixel element within the sensor, wherein an integration time can be adjusted, comprising the steps of:

illuminating an object having a plurality of object points with one or more light pulses each having a predetermined period Δ

_L;

sensing light pulses with the sensor that have been backscattered by object points of the object at corresponding pixels of the sensor within a predetermined short integration time Δ

_A, where Δ

<sub>A≦

Δ

L</sub>, and wherein a time instant for a beginning of the predetermined short integration time Δ

_A precedes incidence of the first backscattered light pulse at the sensor, which corresponds to a nearest object point;

registering intensities of each of the sensed light pulses that have been backscattered by the object points; and

computing distance values from different registered intensities of the backscattered light pulses resulting from their different transit times.

2. The method as claimed in claim 1, wherein, all backscattered light pulses are sensed in their entirety with a long integration time Δ

_B where Δ

_B>

>

Δ

_L for one of simultaneous normalizing and successive normalizing of surface reflection of the object.

3. The method as claimed in claim 1, wherein the time length of the light pulse is less than 100 ns.

4. The method as claimed in claim 1, wherein the object is illuminated with light pulses from one of a laser and a pulsed LED.

5. The method as claimed in claim 1, wherein the integrated sensor signal is read out for one of every individual pixel and a segment of pixels in a random pixel access process, wherein a segment can be a row of the optoelectronic image sensor.

6. The method as claimed in claim 1, wherein a spatial object is illuminated using two or more light sources that respectively partially illuminate the object;

wherein the two or more light sources are activated in series; and

evaluation is performed for each respective partial illumination.

7. The method as claimed in claim 1, wherein the integration time is configured to be adjusted pixel by pixel.

8. The method as claimed in claim 1, wherein a beginning of integration times Δ

_A and Δ

_B is delayed by a trigger impulse delay relative to the emitted pulse.

9. The method as claimed in claim 1, wherein the integration time Δ

_A is less than 100 ns.

10. The method as claimed in claim 1, wherein the integration time Δ

_B is approximately 1 μ

s.

11. The method as claimed in claim 1, wherein different integration times Δ

_A and Δ

_B are respectively adjusted row by row in order to simultaneously acquire a three-dimensional image and a gray value image on the sensor.

12. The method as claimed in claim 1, wherein different integration times Δ

_A and Δ

_B are respectively adjusted pixel by pixel in order to simultaneously acquire a three-dimensional image and a gray value image on the sensor.

13. The method as claimed in claim 1, wherein the sensor is read out randomly.

14. The method as claimed in claim 1, wherein the sensor is a CMOS sensor.

15. The method as claimed in claim 1, wherein the distance from the sensor to at least one reference point is set as a reference distance.

16. The method as claimed in claim 15, wherein the reference point is located at a door frame of a vehicle.

17. The method as claimed in claim 1, wherein at least one of static objects and motion sequences is detected.

18. The method as claimed in claim 17, wherein the object is one of physical objects and persons located in at least one of defined spaces and vehicles being monitored.

19. The method as claimed in claim 18, wherein at least one of a seat occupancy and a seat position of a person is detected.

20. The method as claimed in claim 17, wherein at least one of vehicles and crane systems is monitored and wherein a general position determination is executed in a navigation system.

21. A method for picking up a three-dimensional range image of spatial objects using an optoelectronic sensor with pixel resolution having electronic short-time integrators for each pixel element, wherein an integration time can be adjusted, comprising the steps of:

picking up and integrating a sensor signal at the sensor from a beginning of the picking up and integration to a defined integration time T₂, the integration representing a dark current and environmental light;

illuminating an object by an illumination device simultaneous to the beginning of the picking up and integration of the sensor signal at the sensor, wherein integration occurs within a light intensity rise of the light received at the sensor up to an integration time T₁, and T₁ is less than T₂;

repeatedly illuminating the object by the illumination device with simultaneous starting of the picking up and integration of the sensor signal at the sensor, wherein integration occurs within the light intensity rise of the light received at the sensor up to the integration time T₂;

reading out and storing for all pixels the respectively integrated value of the sensor signal at the times T₁ and T₂; and

calculating for each pixel a transit time T₀ of the light from the illumination device to the sensor via the object and a corresponding distance value based on the stored integrated values.

22. The method as claimed in claim 21, wherein the step of picking up the sensor signal is repeated multiple times, and the corresponding resulting signals for each individual pixel are integrated, and the steps of reading out, storing and evaluating of the sensor signals are only performed subsequent to the repeated illumination.

23. The method as claimed in claim 21, wherein the stored values are digitized.

24. The method as claimed in claim 21, wherein an adaptive adjustment to the reflectivity of the object occurs by increasing the number of multiple illuminations until the integrated intensity values at T₁ and at T₂ near a saturation point in a predetermined quantity of pixels of the overall image.

25. The method as claimed in claim 21, wherein at least one of the number of multiple illuminations and the intensity of the illumination device is increased until the integrated intensity values at T₁ and at T₂ near a saturation point for each individual pixel of the overall image.

26. The method as claimed in claim 21, wherein the integrated sensor signal is read out for one of every individual pixel and a segment of pixels in a random pixel access process, wherein a segment can be a row of the optoelectronic image sensor.

27. The method as claimed in claim 21, wherein a spatial object is illuminated using two or more light sources that respectively partially illuminate the object;

and

wherein the two or more light sources are activated in series and

the evaluation is performed for each respective partial illumination.

28. The method as claimed in claim 21, wherein the integration time is configured to be adjusted pixel by pixel.

29. The method as claimed in claim 21, wherein a beginning of an integration time is delayed by a trigger impulse delay relative to the emitted pulse.

30. The method as claimed in claim 21, wherein the integration time is less than 100 ns.

31. The method as claimed in claim 21, wherein the integration time is approximately 1 μ

s.

32. The method as claimed in claim 21, wherein different integration times T₁ and T₂ are respectively adjusted row by row in order to simultaneously acquire a three-dimensional image and a gray value image on the sensor.

33. The method as claimed in claim 21, wherein different integration times T₁ and T₂ are respectively adjusted pixel by pixel in order to simultaneously acquire a three-dimensional image and a gray value image on the sensor.

34. The method as claimed in claim 21, wherein the sensor is read out randomly.

35. The method as claimed in claim 21, wherein the sensor is a CMOS sensor.

36. The method as claimed in claim 21, wherein the distance from the sensor to at least one reference point is set as a reference distance.

37. The method as claimed in claim 36, wherein the reference point is located at a door frame of a vehicle.

38. The method as claimed in claim 21, wherein at least one of static objects and motion sequences is detected.

39. The method as claimed in claim 38, wherein the object is one of physical objects and persons located in at least one of defined spaces and vehicles being monitored.

40. The method as claimed in claim 39, wherein at least one of a seat occupancy and a seat position of a person is detected.

41. The method as claimed in claim 38, wherein at least one of vehicles and crane systems is monitored and wherein a general position determination is executed in a navigation system.

42. An apparatus for picking up a three-dimensional range image, comprising:

an illuminating device that emits light pulses onto an object via a first optical system;

an optoelectronic sensor with a second optical system configured to sense received light pulses backscattered by the object within an adjustable integration time and is comprised of a plurality of pixel elements to provide a pixel resolution, the pixel elements being randomly readable and configured to adjust the integration time pixel by pixel;

a triggering mechanism configured to provide time synchronization between the illumination device and the sensor; and

a computing unit for calculating a three-dimensional image from corresponding charges of pixel elements of the sensor that have been charged by the received light pulses.

43. The apparatus as claimed in claim 42, wherein a short integration time and a long integration time are alternately set for one of row by row and column by column at the pixel elements of the sensor.

44. The apparatus as claimed in claim 42, wherein short and long integration times are alternately set at the pixel elements of the sensor.

45. The apparatus as claimed in claim 42, wherein the computing unit is arranged on the sensor.