Three-dimensional perception of environment

• US 7,230,640 B2
• Filed: 03/26/2002
• Issued: 06/12/2007
• Est. Priority Date: 03/26/2001
• Status: Expired due to Fees

• Current Assignee: Daimler AG (Mercedes-Benz Group AG)
• Original Assignee: Daimler Chrysler Company LLC (Stellantis NV)
• Inventors: Stahs, Thomas, Schanz, Alexander, Regensburger, Uwe
• Primary Examiner(s): Rao; Andy S.
• Application Number:
  US10/106,599
• Publication Number:
  US 20020169537A1
• Time in Patent Office: 1,904 Days
• Field of Search: 348113-133, 348143-159
• US Class Current: 348/115
• CPC Class Codes: B60T 2201/10 Automatic or semi-automatic parking aid systems
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 2013/9314 Parking operations
• CPC Class Codes: G01S 2013/9323 Alternative operation using light waves
• CPC Class Codes: G01S 2013/93271 in the front of the vehicles
• CPC Class Codes: G01S 2013/93272 in the back of the vehicles
• CPC Class Codes: G01S 2013/93274 on the side of the vehicles
• CPC Class Codes: G01S 2013/93277 in the lights
• CPC Class Codes: G06V 20/586 of parking space

Abstract

A vehicle-useful, high-resolution 3D-detection of the environment of a street vehicle with environment scanning sensors is not possible today. Depending on application the first commercially available vehicle integrated systems may be a comprise between resolution of the sensors and size of the sampled area. With the inventive two dimensional distance resolution sensor device it becomes possible, in contrast to those which are known, to produce a system which, installed in a street vehicle, detects and preferably processes complex dynamics scenarios such for example street traffic, from the prospective of the active dynamic moving vehicle. Therein a three dimensional image of the environment is produced using a distance sensor, which produces a two dimensional distance profile (depth profile), and on the other hand, using a data processing and a storage unit, which process and store sequential distance profiles and sequence a series of distance profiles into the three dimensional image of the environment.

Claims

1. A device for three-dimensional determination of the environment about a street vehicle dynamically moving in a travel direction, comprising:

only a single distance sensor, the single distance sensor having a beam coverage with an orientation not parallel to the travel direction of the street vehicle, the distance sensor producing a series of two dimensional distance profiles,

a data processing unit and a storage unit, which preprocess and store a sequence of distance profiles produced at different time points along a travel track in the travel direction of the street vehicle, and produce a three dimensional image of the environment from the sequence of distance profiles, and

a means for determining the distance covered by the street vehicle between each individual produced distance profile.

2. The device according to claim 1, further comprising a supplemental means for determining at least in two spatial dimensions the relative position and the orientation of the vehicle within the environment.

3. The device according to claim 2, wherein the supplemental means for determination of the relative position is a radar.

4. The device according to claim 3, wherein said radar is a millimeter wavelength radar or a lidar.

5. The device according to claim 2, wherein the supplemental means for determination of the relative position is a camera device with associated video image processor is provided.

6. The device according to claim 2, wherein tho supplemental means for determination of the relative position is a navigation system.

7. The device according to claim 2, wherein a moans is provided in connection with a distance sensor, such that a part of the span of the beam scan of the distance sensor is deflected into a different spatial area than that which was to be determined three-dimensionally by sequential arrangement of distance profiles, whereby distance information and variance over time detected from this part of the light sector can be used for determining the relative position.

8. The device according to claim 7, wherein the means for the beam defection is a mirror.

9. The device according to claim 2, further including a means for suppressing the determination of the relative position.

10. The device according to claim 1, wherein the distance sensor is so arranged that the distance sensor is so positioned that the distance profile is recorded at an angle relative to the normal plane on the road surface.

11. The device according to claim 10, wherein said angle relative to the normal plane on the road surface is an angle of approximately 45°

.

12. The device according to claim 1, wherein the device is integrated in an optical element provided on the street vehicle.

13. The device according to claim 12, wherein the optical clement is a back-up light or a headlight.

14. The device according to claim 1, wherein said device is adapted for detection of parking spaces.

15. A process for three-dimensional determination of the environment about a street vehicle dynamically moving in a travel direction, said process comprising:

producing a two-dimensional distance profile by means of only a single distance sensor, the single distance sensor having a beam coverage with an orientation not parallel to the travel direction of the street vehicle,

processing and storing a sequence of distance profiles at different time points along a travel track in the travel direction of the street vehicle via a data processing unit and a storage unit and producing a three dimensional image of the environment from the sequence of distance profiles, and

determining the path covered between each of the individual produced distance profiles using a distance measuring means.

16. The process according to claim 15, further comprising determining the relative position and the orientation of the vehicle within its environment in at least in two spatial dimensions, and

correction for the variations of the orientation of the distance sensors during the production of the three dimensional image of the environment due to the own movement of the vehicle.

17. The process according to claim 16, wherein the relative position is determined by means of a radar with an associated radar signal processor.

18. The process according to claim 16, wherein the relative position is determined using a camera arrangement with associated video image processor.

19. The process according to claim 16, further comprising deflecting a part of the span of the beam scan of the distance sensor into a different spatial area than that which was to be determined three-dimensionally by sequential arrangement of distance profiles, and using distance information and variance over time detected from this part of the light sector for determining the relative position.

20. The process according to claim 16, wherein the determination of the relative position occurs only upon exceeding a preset speed and otherwise is suppressed.

21. The process according to claim 15, wherein for each distance cell within the area to be measured by the distance sensor only the peak point occurring therein is stored and further processed, such that the generated three dimensional image of the environment provides a birds eye view within which in the individual loci the pea values are extracted.

22. The process according to claim 21, wherein task-specific quantification steps are employed for the further quantification.

23. The process according to claim 22, wherein three task specific quantification steps are employed.

24. The process according to claim 23, wherein said three task specific quantification steps are “

no object”

, “

potential collision object”

, and “

not potential collision object”

.

25. The process according to claim 23, comprising using a threshold comparison as the basis for the quantification.

26. The process according to claim 15, comprising

carrying out a continuous analysis of the potential parking spaces adjacent to the vehicle, and

comparing the detected free spares within the framework of the analysis of the three dimensional environmental data with the physical and dynamic measurements of the street vehicle.

27. The process according to claim 15, wherein said process is a process for detection of parking spaces.