Method and device for video-based observation and measurement of the lateral environment of a vehicle

• US 7,365,831 B2
• Filed: 09/19/2003
• Issued: 04/29/2008
• Est. Priority Date: 09/23/2002
• Status: Expired due to Term

• Current Assignee: Conti Temic Microelectronic GmbH (Continental AG)
• Original Assignee: Daimler AG (Mercedes-Benz Group AG)
• Inventors: Braeunl, Thomas Hans, Franke, Uwe
• Primary Examiner(s): Tarcza, Thomas H.
• Assistant Examiner(s): Ratcliffe, Luke D.
• Application Number:
  US10/528,810
• Publication Number:
  US 20070165908A1
• Time in Patent Office: 1,684 Days
• Field of Search: 356/4.01
• US Class Current: 356/4.01
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/586 of parking space

Abstract

The first commercially-available driver assist systems required a compromise between the resolution of the scanning and the extent of the scanned region, according to application. Conventional video-based systems have a good compromise between resolution and recording range, however do not generally provide direct distance information. According to the new arrangement of the object of the invention, it is possible to achieve a system, which, on installation in a road vehicle, can record complex dynamic scenes, for example the lateral 3D geometry to the road edge from the point of view of the dynamically operating vehicle and use the same to advantage on parking. According to the invention, the monitoring and measuring of the lateral environment of a vehicle is displayed on the one hand by means of a camera with the digital images and on the other hand with a computer unit which serves to provide the images with a time stamp and to buffer the same. The movements of the vehicle are further recorded, in order to select image pairs from the buffered images, based on the above data. A local 3D depth image of the lateral environment of a vehicle can thus be generated by an algorithm for stereo image processing.

Claims

1. A method for observing and measuring the lateral environment of a vehicle,

wherein a sequence of digital images are recorded by means of a camera, provided with a time stamp and buffered,

wherein the vehicle'"'"'s own movement is sensed in order to select chronologically overlapping stereoscopic image pairs from the buffered images on the basis of this data,

wherein the position and orientation of the camera which are present at the two recording times are determined,

wherein a local 3D depth image is generated on the basis of the image pairing by means of an algorithm for stereoscopic image processing,

wherein the position and orientation of the camera at the recording times are taken into account within the scope of a synthetic stereoscopic geometry, and

only buffered images during whose recording the orientation of the camera has not changed significantly are selected to form image pairs.

2. The method as claimed in claim 1, wherein the vehicle'"'"'s own movement is sensed on the basis of the speed, the number of revolutions of the wheel, navigation information or data from vehicle movement dynamics systems.

3. The method as claimed in claim 1, wherein buffered images are selected to farm image pairs in such a way that the distance covered between the recording times of the images corresponds to a predefined value.

4. The method as claimed in claim 3, wherein the predefined value is preferably in the range between 0.2 m and 1 m.

5. The method as claimed in claim 1, wherein a sequence of local 3D depth views are accumulated, wherein the image data of the individual local 3D depth views which can be assigned to the same location points in the environment of the vehicle are added to one another.

6. The method as claimed in claim 5, wherein the image data is subjected to weighting before the addition.

7. The method as claimed in claim 6, wherein the volume which is represented by means of the accumulated 3D depth views is divided into individual volume elements within the scope of the weighting.

8. The method as claimed in claim 7, wherein all the volume elements correspond to a predefined uniform volume.

9. The method as claimed in claim 7, wherein the volume elements are stored in a tree structure, in particular in the form of an octree.

10. The method as claimed in claim 7, wherein the respective number of pixels within the individual volume elements is assigned to each of these volume elements as a weighting.

11. The method as claimed in claim 10, wherein a weighting is determined which corresponds to the average weighting of all the volume elements, this overall weighting forming a threshold value on the basis of which it is decided whether the pixels which are contained in the individual volume elements are taken into account in the further processing, wherein only those pixels at which the volume element assigned to them has a weighting which is equal to or greater than the threshold value are taken into account.

12. The method as claimed in claim 11, wherein only those volume elements which contain pixels are used to form the overall weighting.

13. The method as claimed in claim 11, wherein the threshold value is formed by multiplying the overall weighting by a tuning factor.

19. The method as claimed in claim 4, wherein the predefined value is 0.3 m.

14. A device for 3D observation and measurement of the lateral environment of a vehicle, comprising a camera for recording a sequence of digital images which is connected to a data processing unit by means of which the images are provided with a time stamp and buffered,

wherein a unit for sensing the vehicle'"'"'s own movement is provided,

wherein a means for selecting image data on the basis of the vehicle'"'"'s own movement in order to form chronologically overlapping stereoscopic image pairs is provided, and which is connected to a unit for stereoscopic image processing, and

only buffered images during whose recording the orientation of the camera has not changed significantly are selected to form image pairs.

15. The device as claimed in claim 14, wherein the unit for sensing the vehicle'"'"'s own movement is a speedometer, wheel speed meter, navigation system or a vehicle movement dynamics system.

16. The device as claimed in claim 14, wherein the camera has a 90°

orientation with respect to the direction of travel.

17. The device as claimed in claim 14, wherein the camera has an orientation of 45°

-135°

with respect to the direction of travel.

18. The device as claimed in claim 14. wherein the camera is a component of an image recording system which is already located in the vehicle.

20. A method for observing and measuring the lateral environment of a vehicle,

wherein a sequence of digital images are recorded by means of a camera, provided with a time stamp and buffered,

wherein the vehicle'"'"'s own movement is sensed in order to select chronologically overlapping stereoscopic image pairs from the buffered images on the basis of this data,

wherein the position and orientation of the camera which are present at the two recording times are determined,

wherein a local 3D depth image is generated on the basis of the image pairing by means of an algorithm for stereoscopic image processing,

wherein the position and orientation of the camera at the recording times are taken into account within the scope of a synthetic stereoscopic geometry,

wherein a sequence of local 3D depth views are accumulated, wherein the image data of the individual local 3D depth views which can be assigned to the same location points in the environment of the vehicle are added to one another,

wherein the image data is subjected to weighting before the addition,

wherein the volume which is represented by means of the accumulated 3D depth views is divided into individual volume elements within the scope of the weighting, and

wherein the respective number of pixels within the individual volume elements is assigned to each of these volume elements as a weighting.