Method of detecting objects within a wide range of a road vehicle

• US 7,046,822 B1
• Filed: 06/09/2000
• Issued: 05/16/2006
• Est. Priority Date: 06/11/1999
• Status: Expired due to Term

• Current Assignee: OL Security LLC (Intellectual Ventures LLC)
• Original Assignee: DaimlerChrysler AG (Mercedes-Benz Group AG)
• Inventors: Regensburger, Uwe, Knoeppel, Carsten
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US10/009,596
• Time in Patent Office: 2,167 Days
• Field of Search: 382/100, 382/103, 701/210, 348/169
• US Class Current: 382/103
• CPC Class Codes: B60W 30/16 Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/307 using fly-eye lenses, e.g. arrangements of circular lenses
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0092 Image segmentation from stereoscopic image signals

Abstract

The invention relates to a method of detecting objects within a wide range of a road vehicle. According to said method, the distance between a moving or stationary vehicle and one or more objects is calculated by distance-based image segmentation using stereoscopic image processing techniques and the properties of the detected objects are determined by object recognition in the segmented image areas. Image areas of three-dimensional and/or flat objects are detected and said three-dimensional and/or flat objects are detected by clustering 3D pixels according to defined criteria. Three-dimensional objects are determined by features with similar distance values and flat objects by features with similar height values.

Claims

1. A method of detecting objects in a vicinity of a road vehicle up to a considerable distance, in which a distance from a moving or stationary vehicle to one or more objects is calculated by distance-based image segmentation using stereo image processing, and characteristics of the detected objects are determined by object recognition in the segmented image regions, the method comprising the acts of:

determining image regions of elevated objects and/or flat objects;

detecting elevated objects and/or flat objects by combining 3D points in accordance with predetermined criteria, the elevated objects being determined through features with similar distance values and the flat objects being determined through features with similar height values;

tracking over time relevant detected objects and determining the distance and lateral position of the relevant detected objects relative to the road vehicle in order to assess dynamic behavior of the relevant detected objects;

determining object hypothesis for performing object recognition, said object hypothesis being verified by comparison with object models;

scanning segmented image regions in accordance with predetermined, statistically verified 2D features of particular relevant detected objects to be

comparing the particular relevant detected objects using a neural network for classifying a specific object type;

scanning one of recorded pairs of stereo images for significant features of objects to be registered; and

determining a spacing of at least one object'"'"'s significant features by comparing respective features in a stereo image from a pair of stereo images with the same, corresponding features, in the other stereo image from the pair of stereo images recorded at the same time;

wherein disparities that occur are evaluated via cross correlation techniques.

2. The method according to claim 1, wherein elevated relevant detected objects are road vehicles and flat relevant detected objects are road markings and boundaries.

3. The method according to claim 2, further comprising the act of determining a relative position and a relative speed of the relevant detected objects relative to one another and to the road vehicle by evaluating a distance measurement, in order to determine an accurate road-lane object association.

4. The method according to claim 3, wherein the relative position and the relative speed of the relevant detected objects are determined in order to assess a relevance of the detected objects to a particular situation.

5. The method according to claim 1, further comprising the act of determining a relative position and a relative speed of the relevant detected objects relative to one another and to the road vehicle by evaluating a distance measurement, in order to determine an accurate road-lane object association.

6. The method according to claim 5, wherein the relative position and the relative speed of the relevant detected objects are determined in order to assess a relevance of the detected objects to a particular situation.

7. The method according to claim 1, wherein by determining the spacing of the significant features in a pixel range, 3D points in the road vehicle environment are determined relative to a coordinate system of a measuring device performing the detecting method.

8. The method according to claim 1, wherein said objects are detected by at least one of radar, infrared sensing, and stereoscopic or mono sensing.

9. A method of detecting and recognizing an object in a vicinity of a road vehicle, the method comprising the acts of:

performing distance-based image segmentation to calculate a distance from the road vehicle to an object to be detected;

scanning the segmented image regions in accordance with predetermined, statistically verified 2D features of the object to be detected;

comparing the detected object using a neural network for classifying it as a specific object type,

scanning one of recorded pairs of stereo images for significant features of objects to be registered; and

determining a spacing of at least one object'"'"'s significant features by comparing respective features in a stereo image from a pair of stereo images with the same, corresponding features, in the other stereo image from the pair of stereo images recorded at the same time;

wherein disparities that occur are evaluated via cross correlation techniques.