Method for stereo image object detection

• US 5,937,079 A
• Filed: 09/05/1997
• Issued: 08/10/1999
• Est. Priority Date: 09/05/1996
• Status: Expired due to Fees

• Current Assignee: Daimler Chrysler Company LLC (Stellantis NV)
• Original Assignee: Daimler-Benz AG (Mercedes-Benz Group AG)
• Inventors: Franke, Uwe
• Primary Examiner(s): Chang, Jon
• Assistant Examiner(s): Wu, Jingge
• Application Number:
  US08/923,937
• Time in Patent Office: 704 Days
• Field of Search: 382/103, 382/106, 382/107, 382/154, 348/42, 348/43, 348/47, 356/12
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0085 Motion estimation from stereoscopic image signals

Abstract

In a method for detecting and tracking objects by stereo image evaluation a part of structure class images is initially generated from a recorded stereo image pair. Differences in brightness of selected pixels in the environment are determined for each pixel as digital values, which are combined to form a digital value group, with identical groups defining their own structure classes. Structure classes which lack a brightness change along the epipolar line are discarded. Corresponding disparity values are then determined for the pixels in the other structure classes and are collected in a disparity histogram with a given frequency increment. The pixel group that belongs to a given grouping point area of the histogram is then interpreted as an object to be detected.

Claims

1. Method for stereo image object detection, comprising the following steps:

first, recording at least one stereo image pair for an area of interest;

second, generating a structure class image pair from a respective recorded stereo image pair, by the steps of

for each pixel in each recorded image of said recorded stereo image pair, determining digital values representative of differences between a brightness value for such pixel and brightness values for a plurality of predetermined ambient pixels; and

conjoining the resultant determined digital values in a predetermined sequence to form a digital value group, with identical digital value groups defining an independent structure class;

third, performing a correspondence analysis of the structure class image pair in which

for each particular pixel in one structure class image of said structure class image pair, only those structure classes in the other structure class image of said structure class image pair are taken into account which have at least one ambient pixel that lies along an epipolar line corresponding to said particular pixel and has a brightness which differs by one or more brightness digital steps;

for each pixel of a structure class of one structure class image to be taken into account, pixels that lie within a predetermined disparity interval on the epipolar line and have the same structure class are searched, and a corresponding disparity value is determined; and

disparity values thus obtained corresponding to an assigned frequency increment are combined in a disparity histogram; and

fourth, identifying frequency point areas in the resultant disparity histogram and extracting an individual object from the pixel group that belongs to a particular grouping point range.

2. Method according to claim 1, wherein for digital value determination in said second step, a ternary system is used, wherein brightness differences which are smaller than or equal to a predetermined positive brightness threshold value are assigned a first ternary number, brightness differences that are greater than the threshold value are assigned a second ternary number, and the brightness differences that are smaller than the negative of the threshold value being assigned a third of the three ternary number.

3. Method according to claim 1, wherein for each pixel, two pixels are used as ambient pixels, which are located on opposite sides of the respective reference pixel, firstly parallel to and then perpendicular to the epipolar line, said pixels being located immediately adjacent to the reference pixel or separated from the latter by a predeterminable sampling width.

4. Method according to claim 1, wherein the predetermined disparity interval for the correspondence analysis is chosen so that for a given pixel in a structure class image, only the closest pixel on the epipolar line in another structure class image is taken into account, and the disparity value for such closest pixel is added to the disparity histogram.

5. Method according to claim 1, wherein for each pixel of the structure class image pair, in addition to the respective digital value group, a contrast value determined from the brightness differences is assigned as a weight factor, and the disparity value determined for a given pixel pair with a frequency increment weighted by the corresponding contrast values is added to the disparity histogram.

6. Method according to claim 1, wherein the extraction of an object from a pixel group that belongs to a given disparity grouping point area comprises a cluster analysis of the pixel group with which a rectangle of minimum area is determined that contains a predeterminable percentage of the pixels of this pixel group.

7. Method according to claim 1, wherein following initial extraction of an object from the pixel group that belongs to a given disparity grouping point range, movement of said object is tracked by repeated performance of said second, third and fourth steps limited to a range of a minimum cube that contains the object in question.

8. Method according to claim 1, for detection of objects in traffic, wherein assuming an essentially plane street pattern and parallel optical stereo camera axes, those pixels whose associated disparity values lie within a predetermined expectation range are located around a disparity expectation value d_E that satisfies the following condition:

space="preserve" listing-type="equation">d.sub.E (x,y)=(B,H)f.sub.x ( (Y/f.sub.y)cosα

+sinα

),

where B is the base width of the stereo camera system, H is the height of the latter above the road surface, and α

is its tilt angle relative to the road surface, f_x and f_y are the lateral and vertical camera focal lengths and y is the image vertical coordinate pointing downward from the optical axis, interpreted as belonging to an object on the road surface.