Process for recognition of lane markers using image data

• US 6,978,037 B1
• Filed: 11/01/2000
• Issued: 12/20/2005
• Est. Priority Date: 11/01/2000
• Status: Expired due to Fees

• Current Assignee: Daimler AG (Mercedes-Benz Group AG)
• Original Assignee: Daimler Chrysler Company LLC (Stellantis NV)
• Inventors: Heinrich, Stefan, Fechner, Thomas
• Primary Examiner(s): Couso, Jose L.
• Assistant Examiner(s): Lu, Tom Y.
• Application Number:
  US09/704,366
• Time in Patent Office: 1,875 Days
• Field of Search: 382103-104, 382/106, 382/170, 382/181, 382/190, 382/199, 382/209, 382/108, 382/217, 382/266, 340/435, 340/439, 701/28
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

In the inventive process the morphological characteristics of point-shaped vehicle lane markers are evaluated on the basis of a priori knowledge. For this, those estimation processes are particularly useful, which are based on a Kalman Filter. After the selection of suitable ROI a Matched Filter is employed, in order to better extract from the background the image points associated with a vehicle lane marker. The inventive process is obviously not limited to point-shaped vehicle lane markers.

Claims

1. A process for recognition of vehicle lane markings from image data, comprising:

evaluating morphological characteristics of vehicle lane markings using a priori knowledge, and

using a matched-filter in order to extract image points, which are associated with vehicle lane markers, by measuring the average gray value of the background in the environment of the position to be examined, and evaluating an image point, which is potentially to be associated with the vehicle lane marking, on the basis of a comparison between background noise, the average gray value in the environment, and a gray value of the position to be examined.

2. The process according to claim 1, wherein from the image data, areas are extracted for processing, in which vehicle lane markings are contained with high probability based upon a priori knowledge.

3. The process according to claim 2, wherein the a priori knowledge is based upon at least one of:

a camera geometry,

a geometry of the vehicle track,

dimensions of the vehicle lane markings and

a vehicle position.

4. The process according to claim 2, wherein in initialization of the process for recognition of vehicle lane markings model parameters are varied at random sequence so long until vehicle lane markings are found.

5. The process according to claim 4, wherein the model parameter draws upon at least one of:

the width of the vehicle lane,

orientation of the camera with respect to the center of the vehicle lane and

the yaw angle of the vehicle.

6. The process according to claim 2, wherein for repositioning of already initialized ROI parameter predictions, a vehicle street model based on a prediction of an evaluation process is drawn upon for parameter determination.

7. The process according to claim 6, wherein the evaluation process for parameter determination is based upon a Kalman-filter.

8. The process according to claim 6, wherein in the repositioning of the ROI, its values are controlled by variation of the result values of the prediction of the Kalman-filter, when the width is adapted proportionally to the size of the variation of the results.

9. The process according to claim 2, wherein the ROI is limited vertically on the basis of a minimal and a maximal distance in the street plane.

10. The process according to claim 9, wherein in particular in application of the process at night, the vertical limitation of the ROI is determined by the area of the maximal illumination (high beam, low beam).

11. The process according to claim 9, wherein the limitation is controlled by the number of the image points expected to be associated with the vehicle lane marker, and this control or regulation is optimal when the number of the image points to be expected is constant for all distance ranges.

12. The process according to claim 1, wherein the matched-filter is adapted in shape and size to the vehicle lane marking being searched for and/or to the statistic of the background.

13. The process according to claim 1, wherein the matched-filter is implemented in separate form, in which x-y-components are presented separately.

14. The process according to claim 1, wherein in the evaluation of the matched-filter, only the x-component is evaluated.

15. The process according to claim 1, wherein after the extraction of the image points which are to be associated with vehicle lane markings, these are digitized, wherein the intensities of the individual image points are compared with a threshold value, and the image points are only then drawn upon for further evaluation when their intensity exceeds this threshold.

16. The process according to claim 15, wherein the threshold value is determined from background noise using a threshold value regulator or controller.

17. The process according to claim 1, wherein after the extraction of image points potentially belonging to a vehicle lane marker and subsequent digitization, these image points are collected for the further processing into marker objects.

18. The process according to claim 1, wherein in the evaluation of the morphological characteristics of the vehicle lane marker, at least one of:

the size of the marking object, group of image points

the roundness of the image point group and

the distribution of the image points or the number of empty spaces within the image point group is evaluated with respect to whether they satisfy criteria of a vehicle lane marker defined in accordance with a priori knowledge.

19. The process according to claim 18, wherein each image point group, which satisfies the criteria of a vehicle lane marker, is considered to be an actual marker object and is characterized by its image coordinates.

20. The process according to claim 19, wherein as characterizing image coordinates, the coordinates of the center of gravity of the image point group associated with the marking object is selected.

21. The process according to claim 17, wherein the characteristic image coordinates of the marking object are employed in order with curve regression to describe the boundaries of the own vehicle lane with respect to the course of the vehicle track, as well as to describe the own position with respect to the vehicle lane center, and that this description is provided to an estimation process for parameter determination for repositioning of the ROI within the image data.

22. A process for recognition of vehicle lane markings from image data, comprising:

evaluating morphological characteristics of vehicle lane markings using a priori knowledge,

using a matched-filter in order to extract image points, which are associated with vehicle lane markers, by measuring the average gray value of the background in the environment of the position to be examined, and evaluating an image point, which is potentially to be associated with the vehicle lane marking, on the basis of a comparison between background noise, the average gray value in the environment, and a gray value of the position to be examined,

wherein after the extraction of the image points, which are to be associated with vehicle lane markings, these are digitized,

wherein the intensities of the individual image points are compared with a threshold value, and the image points are only then drawn upon for further evaluation when their intensity exceeds this threshold,

wherein the threshold value is determined from background noise using a threshold value regulator or controller, and

wherein the threshold value regulator draws upon a priori knowledge regarding the expected surfaces occupied by the vehicle lane markings, which are directly correlated with the expected number of image points associated with the vehicle lane markings, and wherein the threshold value regulator or controller thereupon aims to supply the number of the image points extracted in the ROI preferably exactly to this expected value.