Method and system for video-based road characterization, lane detection and departure prevention

• US 9,401,028 B2
• Filed: 12/17/2013
• Issued: 07/26/2016
• Est. Priority Date: 01/23/2007
• Status: Active Grant

• Current Assignee: Valeo Schalter und Sensoren GmbH (Valeo SE)
• Original Assignee: Valeo Schalter und Sensoren GmbH (Valeo SE)
• Inventors: Kuehnle, Andreas, Boon, Cathy
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): Schwartz, Raphael
• Application Number:
  US14/109,855
• Publication Number:
  US 20140177914A1
• Time in Patent Office: 952 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A method and system for video-based road departure warning for a vehicle on a road, is provided. Road departure warning involves receiving an image of a road in front of the vehicle from a video imager, and detecting one or more road markings in the image corresponding to markings on the road. Then, analyzing the characteristics of an image region beyond the detected markings to determine a rating for drivability of the road corresponding to said image region, and detecting the lateral offset of the vehicle relative to the markings on the road based on the detected road markings. A warning signal is generated as function of said lateral offset and said rating.

Claims

1. A method of video-based road departure warning for a vehicle on a road, comprising the steps of:

receiving an image of a road in front of the vehicle from a video imager;

detecting one or more road markings in the image corresponding to markings on the road;

analyzing characteristics of an image region beyond the detected markings to determine a rating for drivability of the road corresponding to said image region, wherein the characteristics of the image region beyond the detected markings comprise a median gradient spacing row variation;

detecting a lateral offset of the vehicle relative to the markings on the road based on the detected road markings;

generating a warning signal as a function of said lateral offset and said rating; and

wherein the step of analyzing characteristics of the image region beyond the detected markings to determine the rating for drivability further comprises comparing the characteristics of the image region beyond the detected markings to characteristics of an image region within the detected markings or to values in a database.

2. The method of claim 1, wherein the step of detecting one or more road markings in the image includes determining a region of interest in the image and detecting one or more road markings in the region of interest.

3. The method of claim 1, wherein:

the step of detecting one or more road markings in the image further includes determining attributes of the road markings; and

the step of generating a warning signal further includes generating a warning signal as a function of said lateral offset, said rating, and said attributes.

4. The method of claim 3, wherein determining attributes of the road markings further includes determining the type and the number of the detected road markings.

5. The method of claim 1, wherein detecting road markings further includes detecting image features representing characters on the road, and removing such image features from consideration in detecting road markings.

6. The method of claim 5, further including the step of tracking the detected road markings.

7. The method of claim 1, further including selecting a set of road markings among the detected road markings based on relevancy of the road markings.

8. The method of claim 7, further including the step of tracking the selected road markings.

9. The method of claim 1, wherein detecting road markings further includes detecting clutter in the image, and eliminating the detected clutter in detecting road markings.

26. The method of claim 1, wherein the comparison of characteristics is measured by a Manhattan distance, such that:

Manhattan distance=|f1_a−

f1_b|+|f2_a2_b|+|f3_a−

f3_b|,

where;

f1_a is a first characteristic for a group a,

f1_b is a first characteristic for a group b,

f2_a is a second characteristic for the group a,

f2_b is a second characteristic for the group b,

f3_a is a third characteristic for the group a, and

f3_b is a third characteristic for the group b.

29. The method of claim 1, wherein the step of comparing the characteristics of the image region beyond the detected markings to characteristics of an image region within the detected markings or to values in a database further comprises an average mean absolute deviation (MAD) ratio.

10. An apparatus for video-based road departure warning for a vehicle on a road, comprising:

an image processing function configured for receiving an image of a road in front of a vehicle from a video imager;

a road marking detector configured for detecting one or more road markings in the image corresponding to markings on the road using an image processing device;

a characterization module configured for analyzing characteristics of an image region beyond the detected markings to determine a rating for drivability of the road corresponding to said image region, wherein the characteristics of the image region beyond the detected markings comprise a median gradient spacing row variation;

an offset detection function configured for detecting a lateral offset of the vehicle relative to the markings on the road based on the detected road markings,

a lane departure detector configured for generating a warning signal as a function of said lateral offset and said rating;

a controller configured for controlling operation of one or more of the image processing function, road marking detector, characterization module, offset detection function and lane departure detector; and

wherein the characterization module configured for analyzing characteristics of the image region beyond the detected markings to determine the rating for drivability is further configured for comparing the characteristics of the image region beyond the detected markings to characteristics of an image region within the detected markings or to values in a database.

11. The apparatus of claim 10, wherein the road marking detector is further configured for determining a region of interest in the image and detecting one or more road markings in the region of interest.

12. The apparatus of claim 10, further including a marking attribute function configured for determining attributes of the road markings;

and

wherein the lane departure detector is further configured for generating a warning signal as a function of said lateral offset, said rating, and said attributes.

13. The apparatus of claim 12, wherein the marking attribute function is further configured for determining the type and the number of the detected road markings.

14. The apparatus of claim 10, further including a character detection function, configured for detecting image features representing characters on the road, and removing such image features from consideration in detecting road markings.

15. The apparatus of claim 14, further including a tracking module configured for tracking the detected road markings in one or more road image frames.

16. The apparatus of claim 10, further including a marking selection function configured for selecting a set of road markings among the detected road markings based on relevancy of the road markings.

17. The apparatus of claim 16, further including a tracking module configured for tracking the selected road markings in one or more road image frames.

18. The apparatus of claim 10, further including a clutter detector, configured for detecting clutter in the image, and eliminating the detected clutter in detecting road markings.

19. The apparatus of claim 10, further including a reflection detector, configured for detecting road surface reflection features in the image, and eliminating the reflection features from road marking detection.

20. The apparatus of claim 10, further including a tar strip detector, configured for detecting road surface tar strip features in the image, and eliminating the tar strip features from road marking detection.

27. The apparatus of claim 10, wherein the characterization module configured for comparing the characteristics of the image region beyond the detected markings to characteristics of an image region within the detected markings or to values in a database is further configured for measuring the comparison of the characteristics by a Manhattan distance, such that:

Manhattan distance=|f1_a−

f1_b|+|f2_a2_b|+|f3_a−

f3_b|,

where;

f1_a is a first characteristic for a group a,

f1_b is a first characteristic for a group b,

f2_a is a second characteristic for the group a,

f2_b is a second characteristic for the group b,

f3_a is a third characteristic for the group a, and

f3_b is a third characteristic for the group b.

30. The apparatus of claim 10, wherein the characterization module configured for comparing the characteristics of the image region beyond the detected markings to characteristics of an image region within the detected markings or to values in a database is further configured for analyzing an average mean absolute deviation (MAD) ratio.

21. A system for video-based road departure warning for a vehicle on a road, comprising:

a video imager configured for generating an image of a road in front of a vehicle;

an image processing function configured for determining a region of interest in the image using an image processing device,

a road marking detector configured for detecting one or more road markings in the image corresponding to markings on the road;

a characterization module configured for analyzing characteristics of a image region beyond the detected markings, to determine a rating for drivability of the road corresponding to said image region, wherein the characteristics of the image region beyond the detected markings comprise a median gradient spacing row variation;

an offset detection function configured for detecting a lateral offset of the vehicle relative to the markings on the road based on the detected road markings;

a lane departure detector configured for generating a warning signal as a function of said lateral offset and said rating;

a controller configured for controlling operation of one or more of the imager, image processing function, road marking detector, characterization module, offset detection function and lane departure detector; and

wherein the characterization module configured for analyzing characteristics of the image region beyond the detected markings to determine the rating for drivability is further configured for comparing the characteristics of the image region beyond the detected markings to characteristics of an image region within the detected markings or to values in a database.

22. The system of claim 21, further including a marking attribute function configured for determining attributes of the road markings;

and

wherein the lane departure detector is further configured for generating a warning signal as a function of said lateral offset, and said rating and said attributes.

23. The system of claim 22, wherein the marking attribute function is further configured for determining the type and the number of the detected road markings.

24. The system of claim 21, further including a tracking module configured for tracking the detected road markings in one or more road image frames.

25. The system of claim 21, further including a marking selection function configured for selecting a set of road markings among the detected road markings based on relevancy of the road markings, and a tracking module configured for tracking the selected road markings in one or more road image frames.

28. The system of claim 21, wherein the characterization module configured for comparing the characteristics of the image region beyond the detected markings to characteristics of an image region within the detected markings or to values in a database is further configured for measuring the comparison of the characteristics by a Manhattan distance, such that:

Manhattan distance=|f1_a−

f1_b|+|f2_a2_b|+|f3_a−

f3_b|,

where;

f1_a is a first characteristic for a group a,

f1_b is a first characteristic for a group b,

f2_a is a second characteristic for the group a,

f2_b is a second characteristic for the group b,

f3_a is a third characteristic for the group a, and

f3_b is a third characteristic for the group b.

31. The system of claim 21, wherein the characterization module configured for comparing the characteristics of the image region beyond the detected markings to characteristics of an image region within the detected markings or to values in a database is further configured for analyzing an average mean absolute deviation (MAD) ratio.