METHOD AND SYSTEM FOR UNIVERSAL LANE BOUNDARY DETECTION

• US 20100054538A1
• Filed: 01/22/2008
• Published: 03/04/2010
• Est. Priority Date: 01/23/2007
• Status: Active Grant

• Current Assignee: Valeo Schalter und Sensoren GmbH (Valeo SE)
• Original Assignee: Valeo Schalter und Sensoren GmbH (Valeo SE)
• Inventors: Boon, Cathy
• Granted Patent: US 8,462,988 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A method and system for detecting road markings, is provided. One implementation involves receiving an image (3) of a road (4) in front of a vehicle (2) from an imager (1), determining a region of interest in an identified road in the image, and detecting road markings (8L1 8R) by detecting lane markings (7L, 7R) in the region of interest, and if lane markings cannot be detected, then detecting road boundaries in the region of interest.

Claims

1. A method of detecting road or lane identifiers, comprising the steps of:

receiving an image of a road in front of a vehicle from an imager;

determining a region of interest in an identified road in the image; and

detecting road markings by detecting lane markings in the region of interest, and if lane markings cannot be detected, then detecting road boundaries in the region of interest.

2. The method of claim 1 wherein detecting road markings includes:

determining luminance gradients in the region of interest; and

detecting road markings by comparing the luminance gradients to a strength threshold.

3. The method of claim 2 wherein:

determining luminance gradients includes determining edge strength in the region of interest to identify potential road marking edges; and

detecting road markings includes comparing edge strength to said strength threshold by determining peaks and valleys in luminance gradients of image pixels, wherein a gradient peak represents an edge in the image, and a gradient valley represents an edge in the image.

4. The method of claim 3 wherein detecting road markings further includes:

for multiple image rows in the region of interest;

identifying a pair of edges as potential road marking edges by comparing each edge strength to said edge strength threshold;

determining spacing between the pair of edges; and

comparing said spacing to spacing minimum and maximum thresholds;

wherein if the edge pair spacing is between the minimum and maximum thresholds, then the edge pair represents a potential lane marking;

fitting a first line to the center gradient locations corresponding to edge pairs in said rows representing potential lane marking; and

selecting all edge pairs that contributed to an acceptable first approximate line fit, as lane marking identifiers.

5. The method of claim 4 further including determining an acceptable first fit line by checking how well the first line fit fits said edge pairs, and similarity of the first line fit slope to that found in previous road images.

6. The method of claim 4 wherein detecting road markings further includes:

if said edge pair spacing is not between the minimum and maximum thresholds, or an acceptable line fit was not found, then fitting a second line to individual edges in said rows representing potential road boundary identifiers; and

selecting all edges that contributed to an acceptable second approximate line fit, as road boundary identifiers.

7. The method of claim 6 further including determining an acceptable second fit line by checking how well the second line fit fits said edges, and similarity of the second line fit slope to that found in previous road images.

8. The method of claim 6 wherein if no road boundary identifiers were found, then determining mean absolute deviation (MAD) of gradients in said rows, fitting a third line to locations of MAD gradients whose magnitude exceeds a magnitude threshold individual edges in said rows representing potential road markings, selecting MAD locations that contributed to an acceptable third approximate line fit, as road markings.

9. The method of claim 8 further including determining an acceptable third fit line by checking how well the second line fit fits said MAD gradients, and similarity of the third line fit slope to that found in previous road images.

10. An apparatus for detecting road markings, comprising:

an image processor configured for receiving an image of a road in front of a vehicle from an imager, determining a region of interest in an identified road in the image; and

a road marking detection module configured for detecting lane markings in the region of interest, and if lane markings cannot be detected, then detecting road boundaries in the region of interest.

11. The apparatus of claim 10 wherein:

the image processor is further configured for determining luminance gradients in the region of interest; and

the road marking detection module is further configured for detecting road markings by comparing the luminance gradients to a strength threshold.

12. The apparatus of claim 11 wherein:

the image processor is further configured for determining luminance gradients by determining edge strength in the region of interest to identify potential road marking edges; and

the road marking detection module is further configured for comparing edge strength to said strength threshold by determining peaks and valleys in luminance gradients of image pixels, wherein a gradient peak represents an edge in the image, and a gradient valley represents an edge in the image.

13. The apparatus of claim 12 wherein the road marking detection module is further configured such that:

in multiple image rows in the region of interest, the road marking detection module performs;

identifying a pair of edges as potential road marking edges by comparing each edge strength to said strength threshold;

determining spacing between the pair of edges; and

comparing said spacing to spacing minimum and maximum thresholds;

wherein if the edge pair spacing is between the minimum and maximum thresholds, then the edge pair represents potential lane marking;

fitting a first line to the center gradient locations corresponding to edge pairs in said rows representing potential lane marking; and

the road marking detection module selects all edge pairs that contributed to an acceptable first approximate line fit, as lane marking identifiers.

14. The apparatus of claim 13 wherein the road marking detection module is further configured for determining an acceptable first fit line by checking how well the first line fit fits said edge pairs, and similarity of the first line fit slope to that found in previous road images.

15. The apparatus of claim 13 the road marking detection module is further configured such that if said edge pair spacing is not between the minimum and maximum thresholds, or an acceptable line fit was not found, then the road marking detection module performs fitting a second line to individual edges in said rows representing potential road boundary identifiers, and selecting all edges that contributed to an acceptable second approximate line fit, as road boundary identifiers.

16. The apparatus of claim 15 further wherein the road marking detection module is further configured for determining an acceptable second fit line by checking how well the second line fit fits said edges, and similarity of the second line fit slope to that in a previous road image.

17. The apparatus of claim 15 wherein the road marking detection module is further configured such that if no road boundary identifiers were found, then the road marking detection module performs:

determining gradients of mean absolute deviation (MAD) in said rows, fitting a third line to locations of MAD gradients whose magnitude exceeds a magnitude threshold individual edges in said rows representing potential road markings, selecting MAD locations that contributed to an acceptable third approximate line fit, as road markings.

18. The apparatus of claim 17 wherein the road marking detection module is further configured for determining an acceptable third fit line by checking how well the second line fit fits said MAD gradients, and similarity of the third line fit slope to that found in previous road images.

19. A system for detecting road surface reflections, comprising:

an imager configured for generating an image of a road in front of a vehicle;

an image processor configured for determining a region of interest in an identified road in the image;

a road marking detection module configured for detecting lane markings in the region of interest, and if lane markings cannot be detected, then detecting road boundaries in the region of interest; and

a road departure detection module configured for detecting proximity of the vehicle relative to the detected road markings.

20. The system of claim 19 wherein:

the image processor is further configured for determining luminance gradients by determining edge strength in the region of interest to identify potential road marking edges; and

the road marking detection module is further configured for comparing edge strength to said strength threshold by determining peaks and valleys in luminance gradients of image pixels, wherein a gradient peak represents an edge in the image, and a gradient valley represents an edge in the image.

21. The system of claim 20 wherein the road marking detection module comprises a lane marking detection module configured such that:

in multiple image rows in the region of interest, the lane marking detection module performs;

identifying a pair of edges as potential road marking edges by comparing each edge strength to said strength threshold;

determining spacing between the pair of edges; and

comparing said spacing to spacing minimum and maximum thresholds;

wherein if the edge pair spacing is between the minimum and maximum thresholds, then the edge pair represents potential lane marking;

fitting a first line to the center gradient locations corresponding to edge pairs in said rows representing potential lane marking; and

the road marking detection module selects all edge pairs that contributed to an acceptable first approximate line fit, as lane marking identifiers.

22. The system of claim 21, wherein the road marking detection module further comprises a road boundary detection module configured such that if said edge pair spacing is not between the minimum and maximum thresholds, or an acceptable line fit was not found, then the a road boundary detection module performs fitting a second line to individual edges in said rows representing potential road boundary identifiers, and selecting all edges that contributed to an acceptable second approximate line fit, as road boundary identifiers.

23. The system of claim 22 wherein the road marking detection module is further configured such that if no road boundary identifiers were found, then the road marking detection module performs:

determining gradients of the mean absolute deviation (MAD) in said rows, fitting a third line to locations of MAD gradients whose magnitude exceeds a magnitude threshold individual edges in said rows representing potential road markings, selecting MAD locations that contributed to an acceptable third approximate line fit, as road markings.

24. The system of claim 23 wherein the road marking detection module is further configured for determining an acceptable line fit line by checking how well the line fit fits said edges, and similarity of the line fit slope to that found in previous road images.

25. The system of claim 24 wherein the image processing module is further configured for selectively determining luminance gradients based on an input indicating daytime operation or nighttime operation.