System of detecting road white line, method for detecting road white line and storage medium storing program for detecting road white line

• US 6,829,388 B1
• Filed: 09/05/2000
• Issued: 12/07/2004
• Est. Priority Date: 09/03/1999
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Sakurai, Kazuyuki
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): DESIRE, GREGORY M
• Application Number:
  US09/655,765
• Time in Patent Office: 1,554 Days
• Field of Search: 382/105, 382/104, 382/181, 382/190, 382/199, 382/202-203, 382/266, 382/274-275, 348/119, 348/118, 348/142-143, 348/148, 340/435, 340/937
• US Class Current: 382/199
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A system and method are provided which are capable of reliably and effectively detecting a white line in an upper region of a road image by dividing the road image into, at least, two regions positioned up and down and using a result from a detection of the white line in a lowest region where the road image is less affected by noise. When a road surface is planar, both intersections of straight lines, by which left and right white lines are approximated, in a lowest region out of regions, positioned up and down, obtained by dividing a road image containing left and right white lines drawn in parallel on the road surface and intersections of straight lines, by which left and right white lines are approximated in an upper region of out of the two regions existing on one horizontal line. The left and right first straight lines, by which the left and right white lines are approximated, are detected in the lowest region and a horizontal line is calculated from a first vanishing point being the intersection of the left and right straight lines, and based on a characteristic that the intersection of the left and right straight lines, by which the left and right white lines are approximated, are approximated in the upper region, existing on the horizontal line, the intersection is detected and straight lines, by which the left and right white lines are approximated, is obtained in the upper region.

Claims

1. A road marking line detecting system for processing a road image to detect a left side edge marking line and a right side edge marking line of a traveling lane in said road image, said system comprising:

first means for detecting first left and right straight lines by which the left and the right marking lines are approximated in a lowest region of at least two regions positioned up and down on said road image, the regions obtained by dividing said road image;

second means for calculating a horizontal line in a region other than said lowest region in said road image from a first vanishing point, the first vanishing point being an intersection of said first left and right straight lines; and

third means for detecting second straight lines, by which the left and the right marking lines are approximated in a region other than said lowest region, by detecting an intersection of said second left and right straight lines on the horizontal line.

2. The road marking line detecting system according to claim 1, wherein said third means comprises means for detecting a second vanishing point on said horizontal line in a region adjacent to said lowest region in said road image, based on intercepts at lowest ends of the second left and right straight lines on said first left and right straight lines, said second vanishing point being the intersection of said second left and right straight lines on said horizontal line.

3. The road white line detecting system according to claim 2, wherein said third means comprises means for predicting an intersection of third left and right straight lines in an upper region other than said lowest region and not adjacent to said lowest region in said road image, based on an amount of displacement on said horizontal line between intersections of straight lines by which left and right marking lines in two or more regions below said upper region are approximated.

4. The road marking line detecting system according to claim 3, further comprising a means for extracting, from said road image, a sequence of points existing on the left and the right marking lines as a sequence of marking line featuring points, together with a marking line filter value showing similarity to said marking line featuring point and wherein said first means comprises a means for detecting straight lines by which said sequence of marking line featuring points is approximated using a least squares method, with weight assigned to said marking line filter value, for left and right sides, as line approximation of marking lines and wherein said third means comprises means for detecting an intersection, on said horizontal line, of left and right second straight lines by which said sequence of marking line featuring points in said region being adjacent to said lowest region is approximated using said least squares method, for both left and right sides, with weight assigned to said marking line filter value and using upper end points, in the lowest region, of said left and right first straight lines as intercepts at lowest ends in said region being adjacent to said lowest region.

5. A road marking line detecting method for processing a road image to detect a left edge marking line and a right edge marking line of a travelling lane in said road image, said method comprising:

detecting first left and right straight lines, by which the left marking line and the right marking line are approximated in a lowest region of at least two regions positioned up and down on said road image, the regions obtained by dividing said road image;

calculating a horizontal line from a first vanishing point, the first vanishing point being an intersection of said first left and right straight lines; and

detecting second left and right straight lines, by which the left and right marking lines are approximated in an upper region, the upper region being a region other than said lowest region, by detecting an intersection of the second left and right straight lines on said horizontal line.

6. The road marking line detecting method according to claim 5, wherein said detecting second left and right straight lines comprises detecting a second vanishing point on said horizontal line based on a characteristic that at least in an upper region adjacent to said lowest region, intercepts at lowest ends of second left and right straight lines exist on said first left and right straight lines and that said second vanishing point is the intersection of said second left and right straight lines on said horizontal line, the left and the right marking lines in said region adjacent to said lowest region being approximated based on the second left and right straight lines.

7. The road marking line detecting method according to claim 6, wherein said detecting second left and right straight lines comprises a step of predicting an intersection of straight lines by which the left and the right marking lines are approximated, in an upper region other than said lowest region in said road image and not being adjacent to said lowest region, based on an amount of displacement, on said horizontal line, between intersections of straight lines by which the left and the right marking lines in two or more regions below said upper region are approximated.

8. The road marking line detecting method according to claim 7, further comprising a step of extracting a sequence of points existing on the left and the right marking line image, from said road image, together with a marking line filter value showing similarity to said marking line featuring points as a sequence of marking line featuring points, and wherein said detecting first left and right straight lines comprises a step of detecting straight lines by which said sequence of marking line featuring points is approximated using said least squares method, with weight assigned to said marking line filter value, as line approximation for left and right sides and wherein said detecting second left and right straight lines comprises a step of detecting an intersection, on said horizontal line, of left and right second straight lines by which said sequence of marking line featuring points in said region being adjacent to said lowest region is approximated by using said least squares method, for both left and right sides, with weight assigned to said marking line filter value, and using upper end points of said left and right first straight lines in said lowest region as intercepts at their lowest ends in said region being adjacent to said lowest region.

9. A machine readable storage medium storing a program for processing a road image to detect a left edge road marking line and a right edge road marking line of a travelling lane in the road image said program configured to cause a computer to carry out:

(a) detecting first left and right straight lines by which the left marking line and the right marking line are approximated in a lowest region of at least two regions positioned up and down on a road image, the regions obtained by dividing said road image;

(b) calculating a horizontal line from a first vanishing point, the first vanishing point being an intersection of said first left and right straight lines; and

(c) detecting second straight lines by which the left and the right marking lines are approximated in a region other than said lowest region, by detecting an intersection on said horizontal line of the second left and right straight lines by which the left and the right marking lines are approximated in said region other than said lowest region in said road image.

10. The storage medium storing the program for detecting the road marking line according to claim 9, wherein said step (c) comprises detecting a second vanishing point on said horizontal line based on a characteristic that, at least, in a region adjacent to said lowest region of regions other than said lowest region in said road image, intercepts at lowest ends of second left and right straight lines, by which the left and the right marking lines in said region adjacent to said lowest region are approximated, exist on said first left and right straight lines and that said second vanishing point is an intersection on said horizontal line of said second left and right straight lines.

11. The storage medium storing the program for detecting the road marking line according to claim 10, wherein said step (c) comprises a step of predicting an intersection of straight lines by which the left and the right marking lines are approximated, in an upper region other than said lowest region in said road image and not being adjacent to said lowest region, based on an amount of displacement, on said horizontal line, between intersections of straight lines by which the left and the right marking lines in two or more regions below said upper regions are approximated.

12. The storage medium storing the program for detecting the road marking line according to claim 10, wherein said program comprises a program to cause said computer to carry out a step of extracting, from said road image, a sequence of points existing on the left and the right marking lines as a sequence of marking line featuring points, together with a marking line filter value showing similarity to said marking line featuring points and wherein said step (a) comprises process of detecting straight lines by which said sequence of marking line featuring points is approximated using a least squares method, with weight assigned to said marking line filter value, for left and right sides, as line approximation of marking lines and wherein said step (c) comprises a process of detecting an intersection, on said horizontal line, of left and right second straight lines by which said sequence of marking line featuring points in said region being adjacent to said lowest region is approximated, using said least squares method, for both left and right sides, with weight assigned to said marking line filter value and using upper end points, in said lowest region, of said left and right first straight lines as intercepts at their lowest ends in said region being adjacent to said lowest region.

13. A road marking line detecting system for processing a road image to detect a left edge marking line and a right edge marking line of a travelling lane in said road image, said system comprising a computer and a program causing said computer to carry:

(a) detecting first left and right straight lines, by which the left marking line and the right marking line is approximated in a lowest region of at least two regions positioned up and down in said road image obtained by dividing said road image;

(b) calculating a horizontal line from a first vanishing point, the first vanishing point being an intersection of said first left and right straight lines; and

(c) detecting second straight lines by which the left and the right marking lines are approximated in a region other than said lowest region, by detecting on said horizontal line an intersection of the second left and right straight lines, by which the left and the right marking lines are approximated in said region other than said lowest region in said road image.

14. The road marking line detecting system according to claim 13, wherein said step (c) comprises a step of detecting a second vanishing point on said horizontal line based on a characteristic that, at least, in a region being adjacent to said lowest region out of regions other than said lowest region in said road image, intercepts at lowest ends of left and right second straight lines by which the left and the right marking lines in said region being adjacent to said lowest region are approximated, exist on said left and right first straight lines and that said second vanishing point being an intersection of said left and right second straight lines exists on said horizontal line.

15. The road marking line detecting system according to claim 13, wherein said step (c) comprises a step of predicting an intersection of straight lines by which the left and the right marking lines are approximated, in an upper region other than said lowest region in said road image and not being adjacent to said lowest region, based on an amount of displacement, on said horizontal line, between intersections of straight lines by which the left and the right marking lines in two or more regions below said upper regions are approximated.

16. The road marking line detecting system according to claim 13, wherein said program comprises a program to cause said computer to carry out a step of extracting, from said road image, a sequence of points existing on left and right marking lines as a sequence of marking line featuring points, together with a marking line filter value showing similarity to said marking line featuring points and wherein said step (a) comprises a process of detecting straight lines by which said sequence of marking line featuring points is approximated using a least squares method, with weight assigned to said marking line filter value, for left and right sides, as line approximation of marking lines and wherein said step (c) comprises a process of detecting an intersection, on said horizontal line, of left and right second straight lines by which said sequence of marking line featuring points in said region being adjacent to said lowest region is approximated, using said least squares method, for both left and right sides, with weight assigned to said marking line filter value and using upper end points, in said lowest region, of said left and right first straight lines as intercepts at their lowest ends in said region being adjacent to said lowest region.

17. A road marking line detecting system for processing a road image to detect a virtual left edge marking line and a virtual right edge marking line indicating a travelling lane in said road image, said system including a computer and a program causing said computer to carry out:

(a) detecting first left and right first straight lines by which left and right virtual marking lines are approximated in a lowest region of at least two regions positioned up and down on a road image obtained by dividing said road image;

(b) calculating a horizontal line from a first vanishing point, the vanishing point being an intersection of said first left and right straight lines; and

(c) detecting second left and right straight lines by which the left and the right virtual marking lines are approximated in a region other than said lowest region, by detecting on said horizontal line an intersection of the second left and right straight lines, by which the left and the right virtual marking lines are approximated in said region other than said lowest region in said road image.

18. The road marking line detecting system according to claim 17, wherein said step (c) comprises detecting a second vanishing point on said horizontal line based on a characteristic that, at least, in a region adjacent to said lowest region of regions other than said lowest region in said road image, intercepts at lowest ends of the second left and right straight lines, by which the left and the right virtual marking lines in said region being adjacent to said lowest region are approximated, exist on said first left and right straight lines, said second vanishing point being the intersection of said second left and right second straight lines on said horizontal line.

19. The road marking line detecting system according to claim 17, wherein said step (c) comprises a step of predicting an intersection of straight lines by which the left and the right virtual marking lines are approximated, in an upper region other than said lowest region in said road image and not being adjacent to said lowest region, based on an amount of displacement, on said horizontal line, between intersections of straight lines by which the left and the right virtual marking lines in two or more regions below said upper regions are approximated.

20. A road image processor to detect marking lines based on a road image for determining a travelling lane, said processor comprising:

a featuring point extractor configured to construct first left and right straight lines, by which a left marking line and a right marking line are approximated in a lowest region of at least two regions on the road image, the regions obtained by dividing the road image into the lowest region and at least one upper region;

a horizontal line calculator configured to calculate a horizontal line near an upper bound of an upper region in the road image, based on a first vanishing point, the first vanishing point being an intersection of said first left and right straight lines;

an upper region straight line detector configured to detect second left and right straight lines, by which the left marking line and the right marking lines are approximated in the upper region, based on intercepts at the terminal points near a bound of the lowest region of the first left straight line and the first right straight line and based on an intersection of the second left straight line and the second right straight line on the horizontal line, the intersection being a second vanishing point.

21. The road image processor of claim 20, wherein said upper region straight line detector further comprises a vanishing point detector configured to predict another vanishing point at an intersection of third straight lines by which the left marking line and the right marking line are approximated in an upper region other than an upper region adjacent to said lowest region, based on an amount of displacement on the horizontal line, between intersections of second left and right straight lines and first left and right straight lines.