Lane detection system and method

• US 10,102,434 B2
• Filed: 12/22/2015
• Issued: 10/16/2018
• Est. Priority Date: 12/22/2015
• Status: Active Grant

• Current Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Original Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Inventors: Yang, Lei, Wu, Donghui
• Primary Examiner(s): Moyer, Andrew
• Application Number:
  US14/978,873
• Publication Number:
  US 20170177951A1
• Time in Patent Office: 1,029 Days
• Field of Search: 382103, 382104
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/155 involving morphological operators
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A lane detection system includes a non-volatile memory storing machine-readable instructions and an image processor capable of receiving a road image. The image processor, when executing the machine-readable instructions, is capable of: (i) processing the road image to identify a lane candidate within a lane-existing region of the road image, the lane-existing region having (a) a near subregion including an imaged road region nearer to the vehicle and (b) a far subregion including an imaged road region farther from the vehicle, (ii) verifying the lane candidate as a true lane candidate when a minimum distance between (a) a line fit to a portion of the lane candidate in the near subregion and (b) a predetermined reference point in the road image is less than a neighborhood distance; and (iii) extending the true lane candidate into the far subregion to form a detected lane marker demarcating the lane marker.

Claims

1. A method for detecting a lane within a road image captured by a camera having a field of view in front of a vehicle traveling along a road having a lane marker, comprising the steps of:

identifying, within the road image, a reference point located at a reference-point height, with respect to the bottom of the road image, determined by an angle-of-view of the camera in a plane orthogonal to the road;

adjusting the reference-point height according to a location of a vanishing point in the road image to obtain an adjusted reference-point height;

determining a first height as a first fraction of the adjusted reference-point height;

determining a second height equal to a sum of the adjusted reference point height and a second fraction of a height of the road image, each of the first fraction and second fraction being positive and less than one;

processing the road image to identify a lane candidate within a lane-existing region of the road image, the lane-existing region having (i) a first subregion having the first height with respect to the bottom of the road image, (ii) a second subregion having the second height with respect to the bottom of the road image, and (iii) the reference point;

verifying the lane candidate as a true lane candidate when a minimum distance between (a) a line fit to a portion of the lane candidate in the first subregion and (b) the predetermined reference point is less than a neighborhood distance; and

extending the true lane candidate into the second subregion to form a detected lane marker demarcating the lane marker.

2. The method of claim 1, the step of processing comprising:

identifying a ridge feature within the road image to generate a ridge-feature image; and

identifying, within the ridge-feature image, a plurality of lines each at least partially overlapping the ridge feature.

3. The method of claim 2, the plurality of lines being mutually non-collinear, and further comprising conjoining the plurality of lines to identify the lane candidate.

4. The method of claim 3, the step of conjoining comprising, in a blank image having a plurality of pixels each having (i) the same pixel value and (ii) a pixel coordinate in the blank image:

changing the pixel values of a plurality of lane-candidate pixels, a lane-candidate pixel being a pixel having pixel coordinates corresponding to one of the plurality of lines;

executing a morphological operator on the plurality lane-candidate lane pixels.

5. The method of claim 2, the step of identifying a ridge feature comprising:

for each of a plurality of locations of the road image;

computing a first horizontal image gradient partially based on a first image intensity representative of a first image region entirely to the left of the location;

computing a second horizontal image gradient partially based on a second image intensity representative of a second image region entirely to the right of the location; and

determining whether the location is part of a ridge feature based on the first horizontal image gradient and the second horizontal image gradient.

6. The method of claim 5, further comprising determining, based on a vertical position of the location, a horizontal width of at least one of (a) the first image region, (b) the second image region, and (c) the ridge feature.

7. The method of claim 5, further comprising applying a continuity threshold operation to the ridge feature image to disqualify ridge features not corresponding to a lane marker.

8. The method of claim 7, each ridge feature being formed of a plurality of candidate ridge pixels, the continuity threshold operation having a threshold according to a minimum number of spatially-connected candidate ridge pixels in each candidate ridge feature.

9. The method of claim 2, one of the plurality of lines being fit to a plurality of near control points in the ridge-feature image corresponding to the lane candidate, the near control points being in a near subregion of the ridge-feature image corresponding to the first subregion of the road image.

10. The method of claim 1, the step of extending further comprising, for each true lane candidate,

identifying, in the second subregion, a far control point corresponding to the true lane candidate;

and

fitting a curve to the far control point and a plurality of near control points located in the first subregion and corresponding to the true lane candidate.

23. The method of claim 1, the first fraction being equal to three-quarters, the second fraction being equal to one-sixteenth.

25. The method of claim 4, the morphological operator being a closing operator.

11. A lane detection system for detecting a lane within a road image captured along a road having a lane marker comprising:

a non-volatile memory storing machine-readable instructions; and

an image processor communicatively coupled to the memory, capable of receiving the road image; and

, when executing the machine-readable instructions, capable of;

identifying, within the road image, a reference point located at a reference-point height, with respect to the bottom of the road image, determined by an angle-of-view of the camera in a plane orthogonal to the road;

adjusting the reference-point height according to a location of a vanishing point in the road image to obtain an adjusted reference-point height;

determining a first height as a first fraction of the adjusted reference-point height;

determining a second height equal to a sum of the adjusted reference point height and a second fraction of a height of the road image, each of the first fraction and second fraction being positive and less than one;

(i) processing the road image to identify a lane candidate within a lane-existing region of the road image, the lane-existing region having (i) a first subregion having the first height with respect to the bottom of the road image, (ii) a second subregion having the second height with respect to the bottom of the road image, and (iii) the reference point,

(ii) verifying the lane candidate as a true lane candidate when a minimum distance between (a) a line fit to a portion of the lane candidate in the first subregion and (b) the predetermined reference point is less than a neighborhood distance; and

(iii) extending the true lane candidate into the second subregion to form a detected lane marker demarcating the lane marker.

12. The system of claim 11, the image processor being further capable of, when executing the machine-readable instructions to process the road image:

identifying a ridge feature within the road image to generate a ridge-feature image; and

identifying, within the ridge-feature image, a plurality of lines each at least partially overlapping the ridge feature.

13. The system of claim 12, the plurality of lines being mutually non-collinear, and the image processor being further capable of, when executing the machine-readable instructions, conjoining the plurality of lines to identify the lane candidate.

14. The system of claim 13, the image processor being further capable of, when executing the machine-readable instructions to conjoin the plurality of lines:

executing a morphological operator on the plurality of lines to form the lane candidate.

15. The system of claim 12, the image processor being further capable of, when executing the machine-readable instructions to identify a ridge feature:

computing a first horizontal image gradient partially based on a first image intensity representative of a first image region entirely to the left of the location;

computing a second horizontal image gradient partially based on a second image intensity representative of a second image region entirely to the right of the location; and

determining whether the location is part of a ridge feature based on the first horizontal image gradient and the second horizontal image gradient.

16. The system of claim 15, the image processor being further capable of, when executing the machine-readable instructions, determining, based on a vertical position of the location, a horizontal width of at least one of (a) the first image region, (b) the second image region, and (c) the ridge feature.

17. The system of claim 15, the image processor being further capable of, when executing the machine-readable instructions, applying a continuity threshold operation to the ridge feature image to disqualify ridge features not corresponding to a lane marker.

18. The system of claim 17, each ridge feature being formed of a plurality of candidate ridge pixels, the continuity threshold operation having a threshold according to a minimum number of spatially-connected candidate ridge pixels in each candidate ridge feature.

19. The system of claim 12, one of the plurality of lines being fit to a plurality of near control points in the ridge-feature image corresponding to the lane candidate, the near control points being in a near subregion of the ridge-feature image corresponding to the first subregion of the road image.

20. The system of claim 12, the predetermined reference point in the road image being a vanishing point of the road image.

21. The system of claim 11, the image processor being further capable of, when executing the machine-readable instructions to extend the true lane candidate:

identifying, in the second subregion, a far control point corresponding to the true lane candidate; and

fitting a curve to the far control point and a plurality of near control points located in the first subregion and corresponding to the true lane candidate.

22. The system of claim 21, the far control point being adjacent to a pixel of the true lane candidate in the first subregion.

24. The system of claim 11, the first fraction being equal to three-quarters, the second fraction being equal to one-sixteenth.

26. The system of claim 14, the morphological operator being a closing operator.