Lane marker recognizing apparatus

• US 6,813,370 B1
• Filed: 09/22/2000
• Issued: 11/02/2004
• Est. Priority Date: 09/22/1999
• Status: Active Grant

• Current Assignee: Subaru Corp.
• Original Assignee: Fuji Heavy Industries Limited (Subaru Corp.)
• Inventors: Arai, Kazumasa
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US09/667,535
• Time in Patent Office: 1,502 Days
• Field of Search: 382/100, 382/103, 382/104, 382/106, 382/107, 382/154, 382/199, 340/901, 340/902, 340/903, 340/904, 340/905, 340/906, 340/907, 340/908, 340/438, 340/439, 340/938, 340/939, 340/988, 701/23, 701/28, 701/29, 701/36, 701/40, 701/41, 701/44, 701/70, 701/93, 701/96, 701/98, 701/116, 701/117, 701/225, 348/113, 348/149, 404/12, 404/93
• US Class Current: 382/104
• CPC Class Codes: B60T 2210/22 Banked curves
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/457 by analysing connectivity, e.g. edge linking, connected component analysis or slices
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 13/133 Equalising the characteristics of different image components, e.g. their average brightness or colour balance
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0077 Colour aspects
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0092 Image segmentation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0096 Synchronisation or controlling aspects

Abstract

A lane recognition apparatus for recognizing lane markers on a road surface based on a pair of stereo images of a scene in front of an own vehicle obtained from a stereoscopic camera, includes a stereo image processing means for calculating distance information of an image based on the pair of the images, a lane detecting means for detecting a lane marker of a lane in the image based on brightness information of the image and distance information calculated by the stereo image processing means and a lane position recognition means for recognizing a position of the lane marker in real space based on a position of the lane marker detected by the lane detecting means and distance information calculated by the stereo image processing means.

Claims

1. A lane recognition apparatus having an apparatus for recognizing lane markers on a road surface based on a pair of stereo images of a scene in front of an own vehicle obtained from a stereoscopic camera, comprising:

a stereo image processing means for calculating a distance information of an image based on said pair of said images;

a lane detecting means for detecting a lane marker of a lane in said image based on a brightness information of said image; and

a lane position recognition means for recognizing a position of said lane of said lane marker in real space based on a position of said lane marker detected by said lane detecting means and said distance information calculated by said stereo image processing means.

2. The lane recognition apparatus according to claim 1, further comprising:

a detection area establishing means for establishing a detection area for every horizontal line in said image.

3. A lane recognition apparatus for recognizing lane markers on a road surface based on a pair of stereo images of a scene in front of an own vehicle obtained from a stereoscopic camera, comprising:

a stereo image processing means for calculating a distance information of an image based on said pair of said images;

a lane detecting means for detecting a lane marker of a lane in said image based on a brightness information of said image and said distance information calculated by said stereo image processing means; and

a lane position recognition means for recognizing a position of said lane marker in real space based on a position of said lane marker detected by said lane detecting means and said distance information calculated by said stereo image processing means.

4. The lane recognition apparatus according to claim 3, further comprising:

a detection area establishing means for establishing a detection area for every horizontal line in said image.

5. A lane recognition apparatus having an apparatus for recognizing lane markers on a road surface based on a pair of stereo images of a scene in front of an own vehicle obtained from a stereoscopic camera, comprising:

a stereo image processing means for calculating a distance information of an image based on said pair of said images;

a lane detecting means for detecting a lane marker of a lane in said image based on a brightness information of said image;

a lane position recognition means for recognizing a position of said lane of said lane marker in real space based on a position of said lane marker detected by said lane detecting means and said distance information calculated by said stereo image processing means; and

a detection area establishing means for establishing a detection area for every horizontal line in said image,

wherein said detection area establishing means has a first mode in which said detection area is established based on the result of detection of said lane marker in a previous image on a horizontal line as a detection object and a second mode in which said detection area is established based on the result of detection of said lane marker is a present image on a horizontal line as a detection object and said detection area is established on said horizontal line according to a mode selected from said first and second modes based on a detecting condition of said lane marker.

6. The lane recognition apparatus according to claim 5, wherein

said lane detecting means detects said lane marker within said detection area established by said detection area establishing means.

7. The lane recognition apparatus according to claim 6, wherein

said lane detecting means judges a position where a horizontal edge intensity is larger than a positive threshold value and a position where said horizontal edge intensity is smaller than a negative threshold value to be a boarder of road surface and lane marker, respectively.

8. The lane recognition apparatus according to claim 7, wherein

said lane detecting means judges an area having larger brightness than a brightness threshold value calculated based on a road surface brightness which is calculated from a brightness of a plurality of horizontal lines located in the vicinity of a horizontal line as a detection object.

9. The lane recognition apparatus according to claim 6, wherein

said lane detecting means judges a position of a boarder of a road surface and lane marker based on a first brightness threshold value and a second brightness threshold value larger than said first brightness threshold value, said first brightness threshold value is a threshold value for judging a boarder of road surface and lane marker and said second brightness threshold value is a threshold value for judging a brightness condition having larger brightness than a road surface in a searching area.

10. The lane recognition apparatus according to claim 5, further comprising:

an evaluation means for calculating a reliability of said lane marker detected in a previous image.

11. The lane recognition apparatus according to claim 9, wherein

said detection area establishing means changes a size of said detection area according to said reliability calculated by said detection area according to said reliability calculated by said evaluation means.

12. The lane recognition apparatus according to claim 10, wherein

said evaluation means evaluates said reliability of said lane marker based on the number of detected data and a continuity between a position of lane marker in present image and a position of lane marker in a previous image.

13. The lane recognition apparatus according to claim 5, further comprising:

a correcting means for evaluating a parallelization on left and right lane markers recognized by said lane position recognition means and for correcting said recognized position of said lane markers based on the result of said evaluated parallelization.

14. The lane recognition apparatus according to claim 13, further comprising;

a road height calculating means for calculating a height of road for every segment of road and a road height estimating means for estimating a height of road of all segments in a present image by correcting a height of road calculated in a previous image of all segments based on a deviation between a present height of road calculated in a specified segment on the near side of an own vehicle and a height of road calculated in a previous image.

15. The lane recognition apparatus according to claim 5, wherein

said lane detecting means detects all candidates of lane markers existing on at least one horizontal line in said image by detecting said lane marker in all areas of said horizontal line.

16. The lane recognition apparatus according to claim 5, wherein

said lane detecting means extracts said lane marker by applying a weight according to a detecting position on at least one horizontal line.

17. The lane recognition apparatus according to claim 16, wherein

said weight is changed according to a vehicle speed.

18. The lane recognition apparatus according to claim 5, wherein

said detection area establishing means makes a horizontal width of said detection area narrower as said detection area goes upward of said image.

19. The lane recognition apparatus according to claim 5, wherein

said detection area establishing means, in case where no lane marker can be detected in a horizontal line as a detection object, broadens said detection area in a horizontal line as a detection object, broadens said detection area in a horizontal line as a detection object.

20. A lane recognition apparatus for recognizing lane markers on a road surface based on a pair of stereo images of a scene in front of an own vehicle obtained from a stereoscopic camera, comprising:

a stereo image processing means for calculating a distance information of an image based on said pair of said images;

a lane detecting means for detecting a lane marker of a lane in said image based on a brightness information of said image and said distance information calculated by said stereo image processing means;

a lane position recognition means for recognizing a position of said lane marker in real space based on a position of said lane marker detected by said lane detecting means and said distance information calculated by said stereo image processing means; and

a detection area establishing means for establishing a detection area for every horizontal line in said image,

wherein said detection area establishing means has a first mode in which said detection area is established based on the result of detection of said lane marker in a previous image on a horizontal line as a detection object and said detection area is established on said horizontal line according to a mode selected from said first and second modes based on a detecting condition of said lane marker.

21. The lane recognition apparatus according to claim 20, wherein

said lane detecting means detects said lane marker within said detection area established by said detection area establishing means.

22. The lane recognition apparatus according to claim 21, wherein

said lane detecting means judges a position where a horizontal edge intensity is larger than a positive threshold value and a position where said horizontal edge intensity is smaller than a negative threshold value to be a boarder of road surface and lane marker, respectively.

23. The lane recognition apparatus according to claim 21, wherein

said lane detecting means judges an area having a larger brightness than a brightness threshold value calculated based on a road surface brightness which is calculated from a brightness of a plurality of horizontal lines located in the vicinity of a horizontal line as a detection object.

24. The lane recognition apparatus according to claim 21, wherein

said lane detecting means judges a position of a boarder of road surface and lane marker based on a first brightness threshold value and a second brightness threshold value larger than said first brightness threshold value for judging a boarder of road surface and lane marker and said second brightness threshold value, said first brightness threshold value is a threshold value for judging a boarder of value is a threshold value for judging a brightness condition having a larger brightness than a road surface in a searching area.

25. The lane recognition apparatus according to claim 20 further comprising:

an evaluation means for calculating a reliability of said lane marker detected in a previous image.

26. The lane recognition apparatus according to claim 25 wherein

said detection area establishing means changes a size of said detection area according to said reliability calculated by said evaluation means.

27. The lane recognition apparatus according to claim 25, wherein

said evaluation means evaluated said reliability of said lane marker based on the number of detected data and a continuity between a position of lane marker in a present image and a position for lane marker in a previous image.

28. The lane recognition apparatus according to claim 25, wherein

said evaluation means divides said lane marker into a plurality of segments and evaluates said reliability of said lane marker in respective segments.

29. The lane recognition apparatus according to claim 25, wherein

said evaluation means divides said lane marker into a plurality of segments and evaluates said reliability of said lane marker in respective segments.

30. The lane recognition apparatus according to claim 20, further comprising:

a correcting means for evaluating a parallelization of left and right lane markers recognized by said lane position recognition means and for correcting said recognized position of said lane markers based on the result of said evaluated parallelization.

31. The lane recognition apparatus according to claim 30, further comprising:

a road height calculating means for calculating a height of road for every segment of road and a road height estimating means for estimating a height of road of all segments in a present image by correcting a height of road calculated in a previous image of all segments based on a deviation between a present height of road calculated in a specified segment on the near side of a own vehicle and a height of road calculated in a previous image.

32. The lane recognition apparatus according to claim 20, wherein

said lane detecting means detects all candidates of lane markers existing on at least one horizontal line in said image by detecting said lane marker in all areas of said horizontal line.

33. The lane recognition apparatus according to claim 20, wherein

said lane detecting means extracts said lane marker by applying a weight according to a detecting position on at least one horizontal line.

34. The lane recognition apparatus according to claim 33, wherein

said weight is changed according to a vehicle speed.

35. The lane recognition apparatus according to claim 20, wherein

said detection area establishing means makes a horizontal width of said detection area narrower as said detection area goes upward of said image.

36. The lane recognition apparatus according to claim 20, wherein

said detection area establishing means, in case where no lane marker can be detected in a horizontal line as a detection object, broadens said detection area in a horizontal line as a detection object.

37. The lane recognition apparatus according to claim 20, wherein

said lane detecting means selects candidates of lane markers on a road surface based on said distance information calculated by said stereo image processing means and identifies a real lane marker from among said candidates of lane markers based on said brightness information of said image.

38. The lane recognition apparatus according to claim 20, wherein

said lane detecting means selects candidates of lane markers based on said brightness information of said image and identifies a real lane marker from among said candidates of lane markers based on said distance information calculated by said stereo image processing means.

39. A lane recognition apparatus having an apparatus for recognizing lane markers on a road surface based on a pair of stereo images of a scene in front of an own vehicle obtained from a stereoscopic camera, comprising:

a stereo image processing means for calculating a distance information of an image based on said pair of said images;

a lane detecting means for detecting a lane marker of a lane in said image based on a brightness information of said image;

a lane position recognition means for recognizing a position of said lane of said lane marker in real space based on a position of said lane marker detected by said lane detecting means and said distance information calculated by said stereo image processing means; and

a detection area establishing means for establishing a detection area for every horizontal line in said image,

wherein said detection area establishing means calculates a gradient of said lane marker in an image plane based on said lane marker already detected and based on said calculated gradient, establishes said detection area in a horizontal line as a detection object.

40. A lane recognition apparatus for recognizing lane markers on a road surface based on a pair of stereo images of a scene in front of an own vehicle obtained from a stereoscopic camera, comprising:

a stereo image processing means for calculating a distance information of an image based on said pair of said images;

a lane detecting means for detecting a lane marker of a lane in said image based on a brightness information of said image and said distance information calculated by said stereo image processing means;

a lane position recognition means for recognizing a position of said lane marker in real space based on a position of said lane marker detected by said lane detecting means and said distance information calculated by said stereo image processing means; and

wherein said detection area establishing means calculates a gradient of said lane marker in an image plane based on said lane marker already detected and based on said calculated gradient, establishes said detection area in a horizontal line as a detection object.

41. A solid object recognition apparatus having an apparatus for recognizing a solid object on a road surface based on a pair of stereo images with respect to an image obtained by imaging a scene in front of an own vehicle by a stereoscopic camera, comprising;

a stereo image processing means for calculating a distance information with respect to said image based on said pair of said stereo images;

a lane detecting means for detecting a lane marker in said image based on a brightness information of said image;

a road height calculating means for calculating a height of said lane marker in real space based on a position of said lane marker detected by said lane detecting means and a distance information calculated by said stereo image processing means; and

a solid object judging means for judging an object projected in said image to be an solid object when said object exists above road surface recognized by said road height calculating means.

42. A vehicle behavior estimating apparatus having an apparatus for detecting a pitching of an own vehicle based on a pair of stereo images with respect to an image obtained by imaging a scene in front of an own vehicle by a stereoscopic camera, comprising:

a stereo image processing means for calculating a distance information with respect to said image based on said pair of said stereo images;

a lane detecting means for detecting a lane marker in said image based on a brightness information of said image;

a road height calculating means for calculating a height of road in real space based on a position of said lane marker detected by said lane detecting means and a distance information calculated by said stereo image processing means; and

a pitching condition estimating means for estimating a pitching condition of an own vehicle based on a deviation between a height of road calculated by said road height calculating means and a height of road calculated in a previous image.

43. The vehicle behavior estimating apparatus according to claim 42, wherein

said road height calculating means calculates said height of road for every successive segment and estimates said pitching condition based on a deviation between a height of road calculated in a specified segment on a near side of an own vehicle and a height of road calculated in a previous image.

44. A vehicle behavior estimating apparatus having an apparatus for estimating a bank angle of a road surface based on a pair of stereo images with respect to an image obtained by imaging a scene in front of an own vehicle by a stereoscopic camera, comprising:

a stereo image processing means for calculating a distance information with respect to said image based on said pair of said stereo images;

a lane detecting means for detecting a lane marker in said image based on a brightness information of said image;

a lane height calculating means for calculating a height of a left lane marker in real space and a height of a right lane marker in real space separately based on a position of said lane marker detected by said lane detecting means and said distance information calculated by said stereo image processing means; and

a roll angle estimating means for estimating a roll angle of an own vehicle based on a deviation between said height of said left lane marker and said height of said right lane marker calculated by said lane height calculating means.

45. A road situation estimating apparatus having an apparatus for estimating a bank angle of a road surface based on a pair of stereo images with respect to an image obtained by imaging a scene in front of an own vehicle by a stereoscopic camera, comprising:

a stereo image processing means for calculating a distance information with respect to said image based on said pair of said stereo images;

a lane detecting means for detecting a lane marker in said image based on a brightness information of said image;

a lane height calculating means for calculating a height of a left lane marker in real space and height of a right lane marker in real space separately based on a position of said lane marker detected by said lane marker detecting means and said distance information calculated by said stereo image processing means; and

a bank angle estimating means for estimating a bank angle of a road surface based on a deviation between said height of said left lane marker and said height of said right lane marker calculated by said lane height calculating means.