METHOD, APPARATUS AND DEVICE FOR DETECTING LANE LINES

• US 20180060677A1
• Filed: 11/25/2016
• Published: 03/01/2018
• Est. Priority Date: 08/29/2016
• Status: Active Grant

• Current Assignee: Neusoft Reach Automotive Technology (Shanghai) Co., Ltd.
• Original Assignee: Neusoft Corporation, Neusoft Reach Automotive Technology (Shanghai) Co., Ltd.
• Inventors: Bai, Wenjing, Du, Yingfang, Liu, Wei, Yuan, Huai
• Granted Patent: US 10,140,529 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A method for detecting a lane line is provided. The method includes: determining candidate lane line regions in a lane image by filtering the lane image; extracting center points for each candidate lane line region; obtaining a fitting curve of each candidate lane line region by performing curve fitting on the center points of each candidate lane line region; classifying the candidate lane line regions as rectilinear and curvilinear candidate lane line regions according to a slope angle of the fitting curve of each candidate lane line region; dividing the candidate lane line regions into groups based on the classifying result; and determining two groups, which meet a first predetermined condition, as target groups, and recognizing candidate lane line regions in the target groups as lane line regions. Besides, an apparatus and a device for detecting a lane line are further provided.

Claims

1. A method for detecting a lane line, comprising:

determining candidate lane line regions in a lane image by filtering the lane image;

extracting center points for each of the candidate lane line regions, wherein the center points comprise a midpoint between two endpoints of each row in each of the candidate lane line regions;

obtaining a fitting curve of each of the candidate lane line regions by performing curve fitting on the center points of each of the candidate lane line regions;

classifying the candidate lane line regions as rectilinear candidate lane line regions and curvilinear candidate lane line regions according to a slope angle of the fitting curve of each of the candidate lane line regions;

dividing the candidate lane line regions into groups based on the classifying result, wherein each curvilinear candidate lane line region alone forms a group, one or more rectilinear candidate lane line regions form a group and candidate lane line regions in a same group correspond to a same lane line; and

determining two of the groups, which meet a first predetermined condition, as target groups, and recognizing candidate lane line regions in the target groups as lane line regions,

wherein the first predetermined condition comprises at least one of;

a distance between lane lines corresponding to the two groups is within a first predetermined range of distance, and an included angle of the lane lines corresponding to the two groups is within a first predetermined range of included angle.

2. The method according to claim 1, wherein two rectilinear candidate lane line regions which meet a second predetermined condition are in a same group,

wherein the second predetermined condition comprises:

fitting line segments of the two rectilinear candidate lane line regions do not overlap each other in a vertical direction, a distance between center points, which are closest to each other in the vertical direction, of the two rectilinear candidate lane line regions is within a second predetermined range of distance, and a directional deviation between the fitting line segments of the two rectilinear candidate lane line regions is within a predetermined range of deviation, and

wherein the fitting line segments of the rectilinear candidate lane line regions are obtained by performing rectilinear fitting on center points of the rectilinear candidate lane line regions.

3. The method according to claim 1, wherein after extracting center points for each of the candidate lane line regions, the method further comprises:

recognizing the number of effective pixels in the center points as an effective pixel number, for each of the candidate lane line regions; and

filtering out ones of the candidate lane line regions, which have effective pixel numbers smaller than a predetermined number threshold, and for the remaining ones of the candidate lane line regions, performing the action of curve fitting on the center points of each of the candidate lane line regions.

4. The method according to claim 1, wherein obtaining the fitting curve of each of the candidate lane line regions by performing curve fitting on the center points of each of the candidate lane line regions comprises:

selecting a plurality of control points from the center points of each of the candidate lane line regions, wherein the control points comprise center points at two ends of each of the candidate lane line regions;

obtaining curve equations of pairs of adjacent control points in each of the candidate lane line regions by perform fitting on the control points of each of the candidate lane line regions; and

forming the fitting curve of each of the candidate lane line regions with the curve equations of the pairs of adjacent control points of each of the candidate lane line regions.

5. The method according to claim 4, wherein selecting the plurality of control points from the center points of each of the candidate lane line regions comprises:

determining whether there is an inflection point in the center points of each of the candidate lane line regions, wherein the inflection point is a peak point or a valley point;

in response to no inflection point being in the center points of one of the candidate lane line regions, selecting first target points from the candidate lane line region as the control points, wherein distances between the pairs of adjacent first target points in the candidate lane line region are equal to each other and the first target points of the candidate lane line region comprise center points at two ends of the candidate lane line region; and

in response to an inflection point being in the center points of the candidate lane line region, selecting second target points from the candidate lane line region as the control points, wherein the second target points as the control points of the candidate lane line region comprise the center points at two ends of the candidate lane line region and the inflection point, distances between adjacent two second target points located between a center point at a first end of the candidate lane line region and the inflection point are equal to each other, and distances between adjacent two second target points located between a center point at a second end of the candidate lane line region and the inflection point are equal to each other.

6. The method according to claim 1, further comprising:

calculating a real length of each of the candidate lane line regions and determining whether there is an inflection point in the center points of each of the candidate lane line regions, wherein the inflection point is a peak point or a valley point; and

recognizing ones of the candidate lane line regions, real lengths of which are within a predetermined range of length and the center points of which do not include an inflection point, as rectilinear candidate lane line regions, and, for the rest of the candidate lane line regions, performing the step of classifying the candidate lane line regions as rectilinear candidate lane line regions and curvilinear candidate lane line regions according to the slope angle of the fitting curve of each of the candidate lane line regions.

7. The method according to claim 1, wherein classifying the candidate lane line regions as rectilinear candidate lane line regions and curvilinear candidate lane line regions according to the slope angle of the fitting curve of each of the candidate lane line regions comprises:

selecting a first sampling point, a second sampling point and a third sampling point from the center points of each of the candidate lane line regions;

calculating slope angles of the fitting curve at the first sampling point, the second sampling point and the third sampling point as a first slope angle, a second slope angle and a third slope angle, for each of the candidate lane line regions; and

recognizing, based on the first slope angle, the second slope angle and the third slope angle of each of the candidate lane line regions, ones of the candidate lane line regions, which meet a third predetermined condition or a fourth predetermined condition, as curvilinear candidate lane line regions and other ones of the candidate lane line regions, which meet neither the third predetermined condition nor the fourth predetermined condition, as rectilinear candidate lane line regions,

wherein the third predetermined condition is that both a difference between the first slope angle and the second slope angle of the corresponding candidate lane line region and a difference between the second slope angle and the third slope angle of the corresponding candidate lane line region are greater that a first slope angle threshold, and

wherein the fourth predetermined condition is that a difference between the first slope angle and the second slope angle of the corresponding candidate lane line region is smaller than the first slope angle threshold and a difference between the second slope angle and the third slope angle of the corresponding candidate lane line region is greater than a second slope angle threshold.

8. The method according to claim 7, wherein selecting the first sampling point, the second sampling point and the third sampling point from the center points of each of the candidate lane line regions comprises:

determining whether there is an inflection point in the center points of each of the candidate lane line regions, wherein the inflection point is a peak point or a valley point;

in response to no inflection point being in the center points of one of the candidate lane line regions, selecting any three center points from the candidate lane line region as the first sampling point, the second sampling point and the third sampling point of the candidate lane line region; and

in response to an inflection point being in the center points of the candidate lane line region, determining the inflection point in the candidate lane line region as the second sampling point of the candidate lane line region and selecting two center points at two sides of the second sampling point in the candidate lane line region as the first sampling point and the third sampling point of the candidate lane line region.

9. The method according to claim 5, wherein determining whether there is the inflection point in the center points of each of the candidate lane line regions comprises:

searching for a third target point with a maximum horizontal coordinate and a fourth target point with a minimum horizontal coordinate in the center points of each of the candidate lane line regions;

in response to the third target point and the fourth target point meeting a fifth predetermined condition or a sixth predetermined condition, determining that there is an inflection point in the center points of the corresponding candidate lane line region; and

in response to the third target point and the fourth target point meeting neither the fifth predetermined condition nor the sixth predetermined condition, determining that there is no inflection point in the center points of the corresponding candidate lane line region,

wherein the fifth predetermined condition is that, the third target point is in the middle of the corresponding candidate lane line region and the fourth target point is in either one of two ends of the corresponding candidate lane line region, and

wherein the sixth predetermined condition is that, the fourth target point is in the middle of the corresponding candidate lane line region and the third target point is in either one of the two ends of the corresponding candidate lane line region.

10. The method according to claim 6, wherein determining whether there is the inflection point in the center points of each of the candidate lane line regions comprises:

searching for a third target point with a maximum horizontal coordinate and a fourth target point with a minimum horizontal coordinate in the center points of each of the candidate lane line regions;

in response to the third target point and the fourth target point meeting a fifth predetermined condition or a sixth predetermined condition, determining that there is an inflection point in the center points of the corresponding candidate lane line region; and

in response to the third target point and the fourth target point meeting neither the fifth predetermined condition nor the sixth predetermined condition, determining that there is no inflection point in the center points of the corresponding candidate lane line region,

wherein the fifth predetermined condition is that, the third target point is in the middle of the corresponding candidate lane line region and the fourth target point is in either one of two ends of the corresponding candidate lane line region, and

wherein the sixth predetermined condition is that, the fourth target point is in the middle of the corresponding candidate lane line region and the third target point is in either one of the two ends of the corresponding candidate lane line region.

11. The method according to claim 8, wherein determining whether there is the inflection point in the center points of each of the candidate lane line regions comprises:

searching for a third target point with a maximum horizontal coordinate and a fourth target point with a minimum horizontal coordinate in the center points of each of the candidate lane line regions;

in response to the third target point and the fourth target point meeting a fifth predetermined condition or a sixth predetermined condition, determining that there is an inflection point in the center points of the corresponding candidate lane line region; and

in response to the third target point and the fourth target point meeting neither the fifth predetermined condition nor the sixth predetermined condition, determining that there is no inflection point in the center points of the corresponding candidate lane line region,

wherein the fifth predetermined condition is that, the third target point is in the middle of the corresponding candidate lane line region and the fourth target point is in either one of two ends of the corresponding candidate lane line region, and

wherein the sixth predetermined condition is that, the fourth target point is in the middle of the corresponding candidate lane line region and the third target point is in either one of the two ends of the corresponding candidate lane line region.

12. The method according to claim 2, wherein the directional deviation between the fitting line segments of the two rectilinear candidate lane line regions being within the predetermined range of deviation comprises at least of:

an included angle of the fitting line segments of the two rectilinear candidate lane line regions is within a second predetermined range of included angle, an estimated real included angle of the two rectilinear candidate lane line regions is within a third predetermined range of included angle and a deviation distance between the fitting line segments of the two rectilinear candidate lane line regions is within a third predetermined range of distance,

wherein the deviation distance is a distance between a first intersection point and a second intersection point, the first intersection point and the second intersection point are intersection points of extension lines of the fitting line segments of the two rectilinear candidate lane line regions and a horizontal midline, respectively, and distances between closest center points of the two rectilinear candidate lane line regions in the vertical direction and the horizontal midline are equal to each other.

13. The method according to claim 2, wherein the second predetermined condition further comprises:

in a case that the included angle of the fitting line segments of the two rectilinear candidate lane line regions is within a fourth predetermined range of included angle, the deviation distance between the fitting line segments of the two rectilinear candidate lane line regions is within a fourth predetermined range of distance,

wherein the second predetermined range of included angle is wider than the fourth predetermined range of included angle and the third predetermined range of distance is wider that the fourth predetermined range of distance.

14. The method according to claim 1, wherein after recognizing candidate lane line regions in the target groups as lane line regions, the method further comprises:

performing fitting on the lane line regions in the target groups, to obtain lane lines in the lane image,

wherein the lane line regions in each of the target groups are fitted into a lane line.

15. An apparatus for detecting a lane line, comprising:

a filtering unit, configured to determine candidate lane line regions in a lane image by filtering the lane image;

an extracting unit, configured to extract center points for each of the candidate lane line regions, wherein the center points comprise a midpoint between two endpoints of each row in each of the candidate lane line regions;

a first fitting unit, configured to obtain a fitting curve of each of the candidate lane line regions by performing curve fitting on the center points of each of the candidate lane line regions;

a classifying unit, configured to classify the candidate lane line regions as rectilinear candidate lane line regions and curvilinear candidate lane line regions according to a slope angle of the fitting curve of each of the candidate lane line regions;

a grouping unit, configured to divide the candidate lane line regions into groups based on the classifying result, wherein each curvilinear candidate lane line region alone forms a group, one or more rectilinear candidate lane line regions form a group and candidate lane line regions in a same group correspond to a same lane line; and

a first recognition unit, configured to determine two of the groups, which meet a first predetermined condition, as target groups and recognize candidate lane line regions in the target groups as lane line regions,

wherein the first predetermined condition comprises at least one of;

a distance between lane lines corresponding to the two groups is within a first predetermined range of distance, and an included angle of the lane lines corresponding to the two groups is within a first predetermined range of included angle.

16. The apparatus according to claim 15, wherein two rectilinear candidate lane line regions which meet a second predetermined condition are in a same group,

wherein the second predetermined condition comprises:

fitting line segments of the two rectilinear candidate lane line regions do not overlap each other in a vertical direction, a distance between center points, which are closest to each other in the vertical direction, of the two rectilinear candidate lane line regions is within a second predetermined range of distance, and a directional deviation between the fitting line segments of the two rectilinear candidate lane line regions is within a predetermined range of deviation, and

wherein the fitting line segments of the rectilinear candidate lane line regions are obtained by performing rectilinear fitting on center points of the rectilinear candidate lane line regions.

17. The apparatus according to claim 15, further comprising:

a second recognition unit, configured to, after extracting the center points for each of the candidate lane line regions, recognize the number of effective pixels in the center points as an effective pixel number, for each of the candidate lane line regions; and

a filtering unit, configured to filter out ones of the candidate lane line regions, which have effective pixel numbers smaller than a predetermined number threshold, and for the remaining ones of the candidate lane line regions, perform the action of curve fitting on the center points of each of the candidate lane line regions.

18. The apparatus according to claim 15, wherein the first fitting unit comprises:

a first selecting unit, configured to select multiple control points from the center points of each of the candidate lane line regions, where the control points include center points at two ends of each of the candidate lane line regions;

a second fitting unit, configured to obtain curve equations of pairs of adjacent control points in each of the candidate lane line regions by perform fitting on the control points of each of the candidate lane line regions; and

a forming unit, configured to form the fitting curve of each of the candidate lane line regions with the curve equations of the pairs of adjacent control points of each of the candidate lane line regions.

19. The apparatus according to claim 15, further comprising:

a first calculating unit, configured to calculate a real length of each of the candidate lane line regions and determining whether there is an inflection point in the center points of each of the candidate lane line regions, where the inflection point is a peak point or a valley point; and

a third recognition unit, configured to recognize ones of the candidate lane line regions, real lengths of which are within a predetermined range of length and the center points of which do not include an inflection point, as rectilinear candidate lane line regions, and, for the rest of the candidate lane line regions, perform the step of classifying the candidate lane line regions as rectilinear candidate lane line regions and curvilinear candidate lane line regions according to the slope angle of the fitting curve of each of the candidate lane line regions.

20. A device for detecting a lane line, comprising a processor, a memory, a communication interface and a bus system, wherein,

the bus system is configured to couple hardware of the device together;

the communication interface is configured to implement communication connection between the device and at least one other device;

the memory is configured to store program instructions and data; and

the processor is configured to read the instructions and the data stored in the memory and perform the following operations;

determining candidate lane line regions in a lane image by filtering the lane image;

extracting center points for each of the candidate lane line regions, wherein the center points comprise a midpoint between two endpoints of each row in each of the candidate lane line regions;

obtaining a fitting curve of each of the candidate lane line regions by performing curve fitting on the center points of each of the candidate lane line regions;

classifying the candidate lane line regions as rectilinear candidate lane line regions and curvilinear candidate lane line regions according to a slope angle of the fitting curve of each of the candidate lane line regions;

dividing the candidate lane line regions into groups based on the classifying result, wherein each curvilinear candidate lane line region alone forms a group, one or more rectilinear candidate lane line regions form a group and candidate lane line regions in a same group correspond to a same lane line; and

determining two of the groups, which meet a first predetermined condition, as target groups, and recognizing candidate lane line regions in the target groups as lane line regions,

wherein the first predetermined condition comprises at least one of;

a distance between lane lines corresponding to the two groups is within a first predetermined range of distance, and an included angle of the lane lines corresponding to the two groups is within a first predetermined range of included angle.