Method and device of lane departure warning and automobile driving assistance system

• US 10,360,460 B2
• Filed: 03/30/2017
• Issued: 07/23/2019
• Est. Priority Date: 02/03/2016
• Status: Active Grant

• Current Assignee: Beijing Smarter Eye Technology Co., Ltd.
• Original Assignee: Beijing Smarter Eye Technology Co., Ltd.
• Inventors: Jiang, An, Meng, Ran, Zhu, Haitao, Cui, Feng, Xie, Qiwei
• Primary Examiner(s): Beg, Samah A
• Application Number:
  US15/474,755
• Publication Number:
  US 20170262713A1
• Time in Patent Office: 845 Days
• Field of Search: None
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/42 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2520/10 Longitudinal speed
• CPC Class Codes: B60W 2540/18 Steering angle
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A lane departure warning method includes steps of: collecting a road image by an imaging device; detecting a lane marking in accordance with the road image, so as to extract a position of the lane marking and an angle of the lane marking relative to a running direction of the vehicle; acquiring steering information and a movement speed of the vehicle by an OBD system; judging whether or not the vehicle is unconsciously running on the lane marking in accordance with the position of the lane marking, the angle of the lane marking relative to the running direction of the vehicle and the steering information of the vehicle; in the case that the vehicle is unconsciously running on the lane marking, recording a duration within which the vehicle is unconsciously running on the lane marking; and judging whether or not to send a lane departure warning to the vehicle.

Claims

1. A lane departure warning method, an imaging device being arranged on a central axis of a front windshield of a vehicle, the lane departure warning method at least comprising steps of:

collecting a road image by the imaging device;

detecting a lane marking in accordance with the road image, so as to extract a position of the lane marking and an angle of the lane marking relative to a running direction of the vehicle;

acquiring steering information and a movement speed of the vehicle by an On-Board Diagnostic (OBD) system;

judging whether or not the vehicle is unconsciously running on the lane marking in accordance with the position of the lane marking, the angle of the lane marking relative to the running direction of the vehicle and the steering information of the vehicle;

in the case that the vehicle is unconsciously running on the lane marking, recording a duration within which the vehicle is unconsciously running on the lane marking; and

judging whether or not to send a lane departure warning to the vehicle in accordance with the duration within which the vehicle is unconsciously running on the lane marking and the movement speed of the vehicle.

2. The lane departure warning method according to claim 1, wherein the step of detecting the lane marking in accordance with the road image so as to extract the position of the lane marking and the angle of the lane marking relative to the running direction of the vehicle comprises:

subjecting the road image to pretreatment;

subjecting a pretreatment result to morphological operation so as to extract a lane marking characteristic pattern;

subjecting the lane marking characteristic pattern to topical self-adaptive binarization, so as to acquire a binary image; and

subjecting the binary image to Blob analysis, so as to determine an image block that conforms to a lane marking characteristic as the lane marking, and extract the position of the lane marking and the angle of the lane marking relative to the running direction of the vehicle.

3. The lane departure warning method according to claim 2, wherein the step of subjecting the road image to the pretreatment comprises:

selecting a region-of-interest (ROI) from the road image;

subjecting the ROI to gray processing;

subjecting an image acquired after the gray processing to low-pass filtration; and

subjecting an image acquired after the low-pass filtration to contrast adjustment.

4. The lane departure warning method according to claim 2, wherein the step of subjecting the pretreatment result to the morphological operation so as to extract the lane marking characteristic pattern comprises subjecting the road image to top-hat operation using a structural element in a horizontal direction, and the structural element has a length twice a width of the lane marking in the road image.

5. The lane departure warning method according to claim 2, wherein the step of subjecting the lane marking characteristic pattern to the topical self-adaptive binarization so as to acquire the binary image comprises:

with respect to each pixel in the lane marking characteristic pattern, calculating a difference between an average grayscale value of the pixel and an average grayscale value of a pixel at the neighboring region;

determining whether or not the difference is greater than a predetermined threshold; and

in the case that the difference is greater than the predetermined threshold, setting a grayscale value of the pixel as 255, and otherwise setting the grayscale value of the pixel as 0.

6. The lane departure warning method according to claim 2, wherein the step of subjecting the binary image to the Blob analysis, so as to determine the image block that conforms to the lane marking characteristic as the lane marking, and extract the position of the lane marking and the angle of the lane marking relative to the running direction of the vehicle comprises:

calculating a statistical attribute of each image block in the binary image, the statistical attribute comprising, but not limited to, one or more of center of mass, perimeter, area, direction, Euler number, eccentricity, length of principal axis, length of secondary axis, and a ratio of the length of principal axis to the length of secondary axis;

establishing a regression decision tree model in accordance with the statistical attribute and the lane marking characteristic;

determining whether or not the image block is the lane marking in accordance with the regression decision tree model; and

extracting a position of the image block that has been determined as the lane marking, and an angle of the image block relative to the running direction of the vehicle.

7. The lane departure warning method according to claim 1, wherein the step of judging whether or not the vehicle is unconsciously running on the lane marking in accordance with the position of the lane marking, the angle of the lane marking relative to the running direction of the vehicle and the steering information of the vehicle comprises:

judging whether or not the running direction of the vehicle is being changed in accordance with the steering information of the vehicle; and

in the case that the running direction of the vehicle is being changed, the lane marking is located at a predetermined region in the road image, and the angle of the lane marking relative to the running direction of the vehicle is within a predetermined range, determining that the vehicle is unconsciously running on the lane marking.

8. A lane departure warning device, an imaging device being arranged on a central axis of a front windshield of a vehicle, the lane departure warning device at least comprising:

a collection unit configured to collect a road image by the imaging device;

an extraction unit configured to detect a lane marking in accordance with the road image, so as to extract a position of the lane marking and an angle of the lane marking relative to a running direction of the vehicle;

an acquisition unit configured to acquire steering information and a movement speed of the vehicle by an On-Board Diagnostic (OBD) system;

a first judgment unit configured to judge whether or not the vehicle is unconsciously running on the lane marking in accordance with the position of the lane marking, the angle of the lane marking relative to the running direction of the vehicle and the steering information of the vehicle;

a recording unit configured to, in the case that the vehicle is unconsciously running on the lane marking, record a duration within which the vehicle is unconsciously running on the lane marking; and

a second judgment unit configured to judge whether or not to send a lane departure warning to the vehicle in accordance with the duration within which the vehicle is unconsciously running on the lane marking and the movement speed of the vehicle.

9. The lane departure warning device according to claim 8, wherein the extraction unit comprises:

a pretreatment module configured to subject the road image to pretreatment;

a first extraction module configured to subject a pretreatment result to morphological operation so as to extract a lane marking characteristic pattern;

a binarization module configured to subject the lane marking characteristic pattern to topical self-adaptive binarization, so as to acquire a binary image; and

a second extraction module configured to subject the binary image to Blob analysis, so as to determine an image block that conforms to a lane marking characteristic as the lane marking, and extract the position of the lane marking and the angle of the lane marking relative to the running direction of the vehicle.

10. The lane departure warning device according to claim 9, wherein the pretreatment module comprises:

a selection module configured to select a region-of-interest (ROI) from the road image;

a gray processing module configured to subject the ROI to gray processing;

a filtration module configured to subject the image acquired after the gray processing to low-pass filtration; and

an adjustment module configured to subject the image acquired after the low-pass filtration to contrast adjustment.

11. The lane departure warning device according to claim 9, wherein the first extraction module comprises an operation module configured to subject the road image to top-hat operation using a structural element in a horizontal direction, so as to extract the lane marking characteristic pattern, and the structural element has a length twice a width of the lane marking in the road image.

12. The lane departure warning device according to claim 9, wherein the binarization module comprises:

a first calculation module configured to, with respect to each pixel in the lane marking characteristic pattern, calculate a difference between an average grayscale value of the pixel and an average grayscale value of a pixel at the neighboring region; and

a first judgment module configured to judge whether or not the difference is greater than a predetermined threshold, in the case that the difference is greater than the predetermined threshold, set a grayscale value of the pixel as 255, and otherwise set the grayscale value of the pixel as 0.

13. The lane departure warning device according to claim 9, wherein the second extraction module comprises:

a second calculation module configured to calculate a statistical attribute of each image block in the binary image, the statistical attribute comprising, but not limited to, one or more of center of mass, perimeter, area, direction, Euler number, eccentricity, length of principal axis, length of secondary axis, and a ratio of the length of principal axis to the length of secondary axis;

an establishment module configured to establish a regression decision tree model in accordance with the statistical attribute and the lane marking characteristic;

a first determination module configured to determine whether or not the image block is the lane marking in accordance with the regression decision tree model; and

a third extraction module configured to extract a position of the image block that has been determined as the lane marking, and an angle of the image block relative to the running direction of the vehicle.

14. The lane departure warning device according to claim 8, wherein the first judgment unit comprises:

a second judgment module configured to judge whether or not the running direction of the vehicle is being changed in accordance with the steering information of the vehicle; and

a second determination module configured to, in the case that the running direction of the vehicle is being changed, the lane marking is located at a predetermined region in the road image, and the angle of the lane marking relative to the running direction of the vehicle is within a predetermined range, determine that the vehicle is unconsciously running on the lane marking.

15. A vehicle assistant driving system, comprising at least the lane departure warning device according to claim 8.