Lane departure warning system and lane departure warning method

• US 9,659,497 B2
• Filed: 07/01/2013
• Issued: 05/23/2017
• Est. Priority Date: 06/29/2012
• Status: Active Grant

• Current Assignee: LG Innotek Company Limited (LG Corporation)
• Original Assignee: LG Innotek Company Limited (LG Corporation)
• Inventors: Kim, Ki Dae, Woo, Jeong Woo, Manohar, Babu, Gupta, Raghubansh B.
• Primary Examiner(s): Hallenbeck-Huber, Jeremaiah C
• Assistant Examiner(s): Hodges, Susan E
• Application Number:
  US13/932,164
• Publication Number:
  US 20140002656A1
• Time in Patent Office: 1,422 Days
• Field of Search: 348148
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A lane departure warning system includes an image photographing unit attached to a front of a vehicle to photograph an object in a forward direction of the vehicle; a driving unit receiving image data from the image photographing unit, filtering the image data to search for a lane pair, converting an image coordinate of the image data into a real coordinate to calculate a lateral distance between the lane pair and the vehicle and to calculate a lateral speed of the vehicle, and generating a warning generating signal as a time of lane change elapses by obtaining the time of lane change; and a warning unit receiving the warning generating signal to generate a lane departure warning signal.

Claims

1. A lane departure warning system comprising:

an image photographing unit attached to a front of a vehicle to photograph an object in a forward direction of the vehicle;

a driving unit receiving image data from the image photographing unit, filtering the image data to search for lane pairs, converting an image coordinate of the image data into a real coordinate to calculate a lateral distance between the lane pairs and the vehicle and to calculate a lateral speed of the vehicle, and generating a warning generating signal as a time of lane change elapses by obtaining the time of lane change; and

a warning unit receiving the warning generating signal to generate a lane departure warning signal;

wherein the image photographing unit comprises a camera which photographs the object and transfers the image data to the driving unit;

wherein the driving unit comprises;

an image organizer that receives the image data from the image photographing unit, filters the image data to extract candidate lane regions, and groups the candidate lane regions based on features of the extracted candidate lane regions;

a lane searching unit that searches for the lane pairs from the grouped candidate lane regions of the image organizer;

a lane tracking unit that searches for an optimal lane pair from a region of interest (ROI) based on the searched lane pairs, wherein if the optimal line pair is determined, a curved parameter or a straight parameter is calculated and stored; and

a warning generating unit that obtains the time of lane change by converting the image coordinate into the real coordinate, and generates a warning generating signal as the time of lane change elapses;

wherein the warning generating unit selects two points from a lane of the image coordinate, and converts the two points into the real coordinate by setting a coordinate of the vehicle as a zero point to obtain two real points;

wherein the image organizer comprises a preprocessing unit that selects the ROI from the image data to search a lane from the ROI and groups a plurality of regions determined as one lane, filtering being performed according to a data application value of each pixel while moving a mask having a predetermined size in the ROI, and candidate lane regions being generated by grouping a lane pixel according to at least one of a size, a boundary point, a main axis, a sub-axis, or a region gradient;

wherein an image of the ROI is converted to have a black and white color, and the candidate lane regions are recognized as the one lane and grouped as one candidate lane group, and a gradient of the generated group and lower and upper intersection points of the generated group are stored in a flag variable as features;

wherein the lane searching unit searches for lane pairs each including two lines constituting one lane from the grouped regions and searches for an optimal lane pair from the lane pairs, the search for the lane pairs being performed by extracting two lanes determined as a pair by comparing features of left and right candidate lane groups with each other;

wherein in a history check, currently searched lane pairs are compared with previously searched lane pairs, and a lane pair of the currently searched lane pairs matching with the previous lane pairs within a threshold range is determined as an optimal lane pair;

wherein the warning generating unit receives an optimal lane information from the lane searching unit or the lane tracking unit, calculates the lateral distance according to relationship between the lane pairs and the vehicle, and calculates the time of lane change according to the lateral distance between the lane pairs and the vehicle and the lateral speed of the vehicle, and if the time of lane change is within a predetermined range, the warning generating unit outputs the warning generating signal;

wherein the optimal lane pair comprises a left candidate lane, a right candidate lane, and a center candidate lane;

wherein the ROI includes a left ROI having a predetermined area ranging from a left peripheral region of the image on the left candidate lane and the center candidate lane and a right ROI having a predetermined area ranging from a right peripheral region of the image on the right candidate lane and the center candidate lane;

wherein central portions of the left ROI and the right ROI are partially overlapped;

wherein a running direction of the vehicle and a gradient of the lane are obtained using the two real points;

wherein the warning generating unit calculates the lateral distance between the lane pairs and the vehicle by determining whether any one of the lane pairs is a straight lane pair or a curved lane pair;

wherein, when the determined lane pair is the curved lane pair, the warning generating unit selects the two points from a lane under an intersection point between the running direction of the vehicle and the determined lane pair; and

wherein, when a running direction of the vehicle is aligned parallel to the straight lane pair while the curved lane pair starts after the straight lane pair is terminated, the calculation of the lateral distance between the lane pairs and the vehicle is performed by converting the lane pair into the straight lane pair.

2. The lane departure warning system of claim 1, wherein the warning generating unit converts the image coordinate into the real coordinate by taking a perspective into consideration.

3. The lane departure warning system of claim 1, wherein the lateral speed of the vehicle is calculated based on variations of a location of the vehicle in a plurality of frames.

4. The lane departure warning system of claim 1, wherein the lateral speed of the vehicle is calculated based on variations of the lateral distance between the lane pair and the vehicle in a plurality of frames.

5. The lane departure warning system of claim 1, wherein the lateral distance between the lane pairs and the vehicle is calculated with a tilt angle of a camera of the image photographing unit.

6. The lane departure warning system of claim 1, wherein the time of lane change is a value obtained by dividing the lateral distance between the lane pairs and the vehicle by the lateral speed of the vehicle.

11. The lane departure warning system of claim 1, wherein the lane tracking unit and the lane searching unit are selectively driven.

12. The lane departure warning system of claim 1, when the lane searching unit searches the lane pairs, if a lane is changed in a previous frame, a currently searched lane pair is determined as the optimal lane pair, and if the lane is not changed, a history of the currently searched lane pair is checked.

13. The lane departure warning system of claim 1, wherein the overlapped central portions of the left ROI and the right ROI are disposed on the center candidate lane.

7. A lane departure warning method comprising:

generating image data by photographing an object in a forward direction of a vehicle with an image photographing unit attached to a front of the vehicle to photograph the object;

filtering the image data to extract candidate lane regions, and grouping the candidate lane regions based on features of the extracted candidate lane regions;

searching for lane pairs from the grouped candidate lane regions;

obtaining a time of lane change by converting an image coordinate into a real coordinate; and

generating a warning generating signal as the time of lane change elapses,

wherein a driving unit filters the image data, then searches for the lane pairs, then obtains the time of lane change, and then generates a warning generating signal;

wherein the driving unit comprises;

an image organizer that receives the image data from the image photographing unit, filters the image data to extract candidate lane regions, and groups the candidate lane regions based on features of the extracted candidate lane regions;

a lane searching unit that searches for the lane pairs from the grouped candidate lane regions from the image organizer;

a lane tracking unit that searches for an optimal lane pair from a region of interest (ROI) based on the searched lane pairs, wherein if the optimal line pair is determined, a curved parameter or straight parameter is calculated and stored; and

a warning generating unit that obtains the time of lane change by converting the image coordinate into the real coordinate, and generates a warning generating signal as the time of lane change elapses;

wherein the image photographing unit comprises a camera that photographs the object and transfers the image data to the driving unit;

wherein the image organizer comprises a preprocessing unit that selects the ROI from the image data to search a lane from the ROI and groups a plurality of regions determined as one lane, filtering being performed according to a data application value of each pixel while moving a mask having a predetermined size in the ROI, and candidate lane regions being generated by grouping a lane pixel according to at least one of a size, a boundary point, a main axis, a sub-axis, or a region gradient;

wherein an image of the ROI is converted to have a black and white color, and the candidate lane regions are recognized as the one lane and grouped as one candidate lane group, and a gradient of the generated group and lower and upper intersection points of the generated group are stored in a flag variable as features;

wherein the lane searching unit searches for lane pairs each including two lines constituting one lane from the grouped regions and searches for an optimal lane pair from the lane pairs, the search for the lane pairs being performed by extracting two lanes determined as a pair by comparing features of left and right candidate lane groups with each other;

wherein in a history check, currently searched lane pairs are compared with previously search lane pairs, and a lane pair of the currently searched lane pairs matching with the previous lane pairs within a threshold range is determined as an optimal lane pair;

wherein the warning generating unit receives an optimal lane information from the lane searching unit or the lane tracking unit, calculates a lateral distance according to relationship between the lane pairs and the vehicle, and calculates the time of lane change according to the lateral distance between the lane pairs and the vehicle and a lateral speed of the vehicle, and if the time of lane change is within a predetermined range, the warning generating unit outputs the warning generating signal;

wherein the optimal lane pair comprises a left candidate lane, a right candidate lane, and a center candidate lane;

wherein the ROI includes a left ROI having a predetermined area ranging from a left peripheral region of the image on the left candidate lane and the center candidate lane and a right ROI having a predetermined area ranging from a right peripheral region of the image on the right candidate lane and the center candidate lane;

wherein central portions of the left ROI and the right ROI are partially overlapped;

wherein the obtaining of the time of lane change comprises;

selecting two points from a lane of the image coordinate;

converting the two points into the real coordinate by setting a coordinate of the vehicle as a zero point to obtain two real points; and

calculating the lateral distance between the lane pairs and the vehicle by determining whether any one of the lane pairs is a straight lane pair or a curved lane pair;

wherein, when the determined lane pair is the curved lane pair, the warning generating unit selects the two points from a lane under an intersection point between the running direction of the vehicle and the determined lane pair; and

wherein, when a running direction of the vehicle is aligned parallel to the straight lane pair while the curved lane pair starts after the straight lane pair is terminated, the calculating the lateral distance between the lane pairs and the vehicle is performed by converting the lane pair into the straight lane pair.

8. The lane departure warning method of claim 7, further comprising performing a calculation by converting the determined lane pair into a straight line when the determined lane pair is bent at an angle of 90°

.

9. The lane departure warning method of claim 7, wherein the obtaining of the time of lane change comprises converting the image coordinate into the real coordinate by taking a perspective into consideration.

10. The lane departure warning method of claim 7, wherein the lateral speed of the vehicle is calculated based on variations of a location of the vehicle in a plurality of frames or variations of the lateral distance between the lane pairs and the vehicle in a plurality of frames.