Lane departure sensing method and apparatus using images that surround a vehicle

• US 9,056,630 B2
• Filed: 12/24/2009
• Issued: 06/16/2015
• Est. Priority Date: 05/19/2009
• Status: Active Grant

• Current Assignee: Beyond Corporation
• Original Assignee: Imagenext Co., Ltd. (CHEMTRONICS Co., Ltd.)
• Inventors: Han, Young In, Song, Young Gi, Back, Won In
• Primary Examiner(s): Ustaris, Joseph
• Assistant Examiner(s): VOLENTINE, REBECCA A
• Application Number:
  US13/321,518
• Publication Number:
  US 20120062745A1
• Time in Patent Office: 2,000 Days
• US Class Current: 1/1
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B62D 15/029 Steering assistants using warnings or proposing actions to the driver without influencing the steering system
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A method and an apparatus for sensing a lane departure using images that surround a vehicle. The lane departure sensing method includes receiving images that are taken during driving by a plurality of cameras installed in a car, through respective channels connected to the cameras; modifying the obtained images into top-view images to generate a plurality of modified images; compositing the plurality of modified images to generate images that surround a vehicle; extracting a lane from the images that surround the vehicle, and yielding the information on a distance between the vehicle and the extracted lane or the information on an angle between the travel direction of the vehicle and the lane, so as to determine a lane departure; and giving a warning if the lane departure is determined.

Claims

1. A lane departure sensing method using a vehicle surrounding image, comprising:

receiving images captured by a plurality of cameras installed to a car running, over respective channels connected to the cameras;

generating a plurality of revised images by revising the captured images into a top-view;

generating the vehicle surrounding image by synthesizing the plurality of the revised images;

extracting a lane from the vehicle surrounding image, and determining lane departure by calculating distance information between the vehicle and the extracted lane or angle information formed by a moving direction of the vehicle and the extracted lane; and

when determining the lane departure, issuing a warning,

wherein the plurality of the revised images is generated using a look up table,

wherein the generating of the vehicle surrounding image generates the vehicle surrounding image by overlaying the plurality of the revised images using a mask image which comprises region information per channel and weight information for pixels forming each region,

wherein overlapping regions between the plurality of the revised images are overlay-processed using the following equation;

l′

(t+1)=α

l₁(t)+(1−

α

)l₂(t), 0≦

α

≦

1

where l₁(t) and l₂(t) denote image information for the overlapping region input via two channels respectively, a denotes a weight for pixels in the overlapping region, and l′

(t+1) denotes the overlay-processed image information,

wherein the mask image is set to attain a Gradient weight for the pixels in the overlapping region,

wherein the plurality of cameras include a front camera, a rear camera, a left camera and a right camera,

wherein the front camera and the rear camera are installed such that their optical axis is parallel with the horizon, and the left camera and the right camera are installed perpendicularly to the ground, and

wherein the mask image includes a first region duplicately captured through the front camera and the left camera of a vehicle, a third region duplicately captured through the front camera and the right camera of the vehicle, a seventh region duplicately captured through the rear camera and the left camera of the vehicle, a ninth region duplicately captured through the rear camera and the right camera of the vehicle, a second region captured through the front camera of the vehicle, without being duplicately captured through any other camera, a fourth region captured through the left camera of the vehicle, without being duplicately captured through any other camera, a sixth region captured through the right camera of the vehicle, without being duplicately captured through any other camera, an eighth region captured through the rear camera of the vehicle, without being duplicately captured through any other camera, and a fifth region on which an image corresponding to the vehicle is displayed.

2. The lane departure sensing method using the vehicle surrounding image as in claim 1, wherein the determining of the lane departure comprises:

displaying a moving direction line indicating a moving direction of the vehicle, around the vehicle;

generating the distance information or the angle information by measuring a distance between the moving direction line and the extracted lane and measuring a crossing angle of the moving direction line and the extracted lane; and

when the distance information or the angle information exceeds a reference range, determining the lane departure.

3. A lane departure sensing apparatus using a vehicle surrounding image, comprising:

an image input part for receiving images captured by a plurality of cameras installed to a car running, over respective channels connected to the cameras;

an image processing part for generating a plurality of revised images by revising the captured images into a top-view;

an image synthesis part for generating the vehicle surrounding image by synthesizing the plurality of the revised images; and

a lane departure sensing part for extracting a lane from the vehicle surrounding image, calculating distance information of the vehicle and the extracted lane or angle information formed by a moving direction of the vehicle and the extracted lane, and determining lane departure based on the calculated distance information or the angle information,

wherein the image synthesis part generates the vehicle surrounding image by overlaying the plurality of the revised images using a mask image which comprises region information per channel and weight information for pixels forming each region,

wherein the plurality of the revised images is generated using a look up table,

wherein overlapping regions between the plurality of the revised images are overlay-processed using the following equation;

l′

(t+1)=α

l₁(t)+(1−

α

)l₂(t), 0≦

α

≦

1

where l₁(t) and l₂(t) denote image information for the overlapping region input via two channels respectively, a denotes a weight for pixels in the overlapping region, and l′

(t+1) denotes the overlay-processed image information,

wherein the mask image is set to attain a Gradient weight for the pixels in the overlapping region,

wherein the plurality of cameras include a front camera, a rear camera, a left camera and a right camera,

wherein the front camera and the rear camera are installed such that their optical axis is parallel with the horizon, and the left camera and the right camera are installed perpendicularly to the ground, and

wherein the mask image includes a first region duplicately captured through the front camera and the left camera of a vehicle, a third region duplicately captured through the front camera and the right camera of the vehicle, a seventh region duplicately captured through the rear camera and the left camera of the vehicle, a ninth region duplicately captured through the rear camera and the right camera of the vehicle, a second region captured through the front camera of the vehicle, without being duplicately captured through any other camera, a fourth region captured through the left camera of the vehicle, without being duplicately captured through any other camera, a sixth region captured through the right camera of the vehicle, without being duplicately captured through any other camera, an eighth region captured through the rear camera of the vehicle, without being duplicately captured through any other camera, and a fifth region on which an image corresponding to the vehicle is displayed.

4. The lane departure sensing apparatus using the vehicle surrounding image as in claim 3, further comprising:

a warning generation part for issuing a warning when determining the lane departure.

5. A lane departure sensing system using a vehicle surrounding image, comprising:

a plurality of cameras installed to a vehicle which is moving and outputting captured images over respective channels;

an image generating apparatus for generating a plurality of revised images by revising the captured input images into a top-view, and generating the vehicle surrounding image by synthesizing the plurality of the revised images;

a lane departure sensing apparatus for extracting a lane from the vehicle surrounding image, calculating distance information between the vehicle and the extracted lane or angle information formed by a moving direction of the vehicle and the extracted lane, determining lane departure from the distance information or the angle information, and issuing a warning when determining the lane departure; and

a display apparatus for displaying the surrounding image or a warning screen,

wherein the image generating apparatus generates the vehicle surrounding image by overlaying the plurality of the revised images using a mask image which comprises region information per channel and weight information for pixels forming each region,

wherein the plurality of the revised images is generated using a look up table,

wherein overlapping regions between the plurality of the revised images are overlay-processed using the following equation;

l′

(t+1)=α

l₁(t)+(1−

α

)l₂(t), 0≦

α

≦

1

where l₁(t) and l₂(t) denote image information for the overlapping region input via two channels respectively, α

denotes a weight for pixels in the overlapping region, and l′

(t+1) denotes the overlay-processed image information,

wherein the mask image is set to attain a Gradient weight for the pixels in the overlapping region,

wherein the plurality of cameras include a front camera, a rear camera, a left camera and a right camera,

wherein the front camera and the rear camera are installed such that their optical axis is parallel with the horizon, and the left camera and the right camera are installed perpendicularly to the ground,

wherein the mask image includes a first region duplicately captured through the front camera and the left camera of a vehicle, a third region duplicately captured through the front camera and the right camera of the vehicle, a seventh region duplicately captured through the rear camera and the left camera of the vehicle, a ninth region duplicately captured through the rear camera and the right camera of the vehicle, a second region captured through the front camera of the vehicle, without being duplicately captured through any other camera, a fourth region captured through the left camera of the vehicle, without being duplicately captured through any other camera, a sixth region captured through the right camera of the vehicle, without being duplicately captured through any other camera, an eighth region captured through the rear camera of the vehicle, without being duplicately captured through any other camera, and a fifth region on which an image corresponding to the vehicle is displayed.