Apparatus and method for generating driver assistance information of traveling vehicle

• US 7,710,291 B2
• Filed: 07/06/2007
• Issued: 05/04/2010
• Est. Priority Date: 07/06/2006
• Status: Active Grant

• Current Assignee: Postech Academy-Industry Foundation, Samsung Electronics Co. Ltd.
• Original Assignee: Postech Academy-Industry Foundation, Samsung Electronics Co. Ltd.
• Inventors: Oh, Se-Young, Kim, Sam-Yong, Ryu, Young-Woo, Kang, Jeong-Gwan, Kim, Kwang-Soo, Park, Sang-Cheol, Kim, Jin-Won
• Primary Examiner(s): Tweel, Jr.; John A
• Application Number:
  US11/774,253
• Publication Number:
  US 20080055114A1
• Time in Patent Office: 1,033 Days
• Field of Search: 340/995.1, 340/435, 340/436, 340/901, 340/903, 340/988, 340/465, 340/937, 701/41
• US Class Current: 340/995.1
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/23 with a predetermined field of view
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G01S 15/86 Combinations of sonar systems with lidar systems; Combinations of sonar systems with systems not using wave reflection
• CPC Class Codes: G01S 15/931 of land vehicles
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

A method for generating driver assistance information of a traveling vehicle. The method can restructure traveling environments into an image based on a sensor system capable of removing blind spots around a traveling vehicle, and a scheme capable of exactly detecting a lane and a vehicle, and give a collision warning when there is a possibility of collision. Further, the method integrates recognition results of front/rear vehicles based on a lane recognition method applicable to various road environments, including the shape, curvature and loss of a lane, improves the stability of lane recognition, and distinguishes a solid line from a dot line, so it is possible to provide a driver with the subdivided degrees of danger. In addition, when a vehicle deviates from a lane, a voice warning may be output and the position of the vehicle for a road is restructured as an image.

Claims

1. An apparatus for generating driver assistance information of a traveling vehicle, the apparatus comprising:

at least one image receiver installed on the traveling vehicle to photograph front/rear images, wherein the image receiver provides the photographed front/rear images;

at least one ultrasonic signal receiver installed on the traveling vehicle, wherein the ultrasonic signal receiver emits ultrasonic signals, and computes distance information based on return time of the ultrasonic signals after being reflected from another vehicle;

a lane recognizer for detecting a lane from the front/rear images, determining whether the detected lane is one of a solid line and a dot line, and synthesizing recognition results of vehicles to generate an image of a road;

a vehicle recognizer for setting a road area as an area of interest by using the lane recognition results of the lane recognizer, detecting a vehicle by using an image and ultrasonic signals, detecting vehicles by using a distance obtained from the ultrasonic signals received from the ultrasonic signal receiver, determining a traveling tendency of a vehicle based on changes in the distance, and outputting a position of the vehicle in a vehicle coordinate system; and

an environment restructuring unit for generating the driver assistance information by applying the vehicle detected by the vehicle recognizer to the image restructured by the lane recognizer.

2. The apparatus as claimed in claim 1, further comprising a display unit for displaying the driver assistance information generated by the environment restructuring unit.

3. The apparatus as claimed in claim 1, wherein, when a front/rear image is received from the image receiver, the lane recognizer extracts lane candidates according to changes in brightness of the image, generates a histogram of the extracted lane candidates, determines a position of a lane adjacent to the traveling vehicle, computes a curvature of the lane candidates located in a long distance, and recognizes a long distance lane by using the computed curvature, thereby detecting a vehicle from the front/rear image, where a long distance is a distance greater than a preset distance in order to determine long distance and short distance.

4. The apparatus as claimed in claim 3, wherein, when a curvature computed in a current image frame changes more than a threshold value as compared to a curvature computed in a previous frame in the recognition of the long distance lane through the computed curvature, a curvature increasing step-by-step from the current computed curvature in a set of lane curvatures is used instead of using the curvature computed in the current image frame.

5. The apparatus as claimed in claim 1, wherein the lane recognizer judges brightness for determining whether the lane is a solid line or a dot line by computing a Solid Rate value of a ratio of areas R and L_i, R denoting an area including a lane, and L_i denoting a part in which the lane is actually drawn in the area R, the Solid Rate value being computed using

$\mathrm{SolidRate}=\frac{\sum _{}^{}}{}$

i N ⁢

⁢

L i R ,

where the lane is a solid line when the Solid Rate value is greater than a threshold, and the lane is a dot line when the Solid Rate value is less than or equal to the threshold.

6. The apparatus as claimed in claim 1, wherein, the lane recognizer recognizes that another lane exists adjacent to a lane along which the traveling vehicle is traveling, and generates a solid line in said another lane when a lane to be used for restructuring of the image is a dot line.

7. A method for generating driver assistance information by a traveling vehicle, the traveling vehicle including at least one image receiver installed on the traveling vehicle to photograph front/rear images, the image receiver providing the photographed front/rear images, and at least one ultrasonic signal receiver installed on the traveling vehicle, the ultrasonic signal receiver emitting ultrasonic signals, and computing distance information based on return time of the ultrasonic signals after being reflected from another vehicle, the method comprising the steps of:

detecting a lane from the front/rear images;

determining whether the detected lane is one of a solid line and a dot line;

synthesizing recognition results of front/rear vehicles to generate an image of a road;

setting a road area as an area of interest by using the lane recognition results, and detecting a vehicle by using an image and ultrasonic signals;

detecting right/left vehicles by using a distance obtained from the ultrasonic signals received from the ultrasonic signal receiver, determining a traveling tendency of a vehicle based on changes in the distance, and outputting a position of the vehicle in a vehicle coordinate system; and

generating the driver assistance information by applying the detected vehicle to a restructured image.

8. The method as claimed in claim 7, further comprising displaying the generated driver assistance information.

9. The method as claimed in claim 7, wherein the step of detecting the lane from the front/rear-images comprises:

extracting lane candidates according to changes in brightness of the image when the front or rear-image is received from the image receiver;

generating a histogram of the extracted lane candidates;

determining a position of a lane adjacent to the traveling vehicle, and computing a curvature of the lane candidates located in a long distance; and

recognizing a long distance lane by using the computed curvature, where a long distance is a distance greater than a preset distance in order to determine long distance and short distance.

10. The method as claimed in claim 9, wherein, in the step of recognizing the long distance lane, when a curvature computed in a current image frame changes more than a threshold value as compared to a curvature computed in a previous frame in the recognition of the long distance lane through the computed curvature, a curvature increasing step-by-step from the current computed curvature in a set of lane curvatures is used instead of using the curvature computed in the current image frame.

11. The method as claimed in claim 7, wherein, in the step of determining whether the detected lane is a solid line or a dot line, brightness for determining whether the lane is a solid line or a dot line is judged by computing a Solid Rate value of a ratio of areas R and L_i, R denoting an area including a lane, and L_i denoting a part in which the lane is actually drawn in the area R, the Solid Rate value being computed using

$\mathrm{SolidRate}=\frac{\sum _{}^{}}{}$

i N ⁢

⁢

L i R ,

where the lane is a solid line when the Solid Rate value is greater than a threshold, and the lane is a dot line when the Solid Rate value is less than or equal to the threshold.

12. The method as claimed in claim 7, wherein, after the step of determining when the detected lane is one of a solid line and a dot line, recognizing that another lane exists adjacent to a lane along which the traveling vehicle is traveling, and generating a solid line in said another lane when a lane to be used for restructuring of the image is a dot line.