Driver assistance apparatus capable of performing distance detection and vehicle including the same

• US 9,308,917 B2
• Filed: 03/27/2015
• Issued: 04/12/2016
• Est. Priority Date: 05/28/2014
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Oh, Sangguel, Kim, Ikkyu
• Primary Examiner(s): Tarcza, Thomas
• Assistant Examiner(s): Dunn, Alex C
• Application Number:
  US14/671,190
• Publication Number:
  US 20150344032A1
• Time in Patent Office: 382 Days
• Field of Search: 701/37
• US Class Current: 1/1
• CPC Class Codes: B60G 17/0165 to an external condition, e.g. rough road surface, side wind
• CPC Class Codes: B60G 17/018 characterised by the use of a specific signal treatment or control method
• CPC Class Codes: B60G 2400/821 Uneven, rough road sensing affecting vehicle body vibration
• CPC Class Codes: B60G 2401/142 Visual Display Camera, e.g. LCD
• CPC Class Codes: B60T 2201/022 Collision avoidance systems
• CPC Class Codes: B60T 2260/06 Active Suspension System
• CPC Class Codes: B60T 7/22 initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle , or by means of contactless obstacle detectors mounted on the vehicle
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/184 with wheel brakes
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 10/22 including control of suspension systems
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2554/801 Lateral distance
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/16 Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
• CPC Class Codes: B62D 15/025 Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation B62D1/28 takes precedence
• CPC Class Codes: B62D 6/00 Arrangements for automatically controlling steering depending on driving conditions sensed and responded to, e.g. control circuits means for initiating a change in direction B62D1/00; steering valves B62D5/06; combined with means for inclining the vehicle body or wheels on bends B62D9/00
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30261 : Obstacle
• CPC Class Codes: G06T 5/50 : using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/593 : from stereo images
• CPC Class Codes: G06T 7/85 : Stereo camera calibration
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 13/204 : using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/211 : using temporal multiplexing
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 : Calibration of cameras
• CPC Class Codes: H04N 13/271 : wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0085 : Motion estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0092 : Image segmentation from stereoscopic image signals

Abstract

A driver assistance apparatus and a vehicle including the same are disclosed. The driver assistance apparatus includes a stereo camera and a processor to perform a calibration based on first regions that include objects for vehicle structures or external to the vehicle in stereo images acquired by the stereo camera in a calibration mode and to detect a distance to an object ahead of the vehicle based on second regions not including the objects for the vehicle structures in the stereo images acquired by the stereo camera in a normal mode. Consequently, it is possible to accurately perform distance detection based on images photographed by the stereo camera.

Claims

1. A driver assistance apparatus comprising:

a stereo camera to acquire stereo images; and

a processor in communication with the stereo camera, the processor configured to perform calibration based on first regions of the stereo images, the first regions comprising one of vehicle structure objects or objects of structures external to a vehicle in a calibration mode and to detect a distance to an object ahead of the vehicle based on second regions not comprising one of the vehicle structure objects or the objects of structures external to a vehicle in a normal operational mode.

2. The driver assistance apparatus according to claim 1, wherein, in the calibration mode, the processor calculates a disparity between the first regions in the stereo images and compares the calculated disparity with a pre-stored reference disparity to calculate a calibration value.

3. The driver assistance apparatus according to claim 1, wherein, during execution of the normal mode and after completion of the calibration mode, the processor calibrates the second regions in the stereo images using the calibration value calculated in the calibration mode, performs disparity calculation based on the calibrated second regions in the stereo images, performs object detection and verification for the second regions based on disparity information by the disparity calculation, and detects a distance to the detected object.

4. The driver assistance apparatus according to claim 1, wherein the processor generates a control signal to control at least one of a steering drive unit, a brake drive unit, a power source drive unit, or a suspension drive unit in the vehicle based on the detected distance to the object ahead of the vehicle.

5. The driver assistance apparatus according to claim 1, wherein the vehicle structure is a portion of a body of the vehicle and comprises at least one of a character line, a hood emblem, and a hood edge line of the vehicle.

6. The driver assistance apparatus according to claim 1, wherein the structure outside the vehicle comprises at least one of a traffic sign, a signal light, and a streetlight.

7. The driver assistance apparatus according to claim 1, wherein the processor controls the calibration mode to be executed at the time of starting the vehicle, at the time of manipulating a user control, when external impact applied to the vehicle is equal to or greater than a predetermined level, or at the time of temporarily stopping the vehicle during travel of the vehicle.

8. The driver assistance apparatus according to claim 1, further comprising at least one selected from between:

a display unit to display an indicator indicating the calibration mode during execution of the calibration mode; and

an audio output unit to output a sound indicating the calibration mode during execution of the calibration mode.

9. The driver assistance apparatus according to claim 1, further comprising a display unit to display at least one of an indicator indicating a calibration range or a calibration value, information relating to progress time of the calibration mode, and information relating to remaining time of the calibration mode during execution of the calibration mode.

10. The driver assistance apparatus according to claim 3, further comprising:

an interface unit to receive sensor information of the vehicle, wherein if an error between the distance to the object detected based on the sensor information of the vehicle and the distance to the object detected based on the stereo images is equal to or greater than a predetermined value, the processor controls the calibration mode to be re-executed or the calibration value to be adjusted in the normal mode.

11. The driver assistance apparatus according to claim 10, further comprising at least one of a display unit and an audio output unit, and wherein if an error between the distance to the object detected based on the sensor information of the vehicle and the distance to the object detected based on the stereo images is equal to or greater than a predetermined value, the processor controls a notification message to be output through at least one selected from between the display unit and the audio output unit or releases control of the vehicle.

12. A vehicle comprising:

a steering drive unit to drive a steering apparatus;

a brake drive unit to drive a brake apparatus;

a power source drive unit to drive a power source;

a suspension drive unit to drive a suspension apparatus;

a stereo camera to acquire stereo images; and

a processor in communication with the stereo camera, the processor configured to perform calibration based on first regions of the stereo images, the first regions comprising one of vehicle structure objects or objects of structures external to a vehicle in a calibration mode and to detect a distance to an object ahead of the vehicle based on second regions not comprising one of the vehicle structure objects or the objects of structures external to a vehicle in a normal operational mode; and

a controller to generate a control signal to control at least one of the steering drive unit, the brake drive unit, the power source drive unit, and the suspension drive unit based on the detected distance to the object ahead of the vehicle.

13. The vehicle according to claim 12, wherein, in the calibration mode, the processor calculates a disparity between the first regions in the stereo images and compares the calculated disparity with a pre-stored reference disparity to calculate a calibration value.

14. The vehicle according to claim 12, wherein, during execution of the normal mode and after completion of the calibration mode, the processor calibrates the second regions in the stereo images using the calibration value calculated in the calibration mode, performs disparity calculation based on the calibrated second regions in the stereo images, performs object detection and verification for the second regions based on disparity information by the disparity calculation, and detects a distance to the detected object.

15. The vehicle according to claim 12, wherein the processor controls the calibration mode to be executed at the time of starting the vehicle, at the time of manipulating a user control, when external impact applied to the vehicle is equal to or greater than a predetermined level, or at the time of temporarily stopping the vehicle during travel of the vehicle.

16. The vehicle according to claim 12, further comprising a display unit to display at least one of an indicator indicating a calibration range or a calibration value, information regarding progress time of the calibration mode, or information relating to remaining time of the calibration mode during execution of the calibration mode.

17. The vehicle according to claim 14, further comprising:

an interface unit to receive sensor information of the vehicle, wherein if an error between the distance to the object detected based on the sensor information of the vehicle and the distance to the object detected based on the stereo images is equal to or greater than a predetermined value, the processor controls the calibration mode to be re-executed or the calibration value to be adjusted in the normal mode.

18. The vehicle according to claim 17, further comprising at least one selected from between a display unit and an audio output unit, and wherein if an error between the distance to the object detected based on the sensor information of the vehicle and the distance to the object detected based on the stereo images is equal to or greater than a predetermined value, the processor controls a notification message to be output through at least one of the display unit and the audio output unit or releases control of the vehicle.

19. The driver assistance apparatus according to claim 12, further comprising:

an interface unit to receive sensor information of the vehicle, wherein if an error between the distance to the object detected based on the sensor information of the vehicle and the distance to the object detected based on the stereo images is equal to or greater than a predetermined value, the processor controls the calibration mode to be re-executed or the calibration value to be adjusted in the normal mode.