Driver assistance system for a vehicle

• US 10,099,610 B2
• Filed: 10/10/2016
• Issued: 10/16/2018
• Est. Priority Date: 07/31/2001
• Status: Expired due to Term

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Schofield, Kenneth
• Primary Examiner(s): Swarthout, Brent
• Application Number:
  US15/289,341
• Publication Number:
  US 20170028916A1
• Time in Patent Office: 736 Days
• Field of Search: None
• CPC Class Codes: B60Q 1/346 with automatic actuation
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60T 2201/08 Lane monitoring; Lane Keeping Systems
• CPC Class Codes: B60T 2201/089 using optical detection
• CPC Class Codes: B62D 15/029 Steering assistants using warnings or proposing actions to the driver without influencing the steering system
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 2013/9315 Monitoring blind spots
• CPC Class Codes: G01S 2013/9316 combined with communication equipment with other vehicles or with base stations
• CPC Class Codes: G01S 2013/932 using own vehicle data, e.g. ground speed, steering wheel direction
• CPC Class Codes: G01S 2013/9323 Alternative operation using light waves
• CPC Class Codes: G01S 2013/9324 Alternative operation using ultrasonic waves
• CPC Class Codes: G01S 2013/9325 for inter-vehicle distance regulation, e.g. navigating in platoons
• CPC Class Codes: G01S 2013/93274 on the side of the vehicles
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/163 involving continuous checking
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A driver assistance system for a vehicle includes a vision system, a sensing system and a control. The vision system includes a camera and the sensing system includes a radar sensor. Image data captured by the camera is provided to the control and is processed by an image processor of the control. Responsive to image processing of captured image data, lane markers on the road being traveled along by the equipped vehicle are detected and the control determines a lane being traveled by the equipped vehicle. Radar data generated by the radar sensor is provided to the control, which receives vehicle data relating to the equipped vehicle via a vehicle bus of the equipped vehicle. Responsive at least in part to processing of generated radar data and captured image data, the control detects another vehicle present on the road being traveled along by the equipped vehicle.

Claims

1. A driver assistance system for a vehicle, said driver assistance system comprising:

a vision system comprising a first camera disposed at a vehicle equipped with said driver assistance system, said first camera having a field of view forward of the equipped vehicle that encompasses a road being traveled along by the equipped vehicle;

wherein the road has at least three lanes comprising (i) a first lane along which the vehicle is traveling, (ii) a second lane immediately adjacent the first lane, and (iii) a third lane immediately adjacent the second lane at the opposite side of the second lane from the first lane;

a control disposed at or in the equipped vehicle and comprising a data processor;

wherein said data processor comprises an image processor;

wherein image data captured by said first camera is provided to said control and is processed by said image processor;

wherein said data processor processes data at a processing speed of at least 30 MIPS;

wherein, responsive to processing by said image processor of image data captured by said first camera, lane markers on the road being traveled along by the equipped vehicle are detected;

wherein, responsive to processing by said image processor of image data captured by said first camera, said control determines the first, second and third lanes of the road being traveled by the equipped vehicle;

a sensing system operable to detect other vehicles present exterior the equipped vehicle;

wherein said sensing system comprises a second camera disposed at the equipped vehicle and having a field of view at least rearward and sideward of the equipped vehicle as the equipped vehicle travels along the road;

wherein said sensing system further comprises a radar sensor disposed at the equipped vehicle and having a field of sensing at least rearward and sideward of the equipped vehicle as the equipped vehicle travels along the road;

wherein image data captured by said second camera is provided to said control;

wherein radar data generated by said radar sensor is provided to said control;

wherein said control receives vehicle data relating to the equipped vehicle via a vehicle bus of the equipped vehicle;

wherein, responsive at least in part to processing at said data processor of radar data generated by said radar sensor and of image data captured by said second camera, said control detects another vehicle present on the road being traveled along by the equipped vehicle and approaching the equipped vehicle from its rear;

wherein, with the equipped vehicle traveling along the first lane, and responsive at least in part to processing at said data processor of radar data generated by said radar sensor and of image data captured by said second camera, said control determines that the other vehicle is travelling in a next but one lane to the lane the second lane or in the third lane of the road along which the equipped vehicle is travelling;

wherein, responsive to determination that the other vehicle is approaching the equipped vehicle and is traveling in the second lane, and responsive to said control detecting a lane change maneuver of the equipped vehicle toward the second lane, said control generates an alert that a lane change by the equipped vehicle from the first lane to the second lane is not safe; and

wherein, responsive to determination that the other vehicle is approaching the equipped vehicle and is traveling in the third lane, and responsive to said control detecting a lane change maneuver of the equipped vehicle toward the second lane, said control determines that a lane change from the first lane to the second lane can proceed without hazard of impact with the other vehicle detected travelling in the third lane.

2. The driver assistance system of claim 1, wherein said control is operable to detect a lane change maneuver of the equipped vehicle to a lane adjacent to the lane the equipped vehicle is travelling along.

3. The driver assistance system of claim 2, wherein said radar sensor comprises a Doppler radar sensor.

4. The driver assistance system of claim 2, wherein, responsive at least in part to processing at said data processor of image data captured by said second camera, said control develops a vehicle path history of the other vehicle.

5. The driver assistance system of claim 4, wherein determination by said control that the lane change can proceed without hazard of impact with the other vehicle detected travelling in the third lane utilizes said vehicle path history.

6. The driver assistance system of claim 1, wherein said radar sensor has a field of sensing to a side of and rearward of the equipped vehicle as the equipped vehicle travels along the road.

7. The driver assistance system of claim 6, wherein said control is operable to detect a lane change maneuver of the equipped vehicle from the first lane to the second lane.

8. The driver assistance system of claim 7, wherein detection of the lane change maneuver by said control is based at least in part on vehicle steering data received at said control via said bus.

9. The driver assistance system of claim 7, wherein, responsive to said control detecting the lane change maneuver of the equipped vehicle, said control determines that the lane change can proceed without hazard of impact with the other vehicle detected approaching from the rear in the third lane.

10. The driver assistance system of claim 9, wherein said vehicle bus system comprises at least one of (a) a CAN bus of the vehicle and (b) a LIN bus of the vehicle.

11. The driver assistance system of claim 10, wherein the other vehicle detected in the third lane is overtaking the equipped vehicle.

12. The driver assistance system of claim 11, wherein said control determines a position history of the other vehicle.

13. The driver assistance system of claim 7, wherein detection of the lane change maneuver by said control is irrespective of activation by a driver of the equipped vehicle of a turn signal indicator of the equipped vehicle.

14. The driver assistance system of claim 1, wherein said vision system further comprises at least one other camera disposed at the equipped vehicle and having a field of view exterior the equipped vehicle, and wherein said sensing system further comprises at least one other radar sensor disposed at the equipped vehicle and having a field of sensing exterior the equipped vehicle, and wherein image data captured by said at least one other camera is provided to and processed at said control, and wherein radar data generated by said at least one other radar sensor is provided to and processed at said control.

15. The driver assistance system of claim 1, wherein said camera comprises a forward-viewing camera disposed in an interior cabin of the equipped vehicle at and behind a windshield of the equipped vehicle and viewing through the windshield to capture image data at least forward of the equipped vehicle.

16. The driver assistance system of claim 15, wherein, responsive at least in part to processing of radar data generated by said radar sensor and of image data captured by said camera, said control determines a hazard of collision of the equipped vehicle with another vehicle.

17. The driver assistance system of claim 16, wherein, responsive at least in part to said control determining said hazard of collision with the other vehicle, said control controls a system of the equipped vehicle to mitigate potential collision with the other vehicle that is in hazard of collision with the equipped vehicle.

18. The driver assistance system of claim 17, wherein, responsive at least in part to said control determining said hazard of collision, said control controls activation of at least one of (i) a horn system of the equipped vehicle to mitigate potential collision with the other vehicle that is in hazard of collision with the equipped vehicle and (ii) a lighting system of the equipped vehicle to mitigate potential collision with the other vehicle that is in hazard of collision with the equipped vehicle.

19. The driver assistance system of claim 17, wherein, responsive at least in part to said control determining said hazard of collision, said control controls a wireless transmission system of the equipped vehicle to transmit a safety warning to the other vehicle that is in hazard of collision with the equipped vehicle.

20. A driver assistance system for a vehicle, said driver assistance system comprising:

a vision system comprising a first camera disposed at a vehicle equipped with said driver assistance system, said first camera having a field of view forward of the equipped vehicle that encompasses a road being traveled along by the equipped vehicle;

wherein the road has at least three lanes comprising (i) a first lane along which the vehicle is traveling, (ii) a second lane immediately adjacent the first lane, and (iii) a third lane immediately adjacent the second lane at the opposite side of the second lane from the first lane;

wherein said first camera comprises a forward-viewing camera disposed in an interior cabin of the equipped vehicle at and behind a windshield of the equipped vehicle and viewing through the windshield to capture image data at least forward of the equipped vehicle;

a control disposed at or in the equipped vehicle and comprising a data processor;

wherein said data processor comprises an image processor;

wherein image data captured by said first camera is provided to said control and is processed by said image processor;

wherein said data processor processes data at a processing speed of at least 30 MIPS;

wherein, responsive to processing by said image processor of image data captured by said first camera, lane markers on the road being traveled along by the equipped vehicle are detected;

wherein, responsive to processing by said image processor of image data captured by said first camera, said control determines the first, second and third lanes of the road being traveled by the equipped vehicle;

a sensing system operable to detect other vehicles present exterior the equipped vehicle;

wherein said sensing system comprises a first radar sensor disposed at the equipped vehicle and having a field of sensing exterior of the equipped vehicle as the equipped vehicle travels along the road;

wherein radar data generated by said first radar sensor is provided to said control;

wherein said control receives vehicle data relating to the equipped vehicle via a vehicle bus of the equipped vehicle;

wherein said vision system further comprises a second camera disposed at the equipped vehicle and having a field of view at least rearward and sideward of the equipped vehicle as the equipped vehicle travels along the road, and wherein said sensing system further comprises a second radar sensor disposed at the equipped vehicle and having a field of sensing at least rearward and sideward the equipped vehicle as the equipped vehicle travels along the road, and wherein image data captured by said second camera is provided to and processed at said control, and wherein radar data generated by said second radar sensor is provided to and processed at said control;

wherein, responsive at least in part to processing at said data processor of radar data generated by at least one of said first and second radar sensors of said sensing system and of image data captured by at least one of said first and second cameras of said vision system, said control determines a hazard of collision of the equipped vehicle with another vehicle;

wherein, with the equipped vehicle traveling along the first lane, and responsive at least in part to processing at said data processor of radar data generated by said radar sensor and of image data captured by said second camera, said control determines that the other vehicle is travelling in the second lane or in the third lane of the road along which the equipped vehicle is travelling;

wherein, responsive to determination that the other vehicle is approaching the equipped vehicle and is traveling in the second lane, and responsive to said control detecting a lane change maneuver of the equipped vehicle toward the second lane, said control generates an alert that a lane change by the equipped vehicle from the first lane to the second lane is not safe;

wherein, responsive to determination that the other vehicle is approaching the equipped vehicle and is traveling in the third lane, and responsive to said control detecting a lane change maneuver of the equipped vehicle toward the second lane, said control determines that a lane change from the first lane to the second lane can proceed without hazard of impact with the other vehicle detected travelling in the third lane; and

wherein, responsive at least in part to said control determining said hazard of collision with the other vehicle, said control controls a system of the equipped vehicle to mitigate potential collision with the other vehicle.

21. The driver assistance system of claim 20, wherein, responsive at least in part to said control determining said hazard of collision, said control controls activation of at least one of (i) a horn system of the equipped vehicle to mitigate potential collision with the other vehicle that is in hazard of collision with the equipped vehicle and (ii) a lighting system of the equipped vehicle to mitigate potential collision with the other vehicle.

22. The driver assistance system of claim 20, wherein, responsive at least in part to said control determining said hazard of collision, said control controls a wireless transmission system of the equipped vehicle to transmit a safety warning to the other vehicle.

23. The driver assistance system of claim 20, wherein the other vehicle is approaching the equipped vehicle from rearward of the equipped vehicle, and wherein said control is operable to detect a lane change maneuver of the equipped vehicle to the second lane the equipped vehicle is travelling along, and wherein detection of the lane change maneuver by said control is based at least in part on vehicle steering data received at said control via said bus.

24. The driver assistance system of claim 23, wherein radar data generated by at least one of said first and second radar sensors of said sensing system and image data captured by at least one of said first and second cameras of said vision system are fused at said control.

25. The driver assistance system of claim 23, wherein radar data generated by at least one of said first and second radar sensors of said sensing system and image data captured by said first camera of said vision system are fused at said control.

26. The driver assistance system of claim 23, wherein said first radar sensor has a field of sensing to a side of and rearward of the equipped vehicle as the equipped vehicle travels along the road, and wherein radar data generated by said first radar sensor of said sensing system and image data captured by at least one of said first and second cameras of said vision system are fused at said control.

27. The driver assistance system of claim 23, wherein, responsive at least in part to processing at said data processor of at least one of (i) radar data generated by at least one of said first and second radar sensors of said sensing system and (ii) image data captured by at least one of said first and second cameras of said vision system, said control determines that the other vehicle is travelling in the third lane.

28. A driver assistance system for a vehicle, said driver assistance system comprising:

a vision system comprising a first camera disposed at a vehicle equipped with said driver assistance system, said first camera having a field of view that encompasses a road being traveled along by the equipped vehicle;

wherein said first camera comprises a forward-viewing camera disposed in an interior cabin of the equipped vehicle at and behind a windshield of the equipped vehicle and viewing through the windshield to capture image data at least forward of the equipped vehicle;

wherein the road has at least three lanes comprising (i) a first lane along which the vehicle is traveling, (ii) a second lane immediately adjacent the first lane, and (iii) a third lane immediately adjacent the second lane at the opposite side of the second lane from the first lane;

a control disposed at or in the equipped vehicle and comprising a data processor;

wherein said data processor comprises an image processor;

wherein image data captured by said first camera is provided to said control and is processed by said image processor;

wherein said data processor processes data at a processing speed of at least 30 MIPS;

wherein, responsive to processing by said image processor of image data captured by said first camera, lane markers on the road being traveled along by the equipped vehicle are detected;

wherein, responsive to processing by said image processor of image data captured by said first camera, said control determines the first, second and third lanes of the road being traveled by the equipped vehicle;

a sensing system operable to detect other vehicles present exterior the equipped vehicle;

wherein said sensing system comprises a first radar sensor disposed at the equipped vehicle and having a field of sensing exterior of the equipped vehicle as the equipped vehicle travels along the road;

wherein radar data generated by said first radar sensor is provided to said control;

wherein said control receives vehicle data relating to the equipped vehicle via a vehicle bus of the equipped vehicle;

wherein said first radar sensor has a field of sensing at least to a side of and rearward of the equipped vehicle as the equipped vehicle travels along the road;

wherein said sensing system further comprises a second camera disposed at the equipped vehicle and having a field of view at least rearward and sideward of the equipped vehicle as the equipped vehicle travels along the road;

wherein image data captured by said second camera is provided to said control;

wherein, responsive at least in part to processing at said data processor of radar data generated by said first radar sensor and of image data captured by said second camera, said control detects another vehicle present on the road being traveled along by the equipped vehicle that is approaching the equipped vehicle from rearward of the equipped vehicle;

wherein said control is operable to detect a lane change maneuver of the equipped vehicle toward the second lane;

wherein detection of the lane change maneuver by said control is based at least in part on vehicle steering data received at said control via said bus;

wherein, responsive to said control detecting a lane change maneuver of the equipped vehicle, said control determines that the lane change can proceed without hazard of impact with the other vehicle approaching the equipped vehicle from rearward of the equipped vehicle;

wherein, with the equipped vehicle traveling along the first lane, and responsive at least in part to processing at said data processor of radar data generated by said first radar sensor and of image data captured by said second camera, said control determines that the other vehicle is travelling in the second lane or in the third lane of the road along which the equipped vehicle is travelling;

wherein, responsive to determination that the other vehicle is approaching the equipped vehicle and is traveling in the second lane, and responsive to said control detecting a lane change maneuver of the equipped vehicle toward the second lane, said control generates an alert that a lane change by the equipped vehicle from the first lane to the second lane is not safe; and

wherein, responsive to determination that the other vehicle is approaching the equipped vehicle and is traveling in the third lane, and responsive to said control detecting a lane change maneuver of the equipped vehicle toward the second lane, said control determines that a lane change from the first lane to the second lane can proceed without hazard of impact with the other vehicle detected travelling in the third lane.

29. The driver assistance system of claim 28, wherein said vehicle bus system comprises a CAN bus of the equipped vehicle.

30. The driver assistance system of claim 28, wherein said control determines a position history of the other vehicle approaching the equipped vehicle from rearward of the equipped vehicle.

31. The driver assistance system of claim 28, wherein detection of the lane change maneuver by said control is irrespective of activation by a driver of the equipped vehicle of a turn signal indicator of the equipped vehicle.

32. The driver assistance system of claim 28, wherein said sensing system further comprises a second radar sensor disposed at the equipped vehicle and having a field of sensing exterior the equipped vehicle, and wherein image data captured by said second camera is provided to and processed at said control, and wherein radar data generated by said second radar sensor is provided to and processed at said control.

33. The driver assistance system of claim 32, wherein, responsive at least in part to processing of radar data generated by at least one of said first and second radar sensors of said sensing system and of image data captured by at least one of said first and second cameras of said vision system, said control determines a hazard of collision of the equipped vehicle with the other vehicle approaching the equipped vehicle from rearward of the equipped vehicle.

34. The driver assistance system of claim 33, wherein, responsive at least in part to said control determining said hazard of collision with the other vehicle approaching the equipped vehicle from rearward of the equipped vehicle, said control controls a system of the equipped vehicle to mitigate potential collision with the other vehicle approaching the equipped vehicle from rearward of the equipped vehicle.

35. The driver assistance system of claim 34, wherein radar data generated by at least one of said first and second radar sensors of said sensing system and image data captured by at least one of said first and second cameras of said vision system are fused at said control.

36. The driver assistance system of claim 34, wherein radar data generated by at least one of said first and second radar sensors of said sensing system and image data captured by said first camera of said vision system are fused at said control.

37. The driver assistance system of claim 34, wherein radar data generated by said first radar sensor of said sensing system and image data captured by at least one of said first and second cameras of said vision system are fused at said control.