Driver assist system with image processing and wireless communication

• US 9,919,705 B2
• Filed: 09/28/2015
• Issued: 03/20/2018
• Est. Priority Date: 10/27/2011
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Ihlenburg, Joern, Gieseke, Achim, Thaler, Bernhard, Pflug, Goerg
• Primary Examiner(s): Melton, Todd
• Assistant Examiner(s): Roberson, Jason R
• Application Number:
  US14/867,069
• Publication Number:
  US 20160016583A1
• Time in Patent Office: 904 Days
• Field of Search: None
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2400/00 Indexing codes relating to detected, measured or calculated conditions or factors
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/801 Lateral distance
• CPC Class Codes: B60W 2556/00 Input parameters relating to data
• CPC Class Codes: B60W 2556/65 Data transmitted between vehicles
• CPC Class Codes: B60W 2710/18 Braking system
• CPC Class Codes: B60W 2710/20 Steering systems
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

A driver assist system for a vehicle includes a camera, a receiver and a control. The camera has a field of view exterior of the vehicle and is operable to capture image data. The receiver is operable to receive a wireless communication from a communication device disposed at another vehicle. The control includes a data processor that processes captured image data to detect objects. The wireless communication includes a car-to-car communication. The driver assist system, responsive at least in part to processing by the data processor of captured image data and to the wireless communication received by the receiver, determines a potential hazard existing in a forward path of travel of the equipped vehicle. The control, responsive at least in part to such determination, controls at least one vehicle function of the equipped vehicle to mitigate the potential hazard in the forward path of travel of the equipped vehicle.

Claims

1. A driver assist system for a vehicle, said driver assist system comprising:

a camera disposed at a vehicle equipped with said driver assist system, said camera having a field of view exterior of the equipped vehicle and operable to capture image data;

a receiver disposed at the equipped vehicle and operable to receive a wireless communication;

wherein the wireless communication is associated with and emanates from a communication device disposed at another vehicle that is forward of the equipped vehicle along a road being traveled by the equipped vehicle and outside the field of view of said camera;

a control comprising a data processor;

wherein said data processor processes image data captured by said camera to detect objects within the field of view of said camera;

wherein said wireless communication comprises a car-to-car communication;

wherein said driver assist system, when a potential hazard exists in a forward path of travel of the equipped vehicle and outside of the field of view of said camera, determines the potential hazard existing in the forward path of travel of the equipped vehicle responsive at least in part to said wireless communication received by said receiver;

wherein said driver assist system, responsive to determination that the potential hazard will move to be within the field of view of said camera, processes image data captured by said camera to determine when the potential hazard is within the field of view of said camera; and

wherein said control, responsive at least in part to determination that the potential hazard is in the forward path of travel of the equipped vehicle and within the field of view of said camera, controls at least one vehicle function of the equipped vehicle to mitigate the potential hazard in the forward path of travel of the equipped vehicle.

2. The driver assist system of claim 1, wherein said at least one vehicle function of the equipped vehicle comprises a braking system of the equipped vehicle.

3. The driver assist system of claim 1, wherein said at least one vehicle function of the equipped vehicle comprises a steering system of the equipped vehicle.

4. The driver assist system of claim 1, wherein said at least one vehicle function of the equipped vehicle comprises at least two vehicle functions of the equipped vehicle, said two vehicle functions of the equipped vehicle comprising a braking system of the equipped vehicle and a steering system of the equipped vehicle.

5. The driver assist system of claim 1, wherein said wireless communication comprises a communication pertaining to a road condition ahead of the equipped vehicle.

6. The driver assist system of claim 5, wherein the road condition wirelessly communicated to the equipped vehicle comprises the potential hazard.

7. The driver assist system of claim 1, wherein said control alerts the driver of the equipped vehicle of the potential hazard ahead of the equipped vehicle.

8. The driver assist system of claim 1, wherein the other vehicle is equipped with a driver assist system.

9. The driver assist system of claim 8, wherein, when the equipped vehicle is in car-to-car communication range of the other vehicle that is equipped with a driver assist system, wireless communication between the vehicles is automatically initialized.

10. The driver assist system of claim 9, wherein said wireless communication comprises time coded time of flight communication.

11. The driver assist system of claim 1, wherein at least one radar sensor is disposed at the equipped vehicle and wherein said radar sensor senses exterior of the equipped vehicle and captures radar data, and wherein said driver assist system processes, using said data processor, radar data captured by said radar sensor to assist in determining the potential hazard existing in the forward path of travel of the equipped vehicle.

12. The driver assist system of claim 1, wherein at least one lidar sensor is disposed at the equipped vehicle and wherein said lidar sensor senses exterior of the equipped vehicle and captures lidar data, and wherein said driver assist system processes, using said data processor, lidar data captured by said lidar sensor to assist in determining the potential hazard existing in the forward path of travel of the equipped vehicle.

13. The driver assist system of claim 1, wherein said driver assist system, responsive at least in part to said processing of image data by said data processor, determines that a detected object is within a threshold distance of the equipped vehicle.

14. The driver assist system of claim 13, wherein said driver assist system adjusts said threshold distance responsive to an environment in which the equipped vehicle is operating.

15. The driver assist system of claim 14, wherein said driver assist system determines the environment in which the equipped vehicle is operating at least in part based on the geographic location of the equipped vehicle as determined by a global positioning system.

16. A driver assist system for a vehicle, said driver assist system comprising:

a camera disposed at a vehicle equipped with said driver assist system, said camera having a field of view exterior of the equipped vehicle and operable to capture image data;

a receiver disposed at the equipped vehicle and operable to receive a wireless communication;

wherein the wireless communication is associated with and emanates from a communication device disposed at another vehicle that is forward of the equipped vehicle along a road being traveled by the equipped vehicle and outside the field of view of said camera;

a control comprising a data processor;

wherein said data processor processes image data captured by said camera to detect objects within the field of view of said camera;

wherein said wireless communication comprises a car-to-car communication;

wherein said driver assist system, when a potential hazard exists in a forward path of travel of the equipped vehicle and outside of the field of view of said camera, determines the potential hazard existing in the forward path of travel of the equipped vehicle responsive at least in part to said to said wireless communication received by said receiver;

wherein said driver assist system, responsive to determination that the potential hazard will move to be within the field of view of said camera, processes image data captured by said camera to determine when the potential hazard is within the field of view of said camera;

wherein said control, responsive at least in part to determination that the potential hazard existing is in the forward path of travel of the equipped vehicle, controls at least one vehicle function of the equipped vehicle to mitigate the potential hazard in the forward path of travel of the equipped vehicle;

wherein said at least one vehicle function of the equipped vehicle comprises a braking system of the equipped vehicle;

wherein said wireless communication comprises a communication pertaining to a road condition ahead of the equipped vehicle; and

wherein said driver assist system, responsive at least in part to said processing of image data by said data processor, is operable to determine that a detected object is within a threshold distance of the equipped vehicle.

17. The driver assist system of claim 16, wherein said driver assist system adjusts said threshold distance responsive to an environment in which the equipped vehicle is operating and wherein said driver assist system determines the environment in which the equipped vehicle is operating at least in part based on the geographic location of the equipped vehicle as determined by a global positioning system.

18. A driver assist system for a vehicle, said driver assist system comprising:

a camera disposed at a vehicle equipped with said driver assist system, said camera having a field of view exterior of the equipped vehicle and operable to capture image data;

wherein at least one radar sensor is disposed at the equipped vehicle and wherein said radar sensor senses exterior of the equipped vehicle and captures radar data;

a receiver disposed at the equipped vehicle and operable to receive a wireless communication;

wherein the wireless communication is associated with and emanates from a communication device disposed at another vehicle that is forward of the equipped vehicle along a road being traveled by the equipped vehicle and outside the field of view of said camera;

a control comprising a data processor;

wherein said data processor processes image data captured by said camera to detect objects within the field of view of said camera;

wherein said wireless communication comprises a car-to-car communication;

wherein said driver assist system, when a potential hazard exists in a forward path of travel of the equipped vehicle and outside of the field of view of said camera, determines the potential hazard existing in the forward path of travel of the equipped vehicle responsive at least in part to processing by said data processor of radar data captured by said radar sensor and to said wireless communication received by said receiver;

wherein said driver assist system, responsive to determination that the potential hazard will move to be within the field of view of said camera, processes image data captured by said camera to determine when the potential hazard is within the field of view of said camera; and

wherein said control, responsive at least in part to determination that the potential hazard is in the forward path of travel of the equipped vehicle, controls at least one vehicle function of the equipped vehicle to mitigate the potential hazard in the forward path of travel of the equipped vehicle;

wherein said at least one vehicle function of the equipped vehicle comprises a braking system of the equipped vehicle;

wherein said wireless communication comprises a communication pertaining to a road condition ahead of the equipped vehicle; and

wherein said driver assist system, responsive at least in part to said processing of image data by said data processor, is operable to determine that a detected object is within a threshold distance of the equipped vehicle.

19. The driver assist system of claim 18, wherein said driver assist system adjusts said threshold distance responsive to an environment in which the equipped vehicle is operating and wherein said driver assist system determines the environment in which the equipped vehicle is operating at least in part based on the geographic location of the equipped vehicle as determined by a global positioning system.

20. The driver assist system of claim 18, wherein, when the equipped vehicle is in car-to-car communication range of the other vehicle that is equipped with a driver assist system, wireless communication between the vehicles is automatically initialized.