VEHICULAR CONTROL SYSTEM

• US 20180197024A1
• Filed: 03/05/2018
• Published: 07/12/2018
• Est. Priority Date: 02/27/2009
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Lynam, Niall R., Nix, Axel
• Granted Patent: US 10,839,233 B2
• Time in Patent Office: Days
• CPC Class Codes: B60Q 9/005 using a video camera
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 2001/1215 with information displays
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/605 the adjustment being automatic
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 40/02 related to ambient conditions
• CPC Class Codes: B60W 50/0098 Details of control systems ensuring comfort, safety or stability not otherwise provided for
• CPC Class Codes: G05D 1/0061 for transition from automatic pilot to manual pilot and vice versa
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A driver active safety control system for a vehicle includes a plurality of image capture sensors, a central control module and at least one radar sensor. The control module at least includes an image processor that processes image data captured by at least a forward viewing image capture sensor for automatic braking of the vehicle. The central control module receives vehicle data relating to operation of the vehicle, the vehicle data relating to at least one of (i) vehicle speed, (ii) vehicle steering, (iii) vehicle yaw rate, (iv) vehicle type and (v) vehicle acceleration. The central control module at least in part controls the vehicle responsive, at least in part, to processing of (i) vehicle data, (ii) image data, (iii) radar data and (iv) data from at least one of (a) the Car2Car communication system and (b) the Car2X communication system.

Claims

1. A vehicular control system, said vehicular control system comprising:

a plurality of image capture sensors disposed at a vehicle equipped with said vehicular control system and having respective fields of view exterior of the equipped vehicle;

wherein said plurality of image capture sensors comprises at least (i) a forward viewing image capture sensor disposed at the equipped vehicle and having a field of view at least forward of the equipped vehicle, (ii) a rearward viewing image capture sensor disposed at the equipped vehicle and having a field of view at least rearward of the equipped vehicle, (iii) a driver-side sideward viewing image capture sensor disposed at a driver side of the equipped vehicle and having a field of view at least sideward of the driver side of the equipped vehicle and (iv) a passenger-side sideward viewing image capture sensor disposed at a passenger side of the equipped vehicle and having a field of view at least sideward of the passenger side of the equipped vehicle;

a central control module;

wherein said central control module at least comprises an image processor;

wherein image data captured by said forward viewing image capture sensor and image data captured by at least one other image capture sensor of said plurality of image capture sensors is provided to said central control module;

wherein said image processor processes, via machine vision analysis, image data captured by at least said forward viewing image capture sensor for automatic braking of the equipped vehicle;

at least one radar sensor disposed at the equipped vehicle and sensing exterior the equipped vehicle;

wherein radar data sensed by said at least one radar sensor is provided to said central control module;

wherein said central control module receives data from at least one of (i) a Car2Car communication system and (ii) a Car2X communication system;

wherein said central control module receives vehicle data relating to operation of the equipped vehicle, said vehicle data relating to at least one of (i) vehicle speed, (ii) vehicle steering, (iii) vehicle yaw rate, (iv) vehicle type and (v) vehicle acceleration; and

wherein said central control module at least in part controls the equipped vehicle responsive, at least in part, to processing of (i) vehicle data, (ii) image data, (iii) radar data and (iv) data from at least one of (a) the Car2Car communication system and (b) the Car2X communication system.

2. The vehicular control system of claim 1, wherein said forward viewing image capture sensor is disposed in the equipped vehicle behind a windshield of the equipped vehicle and views forward through the windshield, and wherein said at least one radar sensor disposed at the equipped vehicle and sensing exterior the equipped vehicle comprises a front radar sensor mounted at a front portion of the equipped vehicle, and wherein said front radar sensor has a field of sensing at least forward of the equipped vehicle.

3. The vehicular control system of claim 2, wherein said central control module at least in part controls automatic braking of the equipped vehicle responsive at least in part to (i) said image processor processing image data captured by at least said windshield-mounted forward viewing image capture sensor and (ii) processing of radar data sensed by said front radar sensor.

4. The vehicular control system of claim 3, wherein said central control module at least in part controls automatic braking of the equipped vehicle responsive at least in part to said image processor processing image data captured by at least said windshield-mounted forward viewing image capture sensor and by the at least one other image capture sensor of said plurality of image capture sensors.

5. The vehicular control system of claim 4, wherein said central control module at least in part controls automatic braking of the equipped vehicle responsive at least in part to processing of radar data sensed by said front radar sensor and by at least one other radar sensor disposed at the equipped vehicle that is sensing exterior of the equipped vehicle.

6. The vehicular control system of claim 5, wherein said image processor and a fusion core are co-established on a common substrate.

7. The vehicular control system of claim 1, wherein said image processor processes image data captured by at least said forward viewing image capture sensor for object detection exterior of the equipped vehicle, and wherein, during a driving maneuver of the equipped vehicle and responsive to processing at said central control module of captured image data and sensed radar data, presence of an object exterior of the equipped vehicle is determined.

8. The vehicular control system of claim 1, wherein presence of an object exterior of the equipped vehicle is determined responsive at least in part to processing at said central control module of (i) captured image data and (ii) sensed radar data.

9. The vehicular control system of claim 1, wherein said at least one radar sensor disposed at the equipped vehicle and sensing exterior the equipped vehicle comprises at least (i) a front radar sensor mounted at a front portion of the equipped vehicle and having a field of sensing at least forward of the equipped vehicle, (ii) a right radar sensor mounted at a right side portion of the equipped vehicle and having a field of sensing at least encompassing a right traffic lane to the right of a traffic lane the equipped vehicle is travelling in and (iii) a left radar sensor mounted at a left side portion of the equipped vehicle and having a field of sensing at least encompassing a left traffic lane to the left of the traffic lane the equipped vehicle is travelling in.

10. The vehicular control system of claim 9, wherein at least one lidar sensor is disposed at the equipped vehicle and senses exterior the equipped vehicle, and wherein lidar data captured by said at least one lidar sensor is provided to and is processed at said central control module

11. The vehicular control system of claim 10, wherein said at least one lidar sensor comprises a scanning lidar sensor.

12. The vehicular control system of claim 10, wherein said at least one lidar sensor comprises a three dimensional sensing lidar sensor.

13. The vehicular control system of claim 12, wherein said at least one lidar sensor comprises an array of infrared emitting light emitters.

14. The vehicular control system of claim 9, wherein image data captured by said rearward viewing image capture sensor, by said driver-side sideward viewing image capture sensor and by said passenger-side sideward viewing image capture sensor is provided to said central control module, and wherein radar data sensed by said front radar sensor, by said right radar sensor and by said left radar sensor is provided to said central control module.

15. The vehicular control system of claim 14, wherein map data pertaining to a geographical location of the equipped vehicle is provided to said central control module, and wherein said central control module at least in part controls the equipped vehicle responsive to processing (i) vehicle data, (ii) image data, (iii) radar data, (iv) map data and (v) data from at least one of (a) the Car2Car communication system and (b) the Car2X communication system.

16. The vehicular control system of claim 15, wherein said image processor processes image data captured by at least two image capture sensors of said plurality of image capture sensors to determine a driving situation, and wherein, in determining said driving situation, processing of image data by said image processor is augmented by processing at said central control module of radar data.

17. The vehicular control system of claim 16, wherein captured image data is communicated from said at least two of said plurality of image capture sensors to said central control module via respective Ethernet links.

18. The vehicular control system of claim 14, wherein said central control module generates an output for at least one vehicle control.

19. The vehicular control system of claim 18, wherein said at least one vehicle control comprises at least one of (i) a vehicle body control, (ii) a vehicle chassis control, (iii) a vehicle supervisory control, (iv) a vehicle brake control, (v) a vehicle steering control and (vi) a vehicle engine control.

20. The vehicular control system of claim 18, wherein said central control module at least in part controls said at least one vehicle control responsive at least in part to image processing of image data captured by at least said forward viewing image capture sensor of said plurality of image capture sensors and responsive at least in part to processing of radar data sensed by at least said front radar sensor.

21. The vehicular control system of claim 14, wherein said forward viewing image capture sensor is disposed in the equipped vehicle behind a windshield of the equipped vehicle and views forward through the windshield.

22. The vehicular control system of claim 21, wherein said driver-side sideward viewing image capture sensor is disposed at a driver-side exterior rearview mirror assembly of the equipped vehicle, and wherein said passenger-side sideward viewing image capture sensor is disposed at a passenger-side exterior rearview mirror assembly of the equipped vehicle.

23. The vehicular control system of claim 21, wherein (i) said forward viewing image capture sensor of said plurality of image capture sensors comprises an at least 1 Megapixel image capture sensor, (ii) said rearward viewing image capture sensor of said plurality of image capture sensors comprises an at least 1 Megapixel image capture sensor, (iii) said driver-side sideward viewing image capture sensor of said plurality of image capture sensors comprises an at least 1 Megapixel image capture sensor and (iv) said passenger-side sideward viewing image capture sensor of said plurality of image capture sensors comprises an at least 1 Megapixel image capture sensor.

24. The vehicular control system of claim 1, wherein said central control module, responsive to map data pertaining to a geographical location of the equipped vehicle, at least in part controls the equipped vehicle.

25. A vehicular control system, said vehicular control system comprising:

a plurality of image capture sensors disposed at a vehicle equipped with said vehicular control system and having respective fields of view exterior of the equipped vehicle;

wherein said plurality of image capture sensors comprises at least (i) a forward viewing image capture sensor disposed at the equipped vehicle and having a field of view at least forward of the equipped vehicle, (ii) a rearward viewing image capture sensor disposed at the equipped vehicle and having a field of view at least rearward of the equipped vehicle, (iii) a driver-side sideward viewing image capture sensor disposed at a driver side of the equipped vehicle and having a field of view at least sideward of the driver side of the equipped vehicle and (iv) a passenger-side sideward viewing image capture sensor disposed at a passenger side of the equipped vehicle and having a field of view at least sideward of the passenger side of the equipped vehicle;

wherein (i) said forward viewing image capture sensor of said plurality of image capture sensors comprises an at least 1 Megapixel image capture sensor, (ii) said rearward viewing image capture sensor of said plurality of image capture sensors comprises an at least 1 Megapixel image capture sensor, (iii) said driver-side sideward viewing image capture sensor of said plurality of image capture sensors comprises an at least 1 Megapixel image capture sensor and (iv) said passenger-side sideward viewing image capture sensor of said plurality of image capture sensors comprises an at least 1 Megapixel image capture sensor;

wherein said forward viewing image capture sensor is disposed in the equipped vehicle behind a windshield of the equipped vehicle and views forward through the windshield;

a central control module;

wherein said central control module at least comprises an image processor;

wherein image data captured by said windshield-mounted forward viewing image capture sensor, by said driver-side sideward viewing image capture sensor, by said passenger-side sideward viewing image capture sensor and by said rearward viewing image capture sensor is provided to said central control module;

a plurality of radar sensors disposed at the equipped vehicle and sensing exterior the equipped vehicle;

wherein said plurality of radar sensors comprises at least (i) a right radar sensor mounted at a right side portion of the equipped vehicle and having a field of sensing at least encompassing a right traffic lane to the right of a traffic lane the equipped vehicle is travelling in, (ii) a left radar sensor mounted at a left side portion of the equipped vehicle and having a field of sensing at least encompassing a left traffic lane to the left of the traffic lane the equipped vehicle is travelling in and (iii) a front radar sensor mounted at a front portion of the equipped vehicle, and having a field of sensing at least forward of the equipped vehicle;

wherein radar data sensed by said front radar sensor, by said left side radar sensor and by said right side radar sensor is provided to said central control module;

wherein said central control module receives vehicle data relating to operation of the equipped vehicle, said vehicle data relating to at least one of (i) vehicle speed, (ii) vehicle steering, (iii) vehicle yaw rate, (iv) vehicle type and (v) vehicle acceleration;

wherein map data pertaining to a geographical location of the equipped vehicle is provided to said central control module, and wherein said central control module at least in part controls the equipped vehicle responsive to processing (i) vehicle data, (ii) image data, (iii) radar data and (iv) map data; and

wherein said central control module at least in part controls the equipped vehicle responsive at least in part to (i) said image processor processing image data captured by at least said windshield-mounted forward viewing image capture sensor and (ii) processing of radar data sensed by at least said front radar sensor of said plurality of radar sensors.

26. The vehicular control system of claim 25, wherein said central control module at least in part controls automatic braking of the equipped vehicle responsive at least in part to said image processor processing image data captured by said windshield-mounted forward viewing image capture sensor and responsive at least in part to processing of radar data sensed by at least said front radar sensor of said plurality of radar sensors.

27. The vehicular control system of claim 26, wherein said central control module at least in part controls automatic braking of the equipped vehicle responsive at least in part to said image processor processing image data captured by at least one other image capture sensor of said image capture sensors and responsive at least in part to processing of radar data sensed by at least one other radar sensor of said plurality of radar sensors.

28. The vehicular control system of claim 25, wherein said plurality of radar sensors comprises a rear radar sensor mounted at a rear portion of the equipped vehicle and having a field of sensing at least rearward of the equipped vehicle, and wherein radar data sensed by said rear radar sensor is provided to said central control module.

29. The vehicular control system of claim 28, wherein said central control module receives data from a Car2Car communication system and wherein said central control module at least in part controls the equipped vehicle responsive, at least in part, to processing of data from the Car2Car communication system.

30. The vehicular control system of claim 28, wherein said central control module receives data from a Car2X communication system, and wherein said central control module at least in part controls the equipped vehicle responsive, at least in part, to processing of data from the Car2X communication system.

31. The vehicular control system of claim 25, wherein said central control module at least in part controls at least one vehicle control responsive at least in part to image processing of image data captured by at least one image capture sensor of said plurality of image capture sensors and responsive at least in part to processing of radar data sensed by at least one radar sensor of said plurality of radar sensors.

32. The vehicular control system of claim 31, wherein said at least one vehicle control comprises at least one of (i) a vehicle body control, (ii) a vehicle chassis control, (iii) a vehicle supervisory control, (iv) a vehicle brake control, (v) a vehicle steering control and (vi) a vehicle engine control.

33. The vehicular control system of claim 25, wherein at least one lidar sensor is disposed at the equipped vehicle and senses exterior the equipped vehicle, and wherein lidar data captured by said at least one lidar sensor is provided to and is processed at said central control module, and wherein said central control module at least in part controls automatic braking responsive at least in part to processing of lidar data sensed by said at least one lidar sensor.

34. The vehicular control system of claim 33, wherein said at least one lidar sensor comprises a scanning lidar sensor.

35. The vehicular control system of claim 33, wherein said at least one lidar sensor comprises a three dimensional sensing lidar sensor.

36. The vehicular control system of claim 33, wherein said at least one lidar sensor comprises an array of infrared emitting light emitters.

37. A vehicular control system, said vehicular control system comprising:

a plurality of image capture sensors disposed at a vehicle equipped with said vehicular control system and having respective fields of view exterior of the equipped vehicle;

wherein said plurality of image capture sensors comprises at least (i) a forward viewing image capture sensor disposed at the equipped vehicle and having a field of view at least forward of the equipped vehicle, (ii) a rearward viewing image capture sensor disposed at the equipped vehicle and having a field of view at least rearward of the equipped vehicle, (iii) a driver-side sideward viewing image capture sensor disposed at a driver side of the equipped vehicle and having a field of view at least sideward of the driver side of the equipped vehicle and (iv) a passenger-side sideward viewing image capture sensor disposed at a passenger side of the equipped vehicle and having a field of view at least sideward of the passenger side of the equipped vehicle;

wherein (i) said forward viewing image capture sensor of said plurality of image capture sensors comprises an at least 1 Megapixel image capture sensor, (ii) said rearward viewing image capture sensor of said plurality of image capture sensors comprises an at least 1 Megapixel image capture sensor, (iii) said driver-side sideward viewing image capture sensor of said plurality of image capture sensors comprises an at least 1 Megapixel image capture sensor and (iv) said passenger-side sideward viewing image capture sensor of said plurality of image capture sensors comprises an at least 1 Megapixel image capture sensor;

wherein said forward viewing image capture sensor is disposed in the equipped vehicle behind a windshield of the equipped vehicle and views forward through the windshield;

a central control module;

wherein said central control module at least comprises an image processor;

wherein image data captured by said windshield-mounted forward viewing image capture sensor and image data captured by at least one other of said plurality of image capture sensors is provided to said central control module;

a plurality of radar sensors disposed at the equipped vehicle and sensing exterior the equipped vehicle;

wherein said plurality of radar sensors disposed at the equipped vehicle and sensing exterior the equipped vehicle comprises a front radar sensor mounted at a front portion of the equipped vehicle, and wherein said front radar sensor has a field of sensing at least forward of the equipped vehicle;

wherein radar data sensed by said front radar sensor and by at least one other radar sensor of said plurality of radar sensors is provided to said central control module;

wherein at least one lidar sensor is disposed at the equipped vehicle and senses exterior the equipped vehicle;

wherein lidar data captured by said at least one lidar sensor is provided to and is processed at said central control module;

wherein said central control module receives vehicle data relating to operation of the equipped vehicle, said vehicle data relating to at least one of (i) vehicle speed, (ii) vehicle steering, (iii) vehicle yaw rate, (iv) vehicle type and (v) vehicle acceleration; and

wherein said central control module at least in part controls the equipped vehicle responsive at least in part to (i) said image processor processing image data captured by said windshield-mounted forward viewing image capture sensor and by the at least one other image capture sensor of said plurality of image capture sensors, (ii) processing of radar data sensed by said front radar sensor and by the at least one other radar sensor of said plurality of radar sensors and (iii) processing of lidar data captured by said at least one lidar sensor.

38. The vehicular control system of claim 37, wherein said plurality of radar sensors comprises a rear radar sensor mounted at a rear portion of the equipped vehicle and having a field of sensing at least rearward of the equipped vehicle, and wherein radar data sensed by said rear radar sensor is provided to said central control module.

39. The vehicular control system of claim 38, wherein said central control module receives data from a Car2Car communication system, and wherein said central control module at least in part controls the equipped vehicle responsive, at least in part, to processing of data from the Car2Car communication system.

40. The vehicular control system of claim 38, wherein said central control module receives data from a Car2X communication system, and wherein said central control module at least in part controls the equipped vehicle responsive, at least in part, to processing of data from the Car2X communication system.

41. The vehicular control system of claim 38, wherein map data pertaining to a geographical location of the equipped vehicle is provided to said central control module, and wherein said central control module at least in part controls the equipped vehicle responsive to processing (i) vehicle data, (ii) image data, (iii) radar data, (iv) map data and (v) lidar data.

42. The vehicular control system of claim 37, wherein said central control module at least in part controls automatic braking of the equipped vehicle responsive at least in part to said image processor processing image data captured by said windshield-mounted forward viewing image capture sensor and responsive at least in part to processing of radar data sensed by at least said front radar sensor of said plurality of radar sensors.

43. The vehicular control system of claim 42, wherein said central control module at least in part controls automatic braking of the equipped vehicle responsive at least in part to said image processor processing image data captured by the at least one other image capture sensor of said plurality of image capture sensors and responsive at least in part to processing of radar data sensed by the at least one other radar sensor of said plurality of radar sensors.

44. The vehicular control system of claim 43, wherein said central control module at least in part controls automatic braking of the equipped vehicle responsive at least in part to processing of lidar data sensed by said at least one lidar sensor.

45. The vehicular control system of claim 37, wherein the at least one other radar sensor of said plurality of radar sensors comprises a left side radar sensor mounted at a left side portion of the equipped vehicle and having a field of sensing at least encompassing a left traffic lane to the left of a traffic lane the equipped vehicle is travelling in.

46. The vehicular control system of claim 37, wherein the at least one other radar sensor of said plurality of radar sensors comprises a right side radar sensor mounted at a right side portion of the equipped vehicle and having a field of sensing at least encompassing a right traffic lane to the right of a traffic lane the equipped vehicle is travelling in.

47. The vehicular control system of claim 37, wherein said at least one lidar sensor comprises a scanning lidar sensor.

48. The vehicular control system of claim 37, wherein said at least one lidar sensor comprises a three dimensional sensing lidar sensor.

49. The vehicular control system of claim 37, wherein said at least one lidar sensor comprises an array of infrared emitting light emitters.

50. The vehicular control system of claim 37, wherein said image processor processes image data captured by at least two image capture sensors of said plurality of image capture sensors to determine a driving situation, and wherein, in determining said driving situation, processing of image data by said image processor is augmented by processing at said central control module of radar data.

51. The vehicular control system of claim 50, wherein, in determining said driving situation, processing of image data by said image processor is augmented by processing at said central control module of lidar data.

52. The vehicular control system of claim 37, wherein said central control module at least in part controls at least one vehicle control responsive at least in part to image processing of image data captured by at least one image capture sensor of said plurality of image capture sensors and responsive at least in part to processing of radar data sensed by at least one radar sensor of said plurality of radar sensors, and wherein said at least one vehicle control comprises at least one of (i) a vehicle body control, (ii) a vehicle chassis control, (iii) a vehicle supervisory control, (iv) a vehicle brake control, (v) a vehicle steering control and (vi) a vehicle engine control.

53. The vehicular control system of claim 52, wherein said central control module at least in part controls the at least one vehicle control responsive at least in part to processing of lidar data sensed by said at least one lidar sensor.

54. The vehicular control system of claim 37, wherein the at least one other radar sensor of said plurality of radar sensors disposed at the equipped vehicle and sensing exterior the equipped vehicle comprises at least one of (i) a right radar sensor mounted at a right side portion of the equipped vehicle and having a field of sensing at least encompassing a right traffic lane to the right of a traffic lane the equipped vehicle is travelling in, (ii) a left radar sensor mounted at a left side portion of the equipped vehicle and having a field of sensing at least encompassing a left traffic lane to the left of the traffic lane the equipped vehicle is travelling in and (iii) a rear radar sensor mounted at a rear portion of the equipped vehicle and having a field of sensing at least rearward of the equipped vehicle.

55. The vehicular control system of claim 54, wherein the at least one other image capture sensor of said plurality of image capture sensors comprises at least one of said rearward viewing image capture sensor, said driver-side sideward viewing image capture sensor and said passenger-side sideward viewing image capture sensor.