Vehicular control system with central electronic control unit

• US 10,425,619 B2
• Filed: 08/30/2018
• Issued: 09/24/2019
• Est. Priority Date: 06/12/2009
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Dellantoni, Nikolaus, Schinkowitsch, Bernhard, Schoenekaes, Andre, Nix, Axel, Lynam, Niall R.
• Primary Examiner(s): Duley, Janese
• Application Number:
  US16/117,252
• Publication Number:
  US 20190014290A1
• Time in Patent Office: 390 Days
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/20 characterised by the type of display used
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 21/3679 Retrieval, searching and output of POI information, e.g. hotels, restaurants, shops, filling stations, parking facilities G01C21/3611 takes precedence
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A vehicular control system includes a plurality of cameras, a radar device and a central electronic control unit. The vehicular control system is operable to fuse image data captured by at least one of the cameras with radar data sensed by the radar device. The central electronic control unit is operable to at least partially control the vehicle. Threat recognition/evaluation by a threat recognizer/evaluator of the central electronic control unit and risk assessment by a risk assessor of the central electronic control unit is responsive, at least in part, to (i) image data captured by at least one of the cameras, (ii) radar data sensed by the radar device and (iii) map data associated with a current geographical location of the vehicle. The central electronic control unit, responsive at least in part to threat recognition/evaluation and risk assessment, controls braking of the equipped vehicle.

Claims

1. A vehicular control system, said vehicular control system comprising:

a plurality of cameras disposed at a vehicle equipped with said vehicular control system;

wherein each camera of said plurality of cameras captures image data in a respective field of view exterior of the equipped vehicle;

said plurality of cameras at least comprising a front camera disposed at a front portion of the equipped vehicle, a rear camera disposed at a rear portion of the equipped vehicle, a first side view camera at a left side portion of the equipped vehicle and a second side view camera at a right side portion of the equipped vehicle;

a radar device disposed at the equipped vehicle;

wherein said radar device senses radar data in a field of sensing exterior of the equipped vehicle;

a central electronic control unit, wherein said central electronic control unit comprises an image processor, and wherein image data captured by at least some of said plurality of cameras is processed by said image processor of said central electronic control unit;

wherein said vehicular control system is operable to fuse image data captured by at least one camera of said plurality of cameras with radar data sensed by said radar device;

wherein said central electronic control unit is operable to at least partially control the equipped vehicle as the equipped vehicle is driven along a road;

wherein said central electronic control unit includes a threat recognizer/evaluator and a risk assessor;

wherein threat recognition/evaluation by the threat recognizer/evaluator of said central electronic control unit and risk assessment by the risk assessor of said central electronic control unit is responsive, at least in part, to (i) image data captured by at least one camera of said plurality of cameras, (ii) radar data sensed by said radar device and (iii) map data associated with a current geographical location of the equipped vehicle; and

wherein said central electronic control unit, responsive at least in part to threat recognition/evaluation by the threat recognizer/evaluator of said central electronic control unit and risk assessment by the risk assessor of said central electronic control unit, controls braking of the equipped vehicle.

2. The vehicular control system of claim 1, wherein said front camera, said rear camera, said first side view camera and said second side view camera connect to said central electronic control unit via Ethernet cabling, and wherein captured image data is provided to said central electronic control unit via the Ethernet cabling.

3. The vehicular control system of claim 1, wherein said vehicular control system comprises a situation analysis vehicle control unit.

4. The vehicular control system of claim 1, wherein said image processor processes image data captured by said front camera for a lane keep assist system of the equipped vehicle.

5. The vehicular control system of claim 4, wherein said vehicular control system determines that the equipped vehicle is stopped at a cross traffic situation at least in part via said image processor processing image data captured by said front camera.

6. The vehicular control system of claim 1, wherein said vehicular control system determines a driving condition of the equipped vehicle at least in part via said image processor processing image data captured by said front camera.

7. The vehicular control system of claim 1, wherein said image processor processes image data captured by at least said front camera for at least one selected from the group consisting of (i) a pedestrian detection system of the equipped vehicle, (ii) a vehicle detection system of the equipped vehicle and (iii) a hazard detection system of the equipped vehicle.

8. The vehicular control system of claim 1, wherein said vehicular control system is operable to detect a stop sign at least in part via said image processor processing image data captured by said front camera.

9. The vehicular control system of claim 1, wherein said vehicular control system determines that the equipped vehicle is stopped at a cross traffic situation at least in part at least in part via said image processor processing image data captured by said front camera.

10. The vehicular control system of claim 1, wherein, responsive at least in part to image processing by said image processor of image data captured by at least one camera of said plurality of cameras, said vehicular control system determines that the equipped vehicle is at a parallel parking situation.

11. The vehicular control system of claim 10, wherein said vehicular control system determines that the equipped vehicle is at the parallel parking situation by determining that the equipped vehicle is driven alongside a vacant parking space and shifted into reverse to back into the vacant parking space.

12. The vehicular control system of claim 1, wherein a plurality of ultrasonic sensors is disposed at the equipped vehicle, and wherein each of said ultrasonic sensors senses ultrasonic data in a respective field of sensing exterior of the equipped vehicle, and wherein sensed ultrasonic data is provided to said central electronic control unit and is processed thereat when the equipped vehicle is being parked.

13. The vehicular control system of claim 1, wherein a plurality of radar devices is disposed at the equipped vehicle, each having a respect field of sensing exterior the equipped vehicle, and wherein the plurality of radar devices comprises said radar device.

14. The vehicular control system of claim 1, wherein said vehicular control system is at least in part responsive to a communication from another vehicle via at least one selected from the group consisting of (i) a car to car communication system and (ii) a car to X communication system.

15. The vehicular control system of claim 1, wherein, responsive at least in part to fusion of image data captured by at least one camera of said plurality of cameras with radar data sensed by said radar device, said vehicular control system controls steering of the equipped vehicle.

16. The vehicular control system of claim 1, wherein the current geographical location of the equipped vehicle is determined responsive to a global positioning system.

17. The vehicular control system of claim 1, wherein said map data associated with the current geographical location of the equipped vehicle comprises at least one selected from the group consisting of (i) longitude coordinates, (ii) latitude coordinates and (iii) altitude coordinates.

18. The vehicular control system of claim 1, wherein said vehicular control system comprises a firewall assuring that consumer imported or downloaded features do not interfere with or degrade vehicular functions or controls of the equipped vehicle.

19. The vehicular control system of claim 1, wherein said vehicular control system comprises a Communication Access Point (CAP) operable as a communication gateway.

20. The vehicular control system of claim 1, wherein said front camera, said rear camera, said first side view camera and said second side view camera are part of a surround vision system of the equipped vehicle.

21. The vehicular control system of claim 1, wherein said central electronic control unit controls steering of the equipped vehicle responsive, at least in part, to threat recognition/evaluation by the threat recognizer/evaluator of said central electronic control unit and risk assessment by the risk assessor of said central electronic control unit.

22. The vehicular control system of claim 21, wherein said central electronic control unit controls speed of the equipped vehicle responsive, at least in part, to threat recognition/evaluation by the threat recognizer/evaluator of said central electronic control unit and risk assessment by the risk assessor of said central electronic control unit.

23. The vehicular control system of claim 1, wherein the equipped vehicle comprises an autonomous vehicle.

24. The vehicular control system of claim 1, wherein said front camera disposed at the front portion of the equipped vehicle is disposed at an in-cabin side of a windshield of the equipped vehicle and views through the windshield forward of the equipped vehicle.

25. The vehicular control system of claim 1, wherein said map data associated with the current geographical location of the equipped vehicle comprises longitude and latitude data.

26. The vehicular control system of claim 1, wherein said map data associated with the current geographical location of the equipped vehicle comprises data relevant to the current geographical location.

27. The vehicular control system of claim 26, wherein the data relevant to the current geographical location comprises data related to the locale the equipped vehicle is travelling through or in.

28. A vehicular control system, said vehicular control system comprising:

a plurality of cameras disposed at a vehicle equipped with said vehicular control system;

wherein each camera of said plurality of cameras captures image data in a respective field of view exterior of the equipped vehicle;

said plurality of cameras at least comprising a front camera disposed at a front portion of the equipped vehicle, a rear camera disposed at a rear portion of the equipped vehicle, a first side view camera at a left side portion of the equipped vehicle and a second side view camera at a right side portion of the equipped vehicle;

a radar device disposed at the equipped vehicle;

wherein said radar device senses radar data in a field of sensing exterior of the equipped vehicle;

a central electronic control unit, wherein said central electronic control unit comprises an image processor, and wherein image data captured by at least some of said plurality of cameras is processed by said image processor of said central electronic control unit;

wherein said vehicular control system is operable to fuse image data captured by at least one camera of said plurality of cameras with radar data sensed by said radar device;

wherein said image processor processes image data captured at least by said front camera for a pedestrian detection system of the equipped vehicle;

wherein said central electronic control unit is operable to at least partially control the equipped vehicle as the equipped vehicle is driven along a road;

wherein said central electronic control unit includes a threat recognizer/evaluator and a risk assessor;

wherein threat recognition/evaluation by the threat recognizer/evaluator of said central electronic control unit and risk assessment by the risk assessor of said central electronic control unit is responsive, at least in part, to (i) image data captured by at least one camera of said plurality of cameras, (ii) radar data sensed by said radar device and (iii) map data associated with a current geographical location of the equipped vehicle; and

wherein said central electronic control unit, responsive at least in part to threat recognition/evaluation by the threat recognizer/evaluator of said central electronic control unit and risk assessment by the risk assessor of said central electronic control unit, controls at least one selected from the group consisting of (i) braking of the equipped vehicle, (ii) speed of the equipped vehicle and (iii) steering of the equipped vehicle.

29. The vehicular control system of claim 28, wherein captured image data is provided to said central electronic control unit via Ethernet cabling.

30. The vehicular control system of claim 29, wherein said image processor processes image data captured by said front camera for a lane keep assist system of the equipped vehicle.

31. The vehicular control system of claim 28, wherein said vehicular control system determines that the equipped vehicle is stopped at a cross traffic situation at least in part via said image processor processing image data captured by said front camera.

32. The vehicular control system of claim 28, wherein said vehicular control system determines a driving condition of the equipped vehicle at least in part via said image processor processing image data captured at least by said front camera.

33. The vehicular control system of claim 28, wherein said image processor processes image data captured at least by said front camera for a vehicle detection system of the equipped vehicle.

34. The vehicular control system of claim 33, wherein said vehicular control system is operable to detect a stop sign at least in part via said image processor processing image data captured by said front camera.

35. The vehicular control system of claim 28, wherein a plurality of ultrasonic sensors is disposed at the equipped vehicle, and wherein each of said ultrasonic sensors senses ultrasonic data in a respective field of sensing exterior of the equipped vehicle, and wherein sensed ultrasonic data is provided to said central electronic control unit and is processed thereat when the equipped vehicle is being parked.

36. The vehicular control system of claim 28, wherein a plurality of radar devices is disposed at the equipped vehicle, each having a respect field of sensing exterior the equipped vehicle, and wherein the plurality of radar devices comprises said radar device.

37. The vehicular control system of claim 36, wherein said vehicular control system is at least in part responsive to a communication from another vehicle via at least one selected from the group consisting of (i) a car to car communication system and (ii) a car to X communication system.

38. The vehicular control system of claim 28, wherein, responsive at least in part to fusion of image data captured by at least one camera of said plurality of cameras with radar data sensed by said radar device, said vehicular control system controls steering of the equipped vehicle.

39. The vehicular control system of claim 28, wherein the current geographical location of the equipped vehicle is determined responsive to a global positioning system.

40. The vehicular control system of claim 39, wherein said map data associated with the current geographical location of the equipped vehicle comprises at least one selected from the group consisting of (i) longitude coordinates, (ii) latitude coordinates and (iii) altitude coordinates.

41. The vehicular control system of claim 28, wherein said vehicular control system comprises a firewall assuring that consumer imported or downloaded features do not interfere with or degrade vehicular functions or controls of the equipped vehicle.

42. The vehicular control system of claim 41, wherein said vehicular control system comprises a Communication Access Point (CAP) operable as a communication gateway.

43. The vehicular control system of claim 28, wherein said front camera, said rear camera, said first side view camera and said second side view camera are part of a surround vision system of the equipped vehicle.

44. The vehicular control system of claim 28, wherein said central electronic control unit controls steering of the equipped vehicle responsive, at least in part, to threat recognition/evaluation by the threat recognizer/evaluator of said central electronic control unit and risk assessment by the risk assessor of said central electronic control unit.

45. The vehicular control system of claim 44, wherein said central electronic control unit controls speed of the equipped vehicle responsive, at least in part, to threat recognition/evaluation by the threat recognizer/evaluator of said central electronic control unit and risk assessment by the risk assessor of said central electronic control unit.

46. The vehicular control system of claim 45, wherein the equipped vehicle comprises an autonomous vehicle.

47. The vehicular control system of claim 28, wherein said front camera disposed at the front portion of the equipped vehicle is disposed at an in-cabin side of a windshield of the equipped vehicle and views through the windshield forward of the equipped vehicle.

48. The vehicular control system of claim 28, wherein said map data associated with the current geographical location of the equipped vehicle comprises longitude and latitude data.

49. The vehicular control system of claim 28, wherein said map data associated with the current geographical location of the equipped vehicle comprises data relevant to the current geographical location.

50. The vehicular control system of claim 49, wherein the data relevant to the current geographical location comprises data related to the locale the equipped vehicle is travelling through or in.

51. A vehicular control system, said vehicular control system comprising:

a plurality of cameras disposed at a vehicle equipped with said vehicular control system;

wherein each camera of said plurality of cameras captures image data in a respective field of view exterior of the equipped vehicle;

said plurality of cameras at least comprising a front camera disposed at a front portion of the equipped vehicle, a rear camera disposed at a rear portion of the equipped vehicle, a first side view camera at a left side portion of the equipped vehicle and a second side view camera at a right side portion of the equipped vehicle;

wherein said front camera disposed at the front portion of the equipped vehicle is disposed at an in-cabin side of a windshield of the equipped vehicle and views through the windshield forward of the equipped vehicle;

a radar device disposed at the equipped vehicle;

wherein said radar device senses radar data in a field of sensing exterior of the equipped vehicle;

a central electronic control unit, wherein said central electronic control unit comprises an image processor, and wherein image data captured by at least said front camera is processed by said image processor of said central electronic control unit;

wherein said vehicular control system is operable to fuse image data captured by at least said front camera with radar data sensed by said radar device;

wherein said image processor processes image data captured by at least said front camera for a pedestrian detection system of the equipped vehicle;

wherein said vehicular control system comprises a firewall assuring that consumer imported or downloaded features do not interfere with or degrade vehicular functions or controls of the equipped vehicle;

wherein said central electronic control unit is operable to at least partially control the equipped vehicle as the equipped vehicle is driven along a road;

wherein said central electronic control unit includes a threat recognizer/evaluator and a risk assessor;

wherein threat recognition/evaluation by the threat recognizer/evaluator of said central electronic control unit and risk assessment by the risk assessor of said central electronic control unit is responsive, at least in part, to (i) image data captured by at least said front camera, (ii) radar data sensed by said radar device and (iii) map data associated with a current geographical location of the equipped vehicle; and

wherein said central electronic control unit, responsive at least in part to threat recognition/evaluation by the threat recognizer/evaluator of said central electronic control unit and risk assessment by the risk assessor of said central electronic control unit, controls speed of the equipped vehicle.

52. The vehicular control system of claim 51, wherein said image processor processes image data captured by said front camera for a lane keep assist system of the equipped vehicle.

53. The vehicular control system of claim 51, wherein said vehicular control system determines that the equipped vehicle is stopped at a cross traffic situation at least in part via said image processor processing image data captured by said front camera.

54. The vehicular control system of claim 51, wherein said vehicular control system determines a driving condition of the equipped vehicle at least in part via said image processor processing image data captured at least by said front camera.

55. The vehicular control system of claim 51, wherein said image processor processes image data captured at least by said front camera for a vehicle detection system of the equipped vehicle.

56. The vehicular control system of claim 55, wherein said vehicular control system is operable to detect a stop sign at least in part via said image processor processing image data captured by said front camera.

57. The vehicular control system of claim 51, wherein a plurality of ultrasonic sensors is disposed at the equipped vehicle, and wherein each of said ultrasonic sensors senses ultrasonic data in a respective field of sensing exterior of the equipped vehicle, and wherein sensed ultrasonic data is provided to said central electronic control unit and is processed thereat when the equipped vehicle is being parked.

58. The vehicular control system of claim 51, wherein a plurality of radar devices is disposed at the equipped vehicle, each having a respect field of sensing exterior the equipped vehicle, and wherein the plurality of radar devices comprises said radar device.

59. The vehicular control system of claim 51, wherein said vehicular control system is at least in part responsive to a communication from another vehicle via at least one selected from the group consisting of (i) a car to car communication system and (ii) a car to X communication system.

60. The vehicular control system of claim 51, wherein, responsive at least in part to fusion of image data captured by said front camera with radar data sensed by said radar device, said vehicular control system controls steering of the equipped vehicle.

61. The vehicular control system of claim 60, wherein the current geographical location of the equipped vehicle is determined responsive to a global positioning system.

62. The vehicular control system of claim 61, wherein said map data associated with the current geographical location of the equipped vehicle comprises at least one selected from the group consisting of (i) longitude coordinates, (ii) latitude coordinates and (iii) altitude coordinates.

63. The vehicular control system of claim 51, wherein said vehicular control system comprises a Communication Access Point (CAP) operable as a communication gateway.

64. The vehicular control system of claim 51, wherein said central electronic control unit controls braking of the equipped vehicle responsive, at least in part, to threat recognition/evaluation by the threat recognizer/evaluator of said central electronic control unit and risk assessment by the risk assessor of said central electronic control unit.

65. The vehicular control system of claim 64, wherein said central electronic control unit controls steering of the equipped vehicle responsive, at least in part, to threat recognition/evaluation by the threat recognizer/evaluator of said central electronic control unit and risk assessment by the risk assessor of said central electronic control unit.

66. The vehicular control system of claim 65, wherein the equipped vehicle comprises an autonomous vehicle.

67. The vehicular control system of claim 51, wherein said map data associated with the current geographical location of the equipped vehicle comprises longitude and latitude data.

68. The vehicular control system of claim 51, wherein said map data associated with the current geographical location of the equipped vehicle comprises data relevant to the current geographical location.

69. The vehicular control system of claim 68, wherein the data relevant to the current geographical location comprises data related to the locale the equipped vehicle is travelling through or in.