Vehicle control using sensing and communication systems

• US 10,493,899 B2
• Filed: 01/24/2017
• Issued: 12/03/2019
• Est. Priority Date: 04/03/2015
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Solar, Martin, Van Dan Elzen, Christopher L.
• Primary Examiner(s): Nolan, Peter D
• Application Number:
  US15/413,463
• Publication Number:
  US 20170129391A1
• Time in Patent Office: 1,043 Days
• Field of Search: None
• CPC Class Codes: B60Q 1/0023 the element being a sensor, e.g. distance sensor, camera
• CPC Class Codes: B60Q 1/143 combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers
• CPC Class Codes: B60Q 2300/132 Pitch
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60Q 2300/45 Special conditions, e.g. pedestrians, road signs or potential dangers
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2556/65 Data transmitted between vehicles
• CPC Class Codes: B60W 30/14 Adaptive cruise control
• CPC Class Codes: G08G 1/161 Decentralised systems, e.g. inter-vehicle communication
• CPC Class Codes: G08G 1/163 involving continuous checking
• CPC Class Codes: G08G 1/164 Centralised systems, e.g. external to vehicles
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

A vehicular control system includes a control and a plurality of sensors that include at least a camera and a 3D point-cloud LIDAR sensor. The control is operable to process captured image data to determine presence of another vehicle and to process captured 3D point-cloud LIDAR data to determine presence of another vehicle. A receiver at the equipped vehicle is operable to receive a wireless communication of information pertaining to other traffic participants in the vicinity of the equipped vehicle. The control is operable to process wirelessly-communicated information pertaining to other traffic participants in the vicinity of the equipped vehicle. The control, responsive at least in part to processing of at least one of (i) received image data, (ii) received 3D point-cloud LIDAR data and (iii) received information pertaining to other traffic participants in the vicinity of the equipped vehicle, controls at least one vehicle function of the equipped vehicle.

Claims

1. A vehicular control system, said vehicular control system comprising:

a plurality of sensors disposed at a vehicle equipped with said vehicular control system;

wherein said plurality of sensors at least comprises (i) a camera having a field of view exterior of the equipped vehicle, said camera capturing image data, and (ii) a 3D point-cloud LIDAR sensor having a field of sensing exterior of the equipped vehicle, said 3D point-cloud LIDAR sensor capturing 3D point-cloud LIDAR data;

wherein the field of view of said camera encompasses at least one selected from the group consisting of (i) a road being traveled by the equipped vehicle and (ii) another road parallel to the road being traveled by the equipped vehicle;

wherein the field of sensing of said 3D point-cloud LIDAR sensor encompasses at least one selected from the group consisting of (i) the road being traveled by the equipped vehicle, and (ii) the other road parallel to the road being traveled by the equipped vehicle;

a control;

wherein image data captured by said camera is provided to said control;

wherein said control comprises an image processor that processes image data captured by said camera to determine presence of vehicles present exterior of the equipped vehicle and in the field of view of said camera and approaching the equipped vehicle on the road being traveled by the equipped vehicle or on the other road parallel to the road being traveled by the equipped vehicle;

wherein 3D point-cloud LIDAR data captured by said 3D point-cloud LIDAR sensor is provided to said control;

wherein said control comprises a data processor that processes 3D point-cloud LIDAR data captured by said 3D point-cloud LIDAR sensor to determine presence of vehicles present exterior of the equipped vehicle and in the field of sensing of said 3D point-cloud LIDAR sensor and approaching the equipped vehicle on the road being traveled by the equipped vehicle or on the other road parallel to the road being traveled by the equipped vehicle;

a receiver disposed at the equipped vehicle and operable to receive a wireless communication of information pertaining to traffic participants in the vicinity of the equipped vehicle;

wherein wirelessly-communicated information pertaining to traffic participants in the vicinity of the equipped vehicle that is received at said receiver is provided to said control;

wherein said control processes the wirelessly-communicated information pertaining to traffic participants in the vicinity of the equipped vehicle;

wherein said control, responsive at least in part to received information pertaining to at least one traffic participant that is in the vicinity of the equipped vehicle and that is located outside the field of view of said camera and outside the field of sensing of said 3D point-cloud LIDAR sensor, controls at least one vehicle function of the equipped vehicle;

wherein, when the at least one traffic participant is in the field of view of said camera or in the field of sensing of said 3D point-cloud LIDAR sensor, and responsive at least in part to one selected from the group consisting of (i) processing at said control of image data captured by said camera and (ii) processing at said control of 3D point-cloud LIDAR data captured by said 3D point-cloud LIDAR sensor, said control controls the at least one vehicle function of the equipped vehicle; and

wherein, when the at least one traffic participant is in the field of view of said camera or in the field of sensing of said 3D point-cloud LIDAR sensor, and when said control is controlling the at least one vehicle function responsive at least in part to said one selected from the group consisting of (i) processing of image data captured by said camera and (ii) processing of 3D point-cloud LIDAR data captured by said 3D point-cloud LIDAR sensor, said control does not control the at least one vehicle function responsive to received information pertaining to the at least one traffic participant.

2. The vehicular control system of claim 1, wherein said control determines, via received information wirelessly communicated from the at least one traffic participant to the equipped vehicle, that the at least one traffic participant is not on the road that is being travelled by the equipped vehicle.

3. The vehicular control system of claim 2, wherein said control determines, via information vehicle-to-vehicle (V2V) wirelessly communicated from the at least one traffic participant to the equipped vehicle, that the at least one traffic participant is on a road that is not the road being travelled by the equipped vehicle and is approaching an intersection ahead of the equipped vehicle.

4. The vehicular control system of claim 2, wherein said control determines, via information vehicle-to-vehicle (V2V) wirelessly communicated from the at least one traffic participant to the equipped vehicle, that the at least one traffic participant is approaching an intersection ahead of the equipped vehicle.

5. The vehicular control system of claim 1, wherein said camera comprises a forward-viewing camera.

6. The vehicular control system of claim 5, wherein said 3D point-cloud LIDAR sensor comprises a forward-sensing 3D point-cloud LIDAR sensor.

7. The vehicular control system of claim 1, wherein said wireless communication comprises a vehicle-to-vehicle (V2V) communication via a Wi-Fi communication link.

8. The vehicular control system of claim 1, wherein said wireless communication comprises a communication using cell towers.

9. The vehicular control system of claim 1, wherein said at least one vehicle function of the equipped vehicle comprises a headlighting function of the equipped vehicle.

10. The vehicular control system of claim 1, wherein said at least one vehicle function of the equipped vehicle comprises an adaptive cruise control function of the equipped vehicle.

11. The vehicular control system of claim 1, wherein said receiver is operable to receive wireless communication of information via at least one of (i) a vehicle-to-vehicle (V2V) communication system and (ii) a vehicle-to-infrastructure communication system.

12. The vehicular control system of claim 1, wherein said 3D point-cloud LIDAR sensor comprises a solid-state 3D point-cloud LIDAR sensor.

13. The vehicular control system of claim 1, wherein said camera comprises a photosensor array having at least one million photosensor elements arranged in a two-dimensional array of at least 640 columns and at least 480 rows.

14. The vehicular control system of claim 1, wherein said image processor comprises an image processing chip that utilizes object detection software in processing image data captured by said camera for detecting objects or other vehicles or pedestrians in the field of view of said camera.

15. The vehicular control system of claim 1, wherein said receiver receives wireless communication of information from at least one of (i) another vehicle and (ii) a mobile device of a pedestrian.

16. The vehicular control system of claim 1, wherein said control determines, via information vehicle-to-vehicle (V2V) wirelessly communicated from the at least one traffic participant to the equipped vehicle, that the at least one traffic participant is approaching up a hill ahead of the equipped vehicle and is not yet in the field of view of said camera.

17. The vehicular control system of claim 1, wherein said control determines, via information vehicle-to-vehicle (V2V) wirelessly communicated from the at least one traffic participant to the equipped vehicle, that the at least one traffic participant is traveling along a road that is generally parallel to the road traveled by the equipped vehicle and is not yet in the field of view of said camera.

18. The vehicular control system of claim 1, wherein said control determines, via information vehicle-to-vehicle (V2V) wirelessly communicated from the at least one traffic participant to the equipped vehicle, that the at least one traffic participant is ahead of the equipped vehicle when the at least one traffic participant is outside of the field of view of said camera.

19. The vehicular control system of claim 1, wherein information pertaining to the at least one traffic participant in the vicinity of the equipped vehicle is wirelessly communicated from the at least one traffic participant to the equipped vehicle via a Wi-Fi (802.11p) wireless communication.

20. The vehicular control system of claim 1, wherein information pertaining to the at least one traffic participant in the vicinity of the equipped vehicle is wirelessly communicated from the at least one traffic participant to the equipped vehicle via 5G wireless communication.

21. The vehicular control system of claim 1, wherein said control, responsive at least in part to processing at said control of (i) received 3D point-cloud LIDAR data and (ii) received information pertaining to traffic participants in the vicinity of the equipped vehicle, controls at least one other vehicle function of the equipped vehicle.

22. A vehicular control system, said vehicular control system comprising:

a plurality of sensors disposed at a vehicle equipped with said vehicular control system;

wherein said plurality of sensors at least comprises (i) a camera having a field of view exterior of the equipped vehicle, said camera capturing image data, and (ii) a 3D point-cloud LIDAR sensor having a field of sensing exterior of the equipped vehicle, said 3D point-cloud LIDAR sensor capturing 3D point-cloud LIDAR data;

wherein the field of view of said camera encompasses at least one selected from the group consisting of (i) a road being traveled by the equipped vehicle and (ii) another road parallel to the road being traveled by the equipped vehicle;

wherein the field of sensing of said 3D point-cloud LIDAR sensor encompasses at least one selected from the group consisting of (i) the road being traveled by the equipped vehicle, and (ii) the other road parallel to the road being traveled by the equipped vehicle;

a control;

wherein image data captured by said camera is provided to said control;

wherein said control comprises an image processor that processes image data captured by said camera to determine presence of vehicles present exterior of the equipped vehicle and in the field of view of said camera and approaching the equipped vehicle on the road being traveled by the equipped vehicle or on the other road parallel to the road being traveled by the equipped vehicle;

wherein 3D point-cloud LIDAR data captured by said 3D point-cloud LIDAR sensor is provided to said control;

wherein said control comprises a data processor that processes 3D point-cloud LIDAR data captured by said 3D point-cloud LIDAR sensor to determine presence of vehicles present exterior of the equipped vehicle and in the field of sensing of said 3D point-cloud LIDAR sensor and approaching the equipped vehicle on the road being traveled by the equipped vehicle or on the other road parallel to the road being traveled by the equipped vehicle;

a receiver disposed at the equipped vehicle and operable to receive a wireless communication of information pertaining to traffic participants in the vicinity of the equipped vehicle;

wherein wirelessly-communicated information pertaining to other traffic participants in the vicinity of the equipped vehicle that is received at said receiver is provided to said control;

wherein said control is processes wirelessly-communicated information pertaining to traffic participants in the vicinity of the equipped vehicle;

wherein said control, responsive at least in part to received information pertaining to at least one traffic participant that is in the vicinity of the equipped vehicle and that is located outside the field of view of said camera and outside the field of sensing of said 3D point-cloud LIDAR sensor, controls at least one vehicle function of the equipped vehicle;

wherein, when the at least one traffic participant is in the field of view of said camera or in the field of sensing of said 3D point-cloud LIDAR sensor, and responsive at least in part to one selected from the group consisting of (i) processing at said control of image data captured by said camera and (ii) processing at said control of 3D point-cloud LIDAR data captured by said 3D point-cloud LIDAR sensor, said control controls the at least one vehicle function of the equipped vehicle; and

wherein, when the at least one traffic participant is in the field of view of said camera or in the field of sensing of said 3D point-cloud LIDAR sensor, and when said control is controlling the at least one vehicle function responsive at least in part to said one selected from the group consisting of (i) processing of image data captured by said camera and (ii) processing of 3D point-cloud LIDAR data captured by said 3D point-cloud LIDAR sensor, said control does not control the at least one vehicle function responsive to received information pertaining to the at least one traffic participant; and

wherein said control is-determines, via information wirelessly communicated from the at least one traffic participant to the equipped vehicle, that the at least one traffic participant is approaching an intersection ahead of the equipped vehicle.

23. The vehicular control system of claim 22, wherein said receiver is operable to receive wireless communication of information via a vehicle-to-vehicle (V2V) communication system.

24. The vehicular control system of claim 23, wherein said receiver is operable to receive wireless communication of information via a vehicle-to-infrastructure communication system.

25. The vehicular control system of claim 24, wherein said 3D point-cloud LIDAR sensor comprises a solid-state 3D point-cloud LIDAR sensor, and wherein said camera comprises a photosensor array having at least one million photosensor elements arranged in a two-dimensional array of at least 640 columns and at least 480 rows, and wherein said image processor comprises an image processing chip that utilizes object detection software in processing image data captured by said camera for detecting objects or other vehicles or pedestrians in the field of view of said camera.

26. The vehicular control system of claim 22, wherein said control determines, via information vehicle-to-vehicle (V2V) wirelessly communicated from the at least one traffic participant to the equipped vehicle, that the at least one traffic participant is ahead of the equipped vehicle when the at least one traffic participant is outside of the field of view of said camera.

27. The vehicular control system of claim 22, wherein said control determines, via information vehicle-to-vehicle (V2V) wirelessly communicated from the at least one traffic participant to the equipped vehicle, that the at least one traffic participant is approaching up a hill ahead of the equipped vehicle and is not yet in the field of view of said camera.

28. The vehicular control system of claim 22, wherein said control determines, via information vehicle-to-vehicle (V2V) wirelessly communicated from the at least one traffic participant to the equipped vehicle, that the at least one traffic participant is traveling along a road that is parallel to the road traveled by the equipped vehicle and is not yet in the field of view of said camera.

29. A vehicular control system, said vehicular control system comprising:

a plurality of sensors disposed at a vehicle equipped with said vehicular control system;

wherein said plurality of sensors at least comprises (i) a camera having a field of view exterior of the equipped vehicle, said camera capturing image data, and (ii) a 3D point-cloud LIDAR sensor having a field of sensing exterior of the equipped vehicle, said 3D point-cloud LIDAR sensor capturing 3D point-cloud LIDAR data;

wherein the field of view of said camera encompasses at least one selected from the group consisting of (i) a road being traveled by the equipped vehicle and (ii) another road parallel to the road being traveled by the equipped vehicle;

wherein the field of sensing of said 3D point-cloud LIDAR sensor encompasses at least one selected from the group consisting of (i) the road being traveled by the equipped vehicle, and (ii) the other road parallel to the road being traveled by the equipped vehicle;

a control;

wherein image data captured by said camera is provided to said control;

wherein said control comprises an image processor that processes image data captured by said camera to determine presence of vehicles present exterior of the equipped vehicle and in the field of view of said camera and approaching the equipped vehicle on the road being traveled by the equipped vehicle or on the other road parallel to the road being traveled by the equipped vehicle;

wherein 3D point-cloud LIDAR data captured by said 3D point-cloud LIDAR sensor is provided to said control;

wherein said control comprises a data processor that processes 3D point-cloud LIDAR data captured by said 3D point-cloud LIDAR sensor to determine presence of vehicles present exterior of the equipped vehicle and in the field of sensing of said 3D point-cloud LIDAR sensor and approaching the equipped vehicle on the road being traveled by the equipped vehicle or on the other road parallel to the road being traveled by the equipped vehicle;

wherein said 3D point-cloud LIDAR sensor comprises a solid-state 3D point-cloud LIDAR sensor;

a receiver disposed at the equipped vehicle and operable to receive a wireless communication of information pertaining to traffic participants in the vicinity of the equipped vehicle;

wherein wirelessly-communicated information pertaining to traffic participants in the vicinity of the equipped vehicle that is received at said receiver is provided to said control;

wherein said control that processes wirelessly-communicated information pertaining to traffic participants in the vicinity of the equipped vehicle;

wherein said control, responsive at least in part to received information pertaining to at least one traffic participant that is in the vicinity of the equipped vehicle and that is located outside the field of view of said camera and outside the field of sensing of said 3D point-cloud LIDAR sensor, controls at least one vehicle function of the equipped vehicle;

wherein information pertaining to the at least one traffic participant in the vicinity of the equipped vehicle is wirelessly communicated to the equipped vehicle via one selected from the group consisting of (i) a vehicle-to-vehicle (V2V) wireless communication and (ii) a vehicle-to-infrastructure wireless communication at least when the at least one traffic participant is located outside the field of view of said camera and is located outside the field of sensing of said 3D point-cloud LIDAR sensor;

wherein, when the at least one traffic participant is in the field of view of said camera or in the field of sensing of said 3D point-cloud LIDAR sensor, and responsive at least in part to one selected from the group consisting of (i) processing at said control of image data captured by said camera and (ii) processing at said control of 3D point-cloud LIDAR data captured by said 3D point-cloud LIDAR sensor, said control controls the at least one vehicle function of the equipped vehicle; and

wherein, when the at least one traffic participant is in the field of view of said camera or in the field of sensing of said 3D point-cloud LIDAR sensor, and when said control is controlling the at least one vehicle function responsive at least in part to said one selected from the group consisting of (i) processing of image data captured by said camera and (ii) processing of 3D point-cloud LIDAR data captured by said 3D point-cloud LIDAR sensor, said control does not control the at least one vehicle function responsive to received information pertaining to the at least one traffic participant.

30. The vehicular control system of claim 29, wherein said control determines, via information wirelessly communicated from the at least one traffic participant to the equipped vehicle, that the at least one traffic participant is approaching an intersection ahead of the equipped vehicle.

31. The vehicular control system of claim 30, wherein the information is received at the equipped vehicle via a vehicle-to-vehicle (V2V) communication system.

32. The vehicular control system of claim 30, wherein the information is received at the equipped vehicle via a vehicle-to-infrastructure communication system.

33. The vehicular control system of claim 29, wherein said control determines, via information vehicle-to-vehicle (V2V) wirelessly communicated from the at least one traffic participant to the equipped vehicle, that the at least one traffic participant is ahead of the equipped vehicle.

34. The vehicular control system of claim 29, wherein said control determines, via information vehicle-to-vehicle (V2V) wirelessly communicated from the at least one traffic participant to the equipped vehicle, that the at least one traffic participant is traveling along a road that is parallel to the road traveled by the equipped vehicle.

35. The vehicular control system of claim 29, wherein said control determines, via information vehicle-to-vehicle (V2V) wirelessly communicated from the at least one traffic participant to the equipped vehicle, that the at least one traffic participant is approaching up a hill.

36. The vehicular control system of claim 29, wherein information pertaining to the at least one traffic participant in the vicinity of the equipped vehicle is wirelessly communicated from the at least one traffic participant to the equipped vehicle via a Wi-Fi (802.11p) wireless communication.

37. The vehicular control system of claim 36, wherein information pertaining to the at least one traffic participant in the vicinity of the equipped vehicle is wirelessly communicated from the at least one traffic participant to the equipped vehicle via 5G wireless communication.

38. The vehicular control system of claim 29, wherein said control, responsive at least in part to processing at said control of (i) received 3D point-cloud LIDAR data and (ii) received information pertaining to traffic participants in the vicinity of the equipped vehicle, controls at least one other vehicle function of the equipped vehicle.