Method and system to allow drivers or driverless vehicles to see what is on the other side of an obstruction that they are driving near, using direct vehicle-to-vehicle sharing of environment data

• US 9,858,817 B1
• Filed: 10/04/2016
• Issued: 01/02/2018
• Est. Priority Date: 10/04/2016
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bletzer, Nicholas J., Floyd, Christopher G., Warwick, Malcolm M.
• Primary Examiner(s): ELAHI, SHAN E
• Application Number:
  US15/285,198
• Time in Patent Office: 455 Days
• Field of Search: 348 36
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/102 using 360 degree surveillance camera system
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/50 characterised by the display information being shared, e.g. external display, data transfer to other traffic participants or centralised traffic controller
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G08G 1/163 involving continuous checking
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

In a vehicle communication system of vehicles, vehicles share environment data such as their location data and 360 degree view of the world with other vehicles using direct vehicle-to-vehicle (V2V) real-time data streams. A displayable map of potentially dangerous obstructions on the surrounding roadway is formed using in vehicle environment sensors allowing a driver or the controls of a driverless vehicle to be warned of the danger. A map of blind spots is built up to speed up the processing of incoming data in order to create a more complete picture of surrounding vehicles. Shared data is used to position each vehicle relative to the target vehicle. By sharing obstruction maps between vehicles, a more complete picture of the roadway can be displayed—and one vehicle can in effect “see” through another vehicle or an obstruction.

Claims

1. A method of building a 360 degree direct view surrounding a target vehicle using direct vehicle-to-vehicle (V2V) real-time data streams comprising:

receiving, by a hardware processor associated with a target vehicle, from one or more sensors attached to the target vehicle, a sensor real-time data stream;

building, at the hardware processor, a 360 degree direct view around the target vehicle based on the received sensor real-time data stream, wherein the 360 degree direct view includes a blind spot, defined by a visual obstruction beyond which another vehicle may be located;

receiving, by the hardware processor from a second vehicle, a first V2V real-time data stream that includes data representing a 360 degree direct view around the second vehicle, and a position of origin of the second vehicle relative to the target vehicle, wherein the target vehicle and the another vehicle in the blind spot are within the 360 degree direct view around the second vehicle;

receiving, by the hardware processor from the second vehicle, a second V2V real-time data stream that includes a composite data stream representing the first V2V real-time data stream and a 360 degree direct view around the another vehicle, and

forming, by the hardware processor, based on the received first and the received second V2V real-time data streams and the position of origin of the second vehicle relative to the target vehicle, a composite 360 degree direct view around the target vehicle in which information representing the blind spot is replaced with a representation of a 360 degree direct view around the target vehicle without the obstruction and the location where the another vehicle is located enabling the target vehicle to see beyond the blind spot.

2. The method of claim 1, wherein the target vehicle is a driverless vehicle.

3. The method of claim 1, wherein the position of origin of the second vehicle relative to the target vehicle, the first V2V real-time data stream, and the second V2V real-time data stream are transmitted from respective locations at the second vehicle.

4. The method of claim 1, further comprising the hardware processor displaying the 360 degree view around the target vehicle.

5. The method of claim 1, wherein additional positions of origin are second vehicles, additional first V2V real-time data streams, and additional second V2V real-time data streams from additional second vehicle are received until any remaining blind spots are replaced.

23. The method of claim 1, wherein the one or more sensors are selected from the group consisting of radar sensor, video camera, and ultrasound sensor.

6. A vehicle communication system of vehicles sharing environment data such as the vehicle location data and 360 degree view of the world with other vehicles using direct vehicle-to-vehicle (V2V) real-time data streams comprising:

a target vehicle including a sensor and a processor building a 360 degree direct view of the target vehicle surroundings containing a blind spot;

the target vehicle sensor determining the position of the target vehicle relative to a sending vehicle and identifying the blind spot in the target vehicle surroundings;

a receiver at the sending vehicle receiving V2V real-time data streams from a hidden vehicle, said V2V data from the hidden vehicle representing a direct view of the hidden vehicle surroundings;

the sending vehicle including a sensor building a V2V real-time direct view data stream of the sending vehicle surroundings and combining the sending vehicle direct view of the sending vehicle surroundings with the V2V real-time data streams from the hidden vehicle for forming a composite V2V real-time data stream and sending the V2V real-time direct view data stream and the composite V2V real-time data stream to the target vehicle, and

the target vehicle using received V2V real-time direct view data stream and the composite V2V real-time data stream from the sending vehicle to fill in the blind spot in the target vehicle 360 degree direct view thereby enabling the target vehicle to locate the hidden vehicle enabling the target vehicle to see beyond the blind spot.

7. The system of claim 6, where the target vehicle is a driverless vehicle.

8. The system of claim 6, where the sending is performed by vehicle-to-vehicle real-time data stream sending between vehicles.

9. The system of claim 6, wherein the view of the surroundings is made using the sensor selected from the group consisting of radar sensor, video camera, or ultrasound sensor.

10. The system of claim 6, wherein the sending vehicle has multiple transmitting locations.

11. The system of claim 6, further comprising displaying a 360 degree view to the target vehicle.

12. The system of claim 6, where there is a plurality of sending vehicles for sending images to another sending vehicle or to the sending vehicle to fill in blind spots of the target vehicle.

13. The system of claim 12, where the sending is performed by vehicle-to-vehicle real-time data stream sending between vehicles.

14. The system of claim 12, wherein the view of the surroundings is made using the sensor selected from the group consisting of radar sensor, video camera, or ultrasound sensor.

15. The system of claim 12, wherein each sending vehicle has multiple transmitting locations.

16. The system of claim 12, further comprising displaying a 360 degree view to the target vehicle.

17. A system for building a 360 degree direct view surrounding a target vehicle using direct (vehicle-to-vehicle) V2V real-time data streams comprising:

a hardware processor associated with a target vehicle receiving from one or more sensors attached to the target vehicle a sensor real-time data stream;

the hardware processor building a 360 degree direct view around the target vehicle based on the received sensor real-time data stream, wherein the 360 degree direct view includes a blind spot, defined by a visual obstruction beyond which another vehicle may be located;

the hardware processor receiving a first V2V real-time data stream that includes data representing a 360 degree direct view around a second vehicle, and a position of origin of the second vehicle relative to the target vehicle, wherein the target vehicle and the another vehicle in the blind spot are within the 360 degree direct view around the second vehicle;

the hardware processor receiving a second V2V real-time data stream that includes a composite data stream representing the first data stream and a 360 degree direct view around the another vehicle, and

the hardware processor forming based on the received first and the received second V2V real-time data streams and the position of origin of the second vehicle relative to the target vehicle, a composite 360 degree direct view around the target vehicle in which information representing the blind spot is replaced with a representation of a 360 degree direct view around the target vehicle without the obstruction and the location where the another vehicle is located enabling the target vehicle to see beyond the blind spot.

18. The system of claim 17, wherein the target vehicle is a driverless vehicle.

19. The system of claim 17, wherein the real-time data streams are vehicle-to vehicle data streams.

20. The system of claim 17, wherein the sensors are selected from the group consisting of radar sensor, video camera, and ultrasound sensor.

21. The system of claim 17, further comprising the hardware processor displaying a 360 degree direct view surrounding the target vehicle.

22. The system of claim 17, comprising a plurality of second vehicles.