Early warning and collision avoidance

• US 10,565,880 B2
• Filed: 12/17/2018
• Issued: 02/18/2020
• Est. Priority Date: 03/19/2018
• Status: Active Grant

• Current Assignee: Derq Inc.
• Original Assignee: Derq Inc.
• Inventors: Aoude, Georges, Abufadel, Amer, Jeanbart, Karl, Choudhary, Rishabh, Sharma, Ankit
• Primary Examiner(s): Refai, Ramsey
• Application Number:
  US16/222,536
• Publication Number:
  US 20190287395A1
• Time in Patent Office: 428 Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 5/046 Forward inferencing; Production systems
• CPC Class Codes: G08G 1/005 including pedestrian guidance indicator
• CPC Class Codes: G08G 1/0112 from the vehicle, e.g. floating car data [FCD]
• CPC Class Codes: G08G 1/0116 from roadside infrastructure, e.g. beacons
• CPC Class Codes: G08G 1/0129 for creating historical data or processing based on historical data
• CPC Class Codes: G08G 1/0137 for specific applications
• CPC Class Codes: G08G 1/0141 for traffic information dissemination
• CPC Class Codes: G08G 1/017 identifying vehicles G08G1/015, G08G1/054 take precedence
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules
• CPC Class Codes: G08G 1/052 with provision for determining speed or overspeed speed measuring in general G01P
• CPC Class Codes: G08G 1/093 Data selection, e.g. prioritizing information, managing message queues, selecting the information to be output
• CPC Class Codes: G08G 1/096716 where the received information does not generate an automatic action on the vehicle control
• CPC Class Codes: G08G 1/096725 where the received information generates an automatic action on the vehicle control
• CPC Class Codes: G08G 1/123 indicating the position of vehicles, e.g. scheduled vehicles; Managing passenger vehicles circulating according to a fixed timetable, e.g. buses, trains, tramstransmission of navigation instructions to vehicles G08G1/0968
• CPC Class Codes: G08G 1/163 involving continuous checking
• CPC Class Codes: G08G 1/164 Centralised systems, e.g. external to vehicles
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Among other things, data is received from a sensor oriented to monitor ground transportation entities at or near an intersection of a transportation network, and a machine learning model predicts behavior of ground transportation entities at or near the intersection at a current time. The machine learning model is based on training data about previous motion and related behavior of ground transportation entities at or near the intersection. Current motion data is applied to the machine learning model to predict imminent behaviors of the ground transportation entities. An imminent dangerous situation for one or more of the ground transportation entities at or near the intersection is inferred from the predicted imminent behaviors. A wireless communication device sends a warning about the dangerous situation to one of the ground transportation entities.

Claims

1. An apparatus comprising

equipment located at an intersection of a transportation network, the equipment comprising

inputs to receive data from sensors oriented to monitor ground transportation entities at or near the intersection,

the data from each of the sensors representing at least one location or motion parameter of at least one of the ground transportation entities

the data from each of the sensors being expressed in a native format of the sensor,

the data received from at least two of the sensors being inconsistent with respect to the location or motion parameters represented by the data or with respect to the native formats of the sensors or with respect to both,

a processor, and

a storage for instructions executable by the processor to

convert the data from the sensors into data having a common format independent of the native formats of the data of one or more of the sensors,

incorporate the data that has the common format and is inconsistent with respect to the location or motion parameters into a global unified representation of the location or motion parameters of the ground transportation entities being monitored at or near the intersection, the global unified representation representing the location, speed, and heading of each of the ground transportation entities, the global unified representation represents the locations of the ground transportation entities in a common reference frame comprising a geographic coordinate system,

determine relationships of locations and motions of two of the ground transportation entities using the global unified representation,

predict a dangerous situation involving the two ground transportation entities based on the determined relationships, and

send a message to at least one of the two ground transportation entities alerting it to the dangerous situation.

2. The apparatus of claim 1 in which the sensors comprise at least two of:

radar, lidar, and a camera.

3. The apparatus of claim 1 in which the data received from one of the sensors comprises image data of a field of view at successive moments.

4. The apparatus of claim 1 in which the data received from one of the sensors comprises points of reflection in 3D space.

5. The apparatus of claim 1 in which the data received from one of the sensors comprises distance from the sensor and speed.

6. The apparatus of claim 1 comprising at least two sensors from which the data is received, the two sensors being mounted in fixed positions at or near the intersection and having at least partially non-overlapping fields of view.

7. The apparatus of claim 6 which one of the sensors comprises radar and converting the data includes determining locations of ground transportation entities from a known location of the radar and distances from the radar to the ground transportation entities.

8. The apparatus of claim 6 which one of the sensors comprises a camera and converting the data includes determining locations of ground transportation entities from a known location, direction of view, and tilt of the camera and the locations of the ground transportation entities within an image frame of the camera.

9. A method comprising

receiving data from sensors oriented to monitor ground transportation entities at or near an intersection of a ground transportation network,

the data from each of the sensors representing at least one location or motion parameter of at least one of the ground transportation entities

the data from each of the sensors being expressed in a native format of the sensor,

the data received from at least two of the sensors being inconsistent with respect to the location or motion parameters represented by the data or with respect to the native formats of the sensors or with respect to both,

converting the data from the sensors into data having a common format independent of the native formats of the data of one or more of the sensors,

incorporate the data that has the common format and is inconsistent with respect to the location or motion parameters into a global unified representation of the location or motion parameters of the ground transportation entities being monitored at or near the intersection, the global unified representation representing the location, speed, and heading of each of the ground transportation entities, the global unified representation represents the locations of the ground transportation entities in a common reference frame comprising a geographic coordinate system,

determining relationships of locations and motions of two of the ground transportation entities using the global unified representation,

predicting a dangerous situation involving the two ground transportation entities based on the determined relationships, and

sending a message to at least one of the two ground transportation entities alerting it to the dangerous situation.

10. The method of claim 9 in which the sensors comprise at least two of:

radar, lidar, and a camera.

11. The method of claim 9 which the data received from one of the sensors comprises image data of a field of view at successive moments.

12. The method of claim 9 which the data received from one of the sensors comprises points of reflection in 3D space.

13. The method of claim 9 in which the data received from one of the sensors comprises distance from the sensor and speed.

14. The method of claim 9 which the data is received from at least two sensors from the two sensors being mounted in fixed positions at or near the intersection and having at least partially non-overlapping fields of view.

15. The method of claim 14 which one of the sensors comprises radar and the method comprises converting the data includes determining locations of ground transportation entities from a known location of the radar and distances from the radar to the ground transportation entities.

16. The method of claim 14 which one of the sensors comprises a camera and the method comprises converting the data includes determining locations of ground transportation entities from a known location, direction of view, and tilt of the camera and the locations of the ground transportation entities within an image frame of the camera.