SYSTEMS AND METHODS FOR NEAR-CRASH DETERMINATION

• US 20180075309A1
• Filed: 09/14/2017
• Published: 03/15/2018
• Est. Priority Date: 09/14/2016
• Status: Active Grant

• Current Assignee: Nauto, Inc.
• Original Assignee: Nauto, Inc.
• Inventors: Sathyanarayana, Suchitra, Satzoda, Ravi Kumar, Heck, Stefan
• Granted Patent: US 9,928,432 B1
• Time in Patent Office: Days
• CPC Class Codes: B60W 2540/229 Attention level, e.g. attentive to driving, reading or sleeping
• CPC Class Codes: B60W 2556/10 Historical data
• CPC Class Codes: B60W 30/095 Predicting travel path or likelihood of collision
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: B60W 60/0015 specially adapted for safety
• CPC Class Codes: G05D 1/0055 with safety arrangements
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G07C 5/0866 the electronic data carrier being a digital video recorder in combination with video camera
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A method for near-collision detection, including determining a risk map for a vehicle and automatically detecting a near-collision event with an object based on vehicle behavior relative to the risk map.

Claims

1. A method comprising:

recording a first video with an external-facing camera mounted to a vehicle;

detecting an object from the first video;

determining object parameters for the object from the first video;

recording a second video with an internal-facing camera mounted to the vehicle;

determining a user behavior score based on the second video;

generating a risk map for the vehicle, the risk map comprising a risk score for each of a set of positions within a volume proximal the vehicle, each risk score calculated using a parametric module based on the user behavior score and object parameters;

detecting a near-collision event, comprising detecting a risk score within the risk map exceeding a threshold score;

storing a segment of the first video encompassing the near-collision event in response to detecting the near-collision event; and

identifying a cause of the near-collision event based on individual parameters of the parametric module.

2. The method of claim 1, further comprising:

aggregating a plurality of first video segments associated with near-collision events from a plurality of vehicles;

for each of the plurality of first video segments, storing a user response recorded during the near-collision event in association with the respective first video;

labeling each first video segment with the cause of the respective near-collision event;

filtering the plurality of first video segments based on the respective cause; and

training an autonomous vehicle control module based on the filtered plurality of first video segments and the respective user responses, wherein the autonomous vehicle control module controls autonomous operation of an autonomous vehicle.

3. The method of claim 2, wherein the user response is determined from a segment of the second video sampled during a timeframe encompassing the near-collision event.

4. The method of claim 1, wherein detecting the near-collision event comprises:

identifying the position associated with the risk score exceeding the threshold score;

determining a vehicle trajectory for the vehicle; and

detecting the near-collision event when the vehicle trajectory is directed toward the position.

5. The method of claim 4, wherein identifying the cause of the near-collision event comprises identifying the user as the cause of the near-collision event when the vehicle trajectory is directed toward the position.

6. The method of claim 1, wherein identifying the cause of the near-collision event comprises determining the independent parameter of the parametric module with a highest weighted value.

7. The method of claim 1, wherein the interior-facing camera is statically mounted at a known orientation relative to the exterior-facing camera by a common housing, wherein the first video is concurrently recorded with the second video, the method further comprising determining a driver gaze direction relative to the object based on the second video and the orientation, wherein the user behavior score is determined based on the driver gaze direction relative to the object.

8. The method of claim 1, wherein recording the first video, detecting the object, determining the object parameters, recording the second video, determining the user behavior score, generating the risk map, and detecting the near-collision event are performed in real-time by a processor on-board the vehicle.

9. The method of claim 8, wherein the risk map excludes a physical volume proximal a rear of the vehicle.

10. The method of claim 1, further comprising determining a geometry of the risk map based on anticipated behavior of a leading vehicle.

11. The method of claim 10, wherein the determining the geometry of the risk map based on anticipated behavior of a leading vehicle comprises:

from the first video, detecting turn indicator operation by the leading vehicle, the turn indicator associated with a first side; and

selecting a predetermined geometry biased toward the first side in response to detecting turn indicator operation as the geometry of the risk map.

12. The method of claim 1, further comprising:

generating risk maps for each of a plurality of locations along a vehicle route;

storing the risk maps for each of a plurality of vehicles in association with the respective locations; and

generating a risk map for each of the plurality of locations based on the risk maps from each of the plurality of vehicles for the respective location.

13. The method of claim 12, wherein the risk maps are generated in real-time, the method further comprising:

receiving a risk map request for a secondary vehicle without the interior-facing camera, exterior-facing camera, and processor, the risk map request comprising a location identifier for a location within the plurality of locations;

retrieving the risk map associated with the location; and

transmitting the retrieved risk map to the secondary vehicle, wherein the secondary vehicle is operated based on the retrieved risk map.

14. The method of claim 1, wherein the parametric module comprises a set of Gaussian models.

15. The method of claim 1, further comprising selecting the parametric module based on a geographic location of the vehicle from a set of predetermined parameter modules.

16. The method of claim 1, further comprising:

generating a recommendation based on the cause of the near-collision event;

transmitting a notification, comprising the recommendation, to the user;

monitoring user behavior based on video sampled by the interior-facing camera for a threshold duration after transmitting the notification; and

in response to an occurrence frequency of the cause falling below a threshold frequency during the threshold duration, transmitting the recommendation to a second user in response to the cause substantially matching a cause of a second near-collision event associated with the second user, wherein the second user is distinct from the first user.

17. A method comprising:

at an on-board system, mounted to the vehicle and comprising an external-facing camera, statically mounted to an internal-facing camera, and a processor electrically connected to the external-facing and internal-facing cameras;

concurrently recording a first video and second video using the external-facing camera and internal-facing camera, respectively;

detecting an object from the first video at the processor;

determining object parameters for the object from the first video at the processor;

determining a user behavior score at the processor based on the second video and a known orientation of the external-facing camera relative to the internal-facing camera;

generating a risk map for the vehicle in near-real time at the processor, the risk map comprising a risk score for each of a set of positions within a volume encompassing the vehicle, each risk score calculated using a parametric module based on the user behavior score and object parameters; and

detecting a near-collision event based on the risk map; and

at a remote computing system remote from the on-board system;

storing the first video in response to detecting the near-collision event;

storing a segment of the second video sampled after the near-collision event; and

training a control module based on the first video and the segment of the second video, wherein a secondary vehicle is automatically controlled by the control module.

18. The method of claim 17, further comprising:

generating risk maps for each of a plurality of locations at the processor;

storing the risk maps for each of a plurality of vehicles in association with the respective locations at the remote computing system;

at the remote computing system, generating a risk map for each of the plurality of locations based on the risk maps from each of the plurality of vehicles for the respective location;

at the remote computing system, receiving a risk map request for a secondary vehicle without the system, the risk map request comprising a location identifier for a location within the plurality of locations;

at the remote computing system, retrieving the risk map associated with the location; and

transmitting the retrieved risk map to the secondary vehicle, wherein the secondary vehicle is operated based on the retrieved risk map.

19. The method of claim 17, wherein the parametric module comprises a set of Gaussian models.

20. The method of claim 17, further comprising:

aggregating a plurality of first videos associated with near-collision events from a plurality of vehicles;

for each first video;

determining a cause of the near-collision event based on individual parameters of the parametric module; and

labeling each first video with the cause of the respective near-collision event;

filtering the plurality of videos based on the respective cause; and

training an autonomous vehicle control module based on the filtered plurality of first videos, wherein the autonomous vehicle control module controls autonomous operation of an autonomous vehicle.