Autonomous vehicle operation feature monitoring and evaluation of effectiveness

• US 10,599,155 B1
• Filed: 02/01/2017
• Issued: 03/24/2020
• Est. Priority Date: 05/20/2014
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Konrardy, Blake, Christensen, Scott T., Hayward, Gregory, Farris, Scott
• Primary Examiner(s): Anderson, Mike
• Application Number:
  US15/421,521
• Time in Patent Office: 1,147 Days
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2030/082 Vehicle operation after collision
• CPC Class Codes: B60W 2420/40 Photo, light or radio wave sensitive means, e.g. infrared sensors
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2540/00 Input parameters relating to occupants
• CPC Class Codes: B60W 2552/00 Input parameters relating to infrastructure
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2710/20 Steering systems
• CPC Class Codes: B60W 2720/10 Longitudinal speed
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/18163 Lane change; Overtaking manoeuvres
• CPC Class Codes: B60W 60/0011 involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0221 involving a learning process
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06N 5/04 : Inference or reasoning models
• CPC Class Codes: G06N 7/01 : Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 40/08 : Insurance
• CPC Class Codes: G07C 5/008 : communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G07C 5/08 : Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time

Abstract

Methods and systems for monitoring use and determining risks associated with operation of a vehicle having one or more autonomous operation features are provided. According to certain aspects, operating data may be recorded during operation of the vehicle. This may include information regarding the vehicle, the vehicle environment, use of the autonomous operation features, and/or control decisions made by the features. The control decisions may include actions the feature would have taken to control the vehicle, but which were not taken because a vehicle operator was controlling the relevant aspect of vehicle operation at the time. The operating data may be recorded in a log, which may then be used to determine risk levels associated with vehicle operation based upon risk levels associated with the autonomous operation features. The risk levels may further be used to adjust an insurance policy associated with the vehicle.

Claims

1. A computer system for monitoring an autonomous vehicle having an autonomous system, comprising:

one or more processors; and

a non-transitory program memory coupled to the one or more processors and storing executable instructions that, when executed by the one or more processors, cause the computer system to;

receive, via wireless communication or data transmission over one or more radio links, initial sensor data indicating the occurrence of a vehicle collision involving the autonomous vehicle, wherein the initial sensor data includes information from at least one vehicle-mounted sensor or mobile device sensor;

receive, via wireless communication or data transmission over the one or more radio links, additional sensor data indicating (i) one or more environmental conditions in which the vehicle collision occurred, (ii) an identification of a person positioned within the autonomous vehicle to operate the autonomous vehicle at the time of the vehicle collision, and (iii) an identification of one or more capabilities or features of the autonomous system, wherein the additional sensor data includes information from at least one vehicle-mounted sensor, autonomous system sensor, or mobile device sensor;

process the additional sensor data using a trained machine learning program to determine one or more preferred control decisions the autonomous system should have made to control the autonomous vehicle immediately before or during the vehicle collision;

receive control decision data indicating one or more actual control decisions the autonomous system made to control the autonomous vehicle immediately before or during the vehicle collision;

determine a degree of similarity between the one or more preferred control decisions that should have been made by the autonomous system to control the autonomous vehicle and the one or more actual control decisions made by the autonomous system to control the autonomous vehicle; and

assign a percentage of fault for the vehicle collision to the autonomous system based upon the determined degree of similarity between the one or more preferred control decisions and the one or more actual control decisions.

2. The computer system of claim 1, wherein the one or more preferred control decisions and the one or more actual control decisions are virtually time-stamped for comparison of such controlled and actual control decisions based upon matching virtual time stamps.

3. The computer system of claim 1, wherein the executable instructions further cause the computer system to:

train the machine learning program to determine control decisions that should be preferably made by autonomous systems based upon data related to capabilities of the autonomous systems.

4. The computer system of claim 3, wherein the executable instructions further cause the computer system to:

train the machine learning program to determine control decisions that should be preferably made by autonomous systems based upon (i) data related to individual driver driving behavior or (ii) telematics data associated with the individual driver driving behavior.

5. The computer system of claim 4, wherein the executable instructions further cause the computer system to:

train the machine learning program to determine control decisions that should be preferably made by autonomous systems based upon data related to a plurality of the following;

environmental conditions, road conditions, construction conditions, and traffic conditions.

6. The computer system of claim 5, wherein the executable instructions further cause the computer system to:

train the machine learning program to determine control decisions that should be preferably made by autonomous systems based upon data related to levels of pedestrian traffic.

7. The computer system of claim 1, wherein the one or more actual control decisions include a control decision to change lanes or to turn the autonomous vehicle.

8. The computer system of claim 1, wherein the one or more actual control decisions include (i) a control decision to accelerate or to decelerate or (ii) an indication of a rate of acceleration or deceleration.

9. The computer system of claim 1, wherein the additional sensor data indicating (i) one or more environmental conditions in which the vehicle collision occurred and (ii) an identification of a person positioned within the autonomous vehicle to operate the autonomous vehicle at the time of the vehicle collision includes at least one of camera image data, radar unit data, or infrared data.

10. The computer system of claim 1, wherein the executable instructions further cause the computer system to adjust a risk level or model parameter associated with the autonomous vehicle or the autonomous system based upon the one or more actual control decisions made by the autonomous system.

11. A tangible, non-transitory computer-readable medium storing executable instructions for monitoring an autonomous vehicle having an autonomous system that, when executed by at least one processor of a computer system, cause the computer system to:

receive, via wireless communication or data transmission over one or more radio links, initial sensor data indicating the occurrence of a vehicle collision involving the autonomous vehicle, wherein the initial sensor data includes information from at least one vehicle-mounted sensor or mobile device sensor;

receive, via wireless communication or data transmission over the one or more radio links, additional sensor data indicating (i) one or more environmental conditions in which the vehicle collision occurred, (ii) an identification of a person positioned within the autonomous vehicle to operate the autonomous vehicle at the time of the vehicle collision, and (iii) an identification of one or more capabilities or features of the autonomous system, wherein the additional sensor data includes information from at least one vehicle-mounted sensor, autonomous system sensor, or mobile device sensor;

process the additional sensor data using a trained machine learning program to determine one or more preferred control decisions the autonomous system should have made to control the autonomous vehicle immediately before or during the vehicle collision;

receive control decision data indicating one or more actual control decisions the autonomous system made to control the autonomous vehicle immediately before or during the vehicle collision;

determine a degree of similarity between the one or more preferred control decisions that should have been made by the autonomous system to control the autonomous vehicle and the one or more actual control decisions made by the autonomous system to control the autonomous vehicle; and

assign a percentage of fault for the vehicle collision to the autonomous system based upon the determined degree of similarity between the one or more preferred control decisions and the one or more actual control decisions.

12. The tangible, non-transitory computer-readable medium of claim 11, wherein the one or more preferred control decisions and the one or more actual control decisions are virtually time-stamped for comparison of such controlled and actual control decisions based upon matching virtual time stamps.

13. The tangible, non-transitory computer-readable medium of claim 11, wherein the instructions further cause the computer system to:

train the machine learning program to determine control decisions that should be preferably made by autonomous systems based upon data related to capabilities of the autonomous systems.

14. The tangible, non-transitory computer-readable medium of claim 11, wherein the instructions further cause the computer system to:

train the machine learning program to determine control decisions that should be preferably made by autonomous systems based upon (i) data related to individual driver driving behavior or (ii) telematics data associated with the individual driver driving behavior.

15. The tangible, non-transitory computer-readable medium of claim 11, wherein the instructions further cause the computer system to:

train the machine learning program to determine control decisions that should be preferably made by autonomous systems based upon data related to a plurality of the following;

environmental conditions, road conditions, construction conditions, and traffic conditions.

16. The tangible, non-transitory computer-readable medium of claim 11, wherein the instructions further cause the computer system to:

train the machine learning program to determine control decisions that should be preferably made by autonomous systems based upon data related to levels of pedestrian traffic.

17. The tangible, non-transitory computer-readable medium of claim 11, wherein the one or more actual control decisions include a control decision to change lanes or to turn the autonomous vehicle.

18. The tangible, non-transitory computer-readable medium of claim 11, wherein the one or more actual control decisions include a control decision to accelerate or to decelerate.

19. The tangible, non-transitory computer-readable medium of claim 11, wherein the additional sensor data indicating (i) one or more environmental conditions in which the vehicle collision occurred and (ii) an identification of a person positioned within the autonomous vehicle to operate the autonomous vehicle at the time of the vehicle collision includes at least one of camera image data, radar unit data, or infrared data.

20. The tangible, non-transitory computer-readable medium of claim 11, wherein the instructions further cause the computer system to adjust a risk level or model parameter associated with the autonomous vehicle or the autonomous system based upon the one or more actual control decisions made by the autonomous system.

21. A computer-implemented method of monitoring an autonomous vehicle having an autonomous system, the method comprising:

receiving, via wireless communication or data transmission over one or more radio links, initial sensor data indicating the occurrence of a vehicle collision involving the autonomous vehicle, wherein the initial sensor data includes information from at least one vehicle-mounted sensor or mobile device sensor;

receiving, via wireless communication or data transmission over the one or more radio links, additional sensor data indicating (i) one or more environmental conditions in which the vehicle collision occurred, (ii) an identification of a person positioned within the autonomous vehicle to operate the autonomous vehicle at the time of the vehicle collision, and (iii) an identification of one or more capabilities or features of the autonomous system, wherein the additional sensor data includes information from at least one vehicle-mounted sensor, autonomous system sensor, or mobile device sensor;

processing the additional sensor data using a trained machine learning program to determine one or more preferred control decisions the autonomous system should have made to control the autonomous vehicle immediately before or during the vehicle collision;

receiving control decision data indicating one or more actual control decisions the autonomous system made to control the autonomous vehicle immediately before or during the vehicle collision;

determining a degree of similarity between the one or more preferred control decisions that should have been made by the autonomous system to control the autonomous vehicle and the one or more actual control decisions made by the autonomous system to control the autonomous vehicle; and

assigning a percentage of fault for the vehicle collision to the autonomous system based upon the determined degree of similarity between the one or more preferred control decisions and the one or more actual control decisions.