Manual vehicle control notification

• US 11,040,725 B2
• Filed: 09/04/2015
• Issued: 06/22/2021
• Est. Priority Date: 09/04/2015
• Status: Active Grant

• Current Assignee: Inrix Incorporated
• Original Assignee: Inrix Incorporated
• Inventors: Scofield, Christopher L.
• Primary Examiner(s): Wallace, Donald J
• Application Number:
  US14/845,356
• Publication Number:
  US 20170066452A1
• Time in Patent Office: 2,118 Days
• CPC Class Codes: B60K 28/00 Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions for electrically-propelled vehicles B60L3/00; road vehicle drive control systems for purposes not related to the control of a particular sub-unit B60W30/00; drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: B60W 2040/0872 Driver physiology
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 2540/229 Attention level, e.g. attentive to driving, reading or sleeping
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/406 Traffic density
• CPC Class Codes: B60W 2555/20 Ambient conditions, e.g. wind or rain
• CPC Class Codes: B60W 50/082 Selecting or switching between different modes of propelling
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0053 from vehicle to occupant
• CPC Class Codes: B60W 60/0059 Estimation of the risk associated with autonomous or manual driving, e.g. situation too complex, sensor failure or driver incapacity
• CPC Class Codes: G05D 1/0061 for transition from automatic pilot to manual pilot and vice versa
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G08G 1/096725 where the received information generates an automatic action on the vehicle control
• CPC Class Codes: G08G 1/096791 where the origin of the information is another vehicle
• CPC Class Codes: G08G 1/163 involving continuous checking
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees

Abstract

One or more techniques and/or systems are provided for notifying drivers to assume manual vehicle control of vehicles. For example, sensor data is acquired from on-board vehicles sensors (e.g., radar, sonar, and/or camera imagery of a crosswalk) of a vehicle that is in an autonomous driving mode. In an example, the sensor data is augmented with driving condition data aggregated from vehicle sensor data of other vehicles (e.g., a cloud service collects and aggregates vehicle sensor data from vehicles within the crosswalk to identify and provide the driving condition data to the vehicle). The sensor data (e.g., augmented sensor data) is evaluated to identify a driving condition of a road segment, such as the crosswalk (e.g., pedestrians protesting within the crosswalk). Responsive to the driving condition exceeding a complexity threshold for autonomous driving decision making functionality, a driver alert to assume manual vehicle control may be provided to a driver.

Claims

1. A method for notifying a driver to assume manual vehicle control, comprising:

evaluating sensor data, acquired from one or more on-board sensors of a vehicle that is in an autonomous driving mode, to determine a driving condition of a road segment that is to be traveled by the vehicle;

determining a cognitive load of the driver based upon a type of activity associated with a first location from which the vehicle is traveling;

responsive to determining that the driving condition exceeds a complexity threshold, derived from the cognitive load, for autonomous driving decision making functionality, generating a driver alert to assume manual vehicle control;

determining a point in time at which to provide the driver alert relative to a point in time at which the road segment is encountered based upon the cognitive load, wherein;

when the cognitive load of the driver has a first cognitive load value, selecting the point in time at which to provide the driver alert to be a first time interval away from the point in time at which the road segment is encountered,

when the cognitive load of the driver has a second cognitive load value different than the first cognitive load value, selecting the point in time at which to provide the driver alert to be a second time interval, different than the first time interval, away from the point in time at which the road segment is encountered; and

providing the driver alert to the driver of the vehicle at the point in time, relative to the point in time at which the road segment is encountered.

2. The method of claim 1, wherein determining the cognitive load comprises determining the cognitive load of the driver by monitoring activity of the driver.

3. The method of claim 1, comprising:

identifying a potential alternate route to avoid the road segment, wherein determining the point in time at which to provide the driver alert comprises determining the point in time based upon a current location of the vehicle and a location of the potential alternate route.

4. The method of claim 1, comprising:

utilizing travel prediction functionality to predict a route of the vehicle; and

identifying the road segment based upon the route and a current location of the vehicle.

5. The method of claim 1, comprising:

receiving driving condition data for the road segment from a remote computing device over a communication connection; and

augmenting the sensor data with the driving condition data.

6. The method of claim 1, comprising:

predicting an amount of time required to determine an appropriate driving action based upon the driving condition; and

setting the complexity threshold based upon the amount of time required to determine the appropriate driving action relative to an amount of time remaining before reaching the road segment.

7. The method of claim 5, the driving condition data corresponding to an aggregation of vehicle sensor data from one or more vehicles while the one or more vehicles traveled the road segment.

8. The method of claim 1, comprising:

receiving, from a remote computing device over a communication connection, a second driving condition for a second road segment that is to be traveled by the vehicle;

responsive to determining that the second driving condition exceeds the complexity threshold for autonomous driving decision making functionality, generating a second driver alert to assume manual vehicle control; and

providing the second driver alert to the driver of the vehicle at a second point in time, relative to the second road segment, based upon the second driving condition.

9. The method of claim 1, the driving condition corresponding to at least one of a road condition, a weather condition, a pedestrian, an accident, an ethical decision, or a driving impediment.

10. The method of claim 1, wherein determining the cognitive load comprises determining the cognitive load of the driver utilizing at least one of a microphone or a camera.

11. The method of claim 1, wherein determining the cognitive load comprises determining the cognitive load based upon conversations between the driver and another person.

12. The method of claim 1, wherein monitoring activity of the driver comprises observing whether the driver is at least one of:

eating, sleeping, or engaging a computing device.

13. A method for notifying a driver to assume manual vehicle control, comprising:

evaluating sensor data, acquired from one or more on-board sensors of a vehicle that is in an autonomous driving mode, to determine a driving condition of a road segment that is to be traveled by the vehicle;

determining a cognitive load of the driver based upon a type of activity associated with a first location from which the vehicle is traveling;

responsive to determining that the driving condition exceeds a complexity threshold, derived from the cognitive load, for autonomous driving decision making functionality, generating a driver alert to assume manual vehicle control;

determining a point in time at which to provide the driver alert relative to a point in time at which the road segment is encountered based upon the cognitive load, wherein;

when the cognitive load of the driver has a first cognitive load value, selecting the point in time at which to provide the driver alert to be a first time interval away from the point in time at which the road segment is encountered,

when the cognitive load of the driver has a second cognitive load value, selecting the point in time at which to provide the driver alert to be a second time interval, different than the first time interval, away from the point in time at which the road segment is encountered; and

providing the driver alert to the driver of the vehicle at the point in time, relative to the point in time at which the road segment is encountered.

14. The method of claim 13, wherein the complexity threshold is based upon the cognitive load.

15. The method of claim 13, wherein determining the cognitive load of the driver comprises determining the cognitive load of the driver by monitoring activity of the driver using at least one of a camera in the vehicle or a microphone in the vehicle.

16. The method of claim 15, wherein the complexity threshold is based upon the cognitive load.

17. The method of claim 13, wherein determining the cognitive load of the driver comprises determining the cognitive load of the driver based upon social media posts associated with the driver.

18. The method of claim 13, wherein social media posts associated with the driver corresponds to social media posts by the driver.

19. The method of claim 13, comprising:

utilizing travel prediction functionality to predict a route of the vehicle; and

identifying the road segment based upon the route and a current location of the vehicle.

20. A system, comprising:

a processor; and

memory that when executed by the processor performs operations for notifying a driver to assume manual vehicle control, the operations comprising;

evaluating sensor data, acquired from one or more on-board sensors of a vehicle that is in an autonomous driving mode, to determine a driving condition of a road segment that is to be traveled by the vehicle;

determining a cognitive load of the driver based upon a type of activity associated with a first location from which the vehicle is traveling;

responsive to determining that the driving condition exceeds a complexity threshold, derived from the cognitive load, for autonomous driving decision making functionality, generating a driver alert to assume manual vehicle control;

determining a point in time at which to provide the driver alert relative to a point in time at which the road segment is encountered based upon the cognitive load, wherein;

when the cognitive load of the driver has a first cognitive load value, selecting the point in time at which to provide the driver alert to be a first time interval away from the point in time at which the road segment is encountered,

when the cognitive load of the driver has a second cognitive load value, selecting the point in time at which to provide the driver alert to be a second time interval away from the point in time at which the road segment is encountered; and

providing the driver alert to the driver of the vehicle at the point in time, relative to the point in time at which the road segment is encountered.