Cognitive load driving assistant

• US 10,399,575 B2
• Filed: 01/11/2016
• Issued: 09/03/2019
• Est. Priority Date: 01/12/2015
• Status: Active Grant

• Current Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Inventors: Spasojevic, Mirjana, Marti, Stefan, Di Censo, Davide, Nahman, Jaime Elliot
• Primary Examiner(s): Troost, Aaron L
• Application Number:
  US15/541,458
• Publication Number:
  US 20180009442A1
• Time in Patent Office: 1,331 Days
• CPC Class Codes: A61B 3/112 for measuring diameter of pupils
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: A61B 5/02438 with portable devices, e.g. worn by the patient
• CPC Class Codes: A61B 5/0531 Measuring skin impedance
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/4064 Evaluating the brain A61B5/031, A61B5/369, A61B5/14553 take precedence
• CPC Class Codes: A61B 5/6893 Cars
• CPC Class Codes: B60K 2360/164 Infotainment
• CPC Class Codes: B60K 2360/166 Navigation
• CPC Class Codes: B60K 2360/167 Vehicle dynamics information
• CPC Class Codes: B60K 2360/168 Target or limit values
• CPC Class Codes: B60K 2360/178 Warnings
• CPC Class Codes: B60K 2360/195 Blocking or enabling display functions
• CPC Class Codes: B60K 2360/197 Blocking or enabling of input functions
• CPC Class Codes: B60K 2360/48 Sensors
• CPC Class Codes: B60K 2360/5915 Inter vehicle communication
• CPC Class Codes: B60K 2360/592 Data transfer involving external databases
• CPC Class Codes: B60K 28/02 responsive to conditions relating to the driver
• CPC Class Codes: B60K 28/06 responsive to incapacity of driver
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: B60K 35/28 : characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/29 : Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60K 35/65 : Instruments specially adapted for specific vehicle types or users, e.g. for left- or right-hand drive
• CPC Class Codes: B60K 35/85 : Arrangements for transferring vehicle- or driver-related data
• CPC Class Codes: B60W 2050/146 : Display means
• CPC Class Codes: B60W 2420/403 : Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2540/21 : Voice
• CPC Class Codes: B60W 2540/22 : Psychological state; Stress level or workload
• CPC Class Codes: B60W 50/0098 : Details of control systems ensuring comfort, safety or stability not otherwise provided for
• CPC Class Codes: B60W 50/14 : Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G01C 21/36 : Input/output arrangements for on-board computers
• CPC Class Codes: G01C 21/3602 : Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 21/3617 : using user history, behaviour, conditions or preferences, e.g. predicted or inferred from previous use or current movement
• CPC Class Codes: G08G 1/04 : using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/096827 : where the route is computed onboard
• CPC Class Codes: G08G 1/096833 : where different aspects are considered when computing the route
• CPC Class Codes: G08G 1/096855 : where the output is provided in a suitable form to the driver details on I/O arrangements for onboard navigation computers G01C21/36
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

In one embodiment, a cognitive load driving assistant increases driving safety based on cognitive loads. In operation, the cognitive load driving assistant computes a current cognitive load of a driver based on sensor data. If the current cognitive load exceeds a threshold cognitive load, then the cognitive load driving assistant modifies the driving environment to reduce the cognitive load required to perform the primary driving task and/secondary task(s), such as texting via a cellular phone. The cognitive load driving assistant may modify the driving environment indirectly via sensory feedback to the driver or directly through reducing the complexity of the primary driving task and/or secondary tasks. In particular, if the driver is exhibiting elevated cognitive loads typically associated with distracted driving, then the cognitive load driving assistant modifies the driving environment to allow the driver to devote appropriate mental resources to the primary driving task, thereby increasing driving safety.

Claims

1. One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:

computing a current cognitive load associated with a driver while the driver is operating a vehicle based on data received via one or more sensors;

computing a confidence factor associated with the current cognitive load based on data received via one or more other sensors;

computing a threshold cognitive load;

determining that the current cognitive load exceeds the threshold cognitive load; and

in response, causing one or more actions to occur that are intended to reduce the current cognitive load associated with the driver.

2. The one or more computer-readable storage media of claim 1, wherein causing one or more actions to occur comprises configuring a display screen to display the current cognitive load.

3. The one or more computer-readable storage media of claim 1, wherein causing one or more actions to occur comprises configuring an entertainment subsystem to reduce the volume of an audio signal.

4. The one or more computer-readable storage media of claim 1, wherein causing one or more actions to occur comprises causing a navigation subsystem to generate a modified driving route that has less cognitive complexity than a current driving route.

5. The one or more computer-readable storage media of claim 1, wherein the one or more sensors measure one or more physiological changes in a body of the driver.

6. The one or more computer-readable storage media of claim 5, wherein the one or more physiological changes are associated with at least one of a brain activity, a heart rate, a skin conductance, and sizes of pupils.

7. The one or more computer-readable storage media of claim 5, wherein computing the current cognitive load comprises:

performing a multiplication operation between a first measurement received via a first sensor included in the one or more sensors and a first weight to generate a first weighted measurement;

performing a multiplication operation between a second measurement received via a second sensor included in the one or more sensors and a second weight to generate a second weighted measurement; and

computing an average of the first weighted measurement and the second weighted measurement.

8. The one or more computer-readable storage media of claim 1, wherein at least one or the one or more sensors measures sizes of pupils and the one or more other sensors measure at least one of a brain activity, a heart rate, and a skin conductance.

9. The one or more computer-readable storage media of claim 1, further comprising computing the threshold cognitive load based on a plurality of previous cognitive loads computed for the driver.

10. The one or more computer-readable storage media of claim 1, further comprising computing the threshold cognitive load based on a plurality of current cognitive loads computed for a plurality of other drivers.

11. A method for sharing cognitive load while driving, the method comprising:

computing a first cognitive load of a first driver based on data received via one or more sensors, wherein the first driver and the one or more sensors are associated with a first vehicle;

sharing the first cognitive load with a second driver that is associated with a second vehicle;

receiving a second cognitive load that is associated with the second driver; and

in response to receiving the second cognitive load, modifying a characteristic of the first vehicle.

12. The method of claim 11, wherein sharing the first cognitive load comprises storing the first cognitive load in a cognitive load database included in a cloud that is accessible to the second vehicle.

13. The method of claim 11, wherein sharing the first cognitive load comprises transmitting vehicle-to-everything data that includes the first cognitive load to the second vehicle.

14. A system configured to manage cognitive load while driving, the system comprising:

a memory storing a cognitive load driving assistant; and

a processor that is coupled to the memory and, when executing the cognitive load driving assistant, is configured to;

compute a current cognitive load associated with a driver, while the driver is operating a vehicle, based on data received via one or more sensors;

compute a confidence factor associated with the current cognitive load based on data received via one or more other sensors;

determine a threshold cognitive load;

determine that the current cognitive load exceeds the threshold cognitive load; and

taking one or more actions that are intended to reduce the current cognitive load below the threshold cognitive load.

15. The system of claim 14, wherein taking one or more actions comprises configuring a display screen to display the current cognitive load.

16. The system of claim 14, wherein taking one or more actions comprises configuring an advanced driver assistance system (ADAS) to increase the conservatism of one or more safety features associated with the vehicle.

17. The system of claim 14, wherein the processor is further configured to:

compute a current driving context based on data received via one or more other sensors;

select a previous driving context from a cognitive load database based on the current driving context; and

assign the threshold cognitive load based on a previous cognitive load associated with the previous driving context.