Vehicle operator emotion management system and method

• US 9,934,667 B1
• Filed: 05/19/2017
• Issued: 04/03/2018
• Est. Priority Date: 03/07/2014
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Fields, Brian Mark, Weeks, Samuel C., Chuenchujit, Thasphon, Yoshie Stiles, Kelsey Catherine, Diaz, Allison Lynne, Deshmukh, Alisha, Angal, Mihir D., Duseja, Akash, Chan, Aaron Scott, Donovan, Matthew Christopher Jon
• Primary Examiner(s): Nguyen, Tien C
• Application Number:
  US15/600,125
• Time in Patent Office: 319 Days
• Field of Search: 705 35, 705 2, 705 3, 705 4, 705 5, 705 38, 705 39, 705 37, 303191, 382104, 382117, 340576, 340435, 340439, 345173, 219204, 348143, 348158, 434236, 701 1, 702 32, 709203, 709232, 700300, 600301, 600509, 600545, 600 26, 707758
• CPC Class Codes: G06Q 30/0283 Price estimation or determination
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G08B 21/02 Alarms for ensuring the safety of persons

Abstract

The method, system, and computer-readable medium facilitates monitoring a vehicle operator during the course of vehicle operation to determine whether the vehicle operator is in an emotionally impaired state (e.g., aggressive or agitated) and presenting appropriate stimuli (e.g., music or sound recordings) to the vehicle operator when impairment is detected. The vehicle operator, the environment surrounding the vehicle, or forces acting on the vehicle may be monitored using a variety of sensors, including optical sensors, accelerometers, or biometric sensors (e.g., skin conductivity, heart rate, or voice modulation). When the vehicle operator is determined to be in an emotionally impaired state, stimuli are selected to improve the emotional state of the vehicle operator. The selection is based on sensor data and data regarding prior responses of the vehicle operator to various stimuli. After selection, the stimuli are presented to the vehicle operator while monitoring continues.

Claims

1. A computer-implemented method for managing an emotional state of a vehicle operator of a vehicle, comprising:

collecting, by one or more physiological sensors within the vehicle, sensor data regarding one or more physiological states of the vehicle operator, wherein the sensor data is physiological data regarding the vehicle operator;

calculating, by one or more processors, a plurality of impairment indicator scores using the sensor data, wherein each impairment indicator score indicates an extent of vehicle operator deviation from a normal emotional state with respect to a physiological metric;

calculating, by one or more processors, one or more total impairment scores as a weighted combination of two or more of the plurality of impairment indicator scores, each total impairment score indicating whether the vehicle operator is in an impaired emotional state;

determining, by one or more processors, the vehicle operator is in an impaired emotional state based on the one or more total impairment scores;

selecting, by one or more processors, one or more musical stimuli to improve the emotional state of the vehicle operator by predicting the ability of the stimuli to improve the emotional state of the vehicle operator based on the received sensor data and a user profile associated with the vehicle operator;

determining, by one or more processors, that a sound component of the vehicle is currently in use;

detecting, by one or more processors, a time at which the sound component is available to play the one or more musical stimuli; and

when the sound component is detected to be available, causing, by controlling the sound component within the vehicle and while the vehicle operator is in the impaired emotional state, the one or more musical stimuli to be played to the user to improve the emotional state of the vehicle operator.

2. The computer-implemented method of claim 1, wherein the one or more musical stimuli include one or more recorded songs.

3. The computer-implemented method of claim 1, wherein detecting the time at which the sound component is available to play the one or more musical stimuli includes detecting the end of a currently playing song.

4. The computer-implemented method of claim 1, further comprising:

receiving, at one or more processors, an indication from the vehicle operator to change the musical stimuli;

updating, by one or more processors, the user profile based upon the received indication;

selecting, by one or more processors, one or more alternate musical stimuli based upon the updated user profile; and

causing, by controlling the sound component within the vehicle, the one or more alternate musical stimuli to be played to the user to improve the emotional state of the vehicle operator.

5. The computer-implemented method of claim 1, further comprising:

receiving, at one or more processors, an indication from the vehicle operator to stop playing musical stimuli;

updating, by one or more processors, the user profile based upon the received indication; and

stopping, by controlling the sound component within the vehicle, the one or more musical stimuli from continuing to be played to the user.

6. The computer-implemented method of claim 1, further comprising:

collecting, by one or more vehicle motion sensors within the vehicle, vehicle sensor data regarding movement of the vehicle during operation; and

calculating, by one or more processors, a plurality of operational impairment indicator scores using the vehicle sensor data, wherein each impairment indicator score indicates a metric associated with proper vehicle operation by the vehicle operator,

wherein at least one of the one or more total impairment scores is calculated as a weighted average of two or more of the impairment indicator scores and one or more of the operational impairment indicator scores.

7. The computer-implemented method of claim 1, wherein the received sensor data comprises one or more of the following types of physiological sensor data regarding the vehicle operator:

a heart rate, heart rate variability data, a grip pressure, electrodermal activity data, a body temperature, an arm movement, a head movement, a vocal amplitude, a vocal frequency, a vocal pattern, a gaze direction, a gaze duration, a head direction, an eyelid opening, a blink rate, pupillometry data, a blood pressure, electroencephalographic data, a respiration rate, a respiration pattern, a galvanic skin response, functional near infrared optical brain imaging data, functional magnetic resonance imaging data, or electromyographic data.

8. The computer-implemented method of claim 1, wherein the user profile for the vehicle operator contains information relating prior stimuli presented to the vehicle operator to prior sensor data received from the one or more physiological sensors at the time the prior stimuli were presented over a period of at least one week.

9. The computer-implemented method of claim 1, wherein determining, by one or more processors, the vehicle operator is in an impaired emotional state further comprises:

determining, by one or more processors, that one or more of the total impairment scores fails to reach a minimum threshold value associated with unimpaired vehicle operation or exceeds a maximum threshold value associated with unimpaired vehicle operation.

10. A computer system for managing an emotional state of a vehicle operator of a vehicle, comprising:

one or more processors;

one or more physiological sensors within the vehicle; and

a program memory storing executable instructions that, when executed by the one or more processors, cause the computer system to;

collect sensor data regarding one or more physiological states of the vehicle operator using the one or more physiological sensors, wherein the sensor data is physiological data regarding the vehicle operator;

calculate a plurality of impairment indicator scores using the sensor data, wherein each impairment indicator score indicates an extent of vehicle operator deviation from a normal emotional state with respect to a physiological metric;

calculate one or more total impairment scores as a weighted combination of two or more of the plurality of impairment indicator scores, each total impairment score indicating whether the vehicle operator is in an impaired emotional state;

determine the vehicle operator is in an impaired emotional state based on the one or more total impairment scores;

select one or more musical stimuli to improve the emotional state of the vehicle operator by predicting the ability of the stimuli to improve the emotional state of the vehicle operator based on the received sensor data and a user profile associated with the vehicle operator;

determine that a sound component of the vehicle is currently in use;

detect a time at which the sound component is available to play the one or more musical stimuli; and

when the sound component is detected to be available and while the vehicle operator is in the impaired emotional state, cause the sound component within the vehicle to play the one or more musical stimuli to the user to improve the emotional state of the vehicle operator.

11. The computer system of claim 10, wherein the one or more musical stimuli include one or more recorded songs.

12. The computer system of claim 10, wherein the executable instructions that cause the computer system to detect the time at which the sound component is available to play the one or more musical stimuli further cause the computer system to detect the end of a currently playing song.

13. The computer system of claim 10, wherein the executable instructions further cause the computer system to:

receive an indication from the vehicle operator to change the musical stimuli;

update the user profile based upon the received indication;

select one or more alternate musical stimuli based upon the updated user profile; and

cause the sound component within the vehicle to play the one or more alternate musical stimuli to the user to improve the emotional state of the vehicle operator.

14. The computer system of claim 10, wherein the executable instructions further cause the computer system to:

receive an indication from the vehicle operator to stop playing musical stimuli;

update the user profile based upon the received indication; and

stop the sound component within the vehicle from continuing to play the one or more musical stimuli.

15. The computer system of claim 10, further comprising one or more vehicle motion sensors within the vehicle, wherein:

the executable instructions further cause the computer system to;

collect vehicle sensor data regarding movement of the vehicle during operation from the one or more vehicle motion sensors within the vehicle; and

calculate a plurality of operational impairment indicator scores using the vehicle sensor data, wherein each impairment indicator score indicates a metric associated with proper vehicle operation by the vehicle operator; and

at least one of the one or more total impairment scores is calculated as a weighted average of two or more of the impairment indicator scores and one or more of the operational impairment indicator scores.

16. A tangible, non-transitory computer-readable medium storing instructions that when executed by one or more processors of a computer system cause the computer system to:

monitor a vehicle operator of a vehicle using sensor data from one or more physiological sensors within the vehicle, wherein the sensor data is physiological data regarding the vehicle operator;

calculate a plurality of impairment indicator scores using the sensor data, wherein each impairment indicator score indicates an extent of vehicle operator deviation from a normal emotional state with respect to a physiological metric;

calculate one or more total impairment scores as a weighted combination of two or more of the plurality of impairment indicator scores, each total impairment score indicating whether the vehicle operator is in an impaired emotional state;

determine the vehicle operator is in an impaired emotional state based on the one or more total impairment scores;

select one or more musical stimuli to improve the emotional state of the vehicle operator by predicting the ability of the stimuli to improve the emotional state of the vehicle operator based on the received sensor data and a user profile associated with the vehicle operator;

determine that a sound component of the vehicle is currently in use;

detect a time at which the sound component is available to play the one or more musical stimuli; and

when the sound component is detected to be available and while the vehicle operator is in the impaired emotional state, cause the sound component within the vehicle to play the one or more musical stimuli to the user to improve the emotional state of the vehicle operator.

17. The tangible, non-transitory computer-readable medium of claim 16, wherein the one or more musical stimuli include one or more recorded songs.

18. The tangible, non-transitory computer-readable medium of claim 16, wherein the executable instructions that cause the computer system to detect the time at which the sound component is available to play the one or more musical stimuli further cause the computer system to detect the end of a currently playing song.

19. The tangible, non-transitory computer-readable medium of claim 16, further storing executable instructions further cause the computer system to:

receive an indication from the vehicle operator to change the musical stimuli;

update the user profile based upon the received indication;

select one or more alternate musical stimuli based upon the updated user profile; and

cause the sound component within the vehicle to play the one or more alternate musical stimuli to the user to improve the emotional state of the vehicle operator.

20. The tangible, non-transitory computer-readable medium of claim 16, further storing executable instructions further cause the computer system to:

receive an indication from the vehicle operator to stop playing musical stimuli;

update the user profile based upon the received indication; and

stop the sound component within the vehicle from continuing to play the one or more musical stimuli.