System and method to monitor and reduce vehicle operator impairment

• US 8,981,942 B2
• Filed: 12/17/2012
• Issued: 03/17/2015
• Est. Priority Date: 12/17/2012
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: He, Jibo, Fields, Brian M., Roberson, Steve, Cielocha, Steve, Peng, Jufeng, Coltea, Julian
• Primary Examiner(s): Backer, Firmin
• Assistant Examiner(s): TUN, NAY L
• Application Number:
  US13/717,506
• Publication Number:
  US 20140167967A1
• Time in Patent Office: 820 Days
• Field of Search: 340/576, 340/575, 340/521, 340/522, 340/584, 340/600, 340/439
• US Class Current: 340/576
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: B60Q 9/00 Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 , e.g. haptic signalling
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 2040/0827 due to sleepiness
• CPC Class Codes: B60W 2040/0836 due to alcohol
• CPC Class Codes: B60W 2040/0845 due to drugs
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/42 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2540/221 Physiology, e.g. weight, heartbeat, health or special needs
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 50/16 Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G07C 5/008 communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G08B 21/02 Alarms for ensuring the safety of persons

Abstract

The method, system, and computer-readable medium facilitates monitoring a vehicle operator, the environment ahead of the vehicle, and/or forces acting on the vehicle during the course of vehicle operation to determine whether the vehicle operator is impaired (e.g., distracted, drowsy, intoxicated), alerting the vehicle operator when impairment is detected, and log data relating to vehicle operator impairment for further analysis. The method, system, and computer-readable medium may monitor the vehicle operator, the environment ahead of the vehicle, and/or forces acting on the vehicle using either or both of optical sensors or accelerometers. In particular, one optical sensor may monitor the vehicle operator to detect eye blinks, head nods, head rotations, and/or gaze fixation. Another optical sensor may monitor the road ahead of the vehicle to detect lane deviation, lane centering, and time to collision. The accelerometers may detect acceleration in the direction of vehicle travel and/or lateral acceleration.

Claims

1. A method comprising:

gathering data about potential vehicle operator impairment by;

monitoring a vehicle operator with a first optical sensor,

monitoring the environment ahead of the vehicle with a second optical sensor, and

monitoring force in one or more directions with an accelerometer,

wherein the data about potential vehicle operator impairment is stored on a computer-readable medium;

assigning, by one or more processors, a plurality of scores based on the data about potential operator impairment, wherein each of the plurality of scores corresponds to a respective impairment indicator;

generating, by the one or more processors, an impairment score by performing a mathematical operation on the plurality of scores, wherein the impairment score is stored on a computer-readable medium;

determining, by the one or more processors, whether the vehicle operator is impaired by comparing the impairment score generated by performing the mathematical operation on the plurality of scores to a threshold value;

when it is determined that the vehicle operator is impaired, alerting the vehicle operator using at least one or more of an audible alert, a visual alert, and a tactile alert;

analyzing, by the one or more processors, the data about potential vehicle operator impairment to identify recommendations to reduce operator impairment; and

electronically communicating the recommendations to reduce operator impairment to the vehicle operator.

2. The method of claim 1, wherein monitoring the vehicle operator with the optical sensor to gather data about potential vehicle operator impairment includes one or more of:

monitoring vehicle operator blinks, including monitoring one or more of number, frequency, and duration of blinks;

monitoring vehicle operator head nods;

monitoring vehicle operator scanning frequency;

or

monitoring vehicle operator gaze fixation.

3. The method of claim 1, wherein monitoring the vehicle operator with the optical sensor includes:

monitoring vehicle operator mirror checking;

monitoring vehicle operator head rotations;

monitoring vehicle operator scanning frequency; and

monitoring vehicle operator gaze fixation.

4. The method of claim 1, wherein monitoring the environment ahead of the vehicle with the optical sensor includes:

monitoring lane deviation;

monitoring lane centering; and

monitoring time to collision.

5. The method of claim 1, wherein monitoring force in one or more directions with an accelerometer includes one or more of:

monitoring fore-aft force to detect hard braking;

monitoring fore-aft force change in speed;

or

monitoring lateral force.

6. The method of claim 1, wherein the mathematical operation is a weighted sum.

7. The method of claim 1, wherein the impairment score is inputted to a game.

8. The method of claim 1, further comprising sending to one of one or more computers the data about potential vehicle operator impairment.

9. The method of claim 8, wherein the one of one or more computers analyzes the data about potential vehicle operator impairment, the method further comprising adjusting an insurance rate based on the analysis.

10. The method of claim 1, wherein the recommendations to reduce operator impairment include one or more of:

avoiding operating a vehicle during at least one specific time period;

reducing mobile device use;

reducing use of vehicle features;

or

ceasing vehicle operation.

11. The method of claim 1, further comprising communicating an analysis of the data about potential vehicle operator impairment as one or more of a graph, chart, table, or log of data entries.

12. A computer system comprising:

a processor;

one or more optical sensors;

a program memory storing executable instructions that when executed by the processor cause the computer system to;

gather data about potential vehicle operator impairment by;

monitor a vehicle operator with an optical sensor,

monitor the environment ahead of the vehicle with an optical sensor, and

monitor force in one or more directions with an accelerometer,

wherein the data about potential vehicle operator impairment is stored on a computer-readable medium;

assign a plurality of scores based on the data about potential operator impairment, wherein each of the plurality of scores corresponds to a respective impairment indicator;

generate an impairment score by performing a mathematical operation on the plurality of scores, wherein the impairment score is stored on a computer-readable medium;

determine whether the vehicle operator is impaired by comparing the impairment score generated by performing the mathematical operation on the plurality of scores to a threshold value;

when it is determined that the vehicle operator is impaired, alert the vehicle operator using at least one or more of an audible alert, a visual alert, and a tactile alert;

analyze, with the processor, the data about potential vehicle operator impairment to identify recommendations to reduce operator impairment; and

communicate the recommendations to reduce operator impairment to the vehicle operator.

13. The computer system of claim 12, wherein the executable instructions that when executed by the processor cause the computer system to monitor a vehicle operator with the optical sensor include executable instructions that when executed by the processor cause the computer system to:

monitor vehicle operator blinks, including monitoring one or more of a number, a frequency, or a duration of blinks;

monitor vehicle operator head nods;

monitor vehicle operator scanning frequency; and

monitor vehicle operator gaze fixation.

14. The computer system of claim 12, wherein the executable instructions that when executed by the processor cause the computer system to monitor a vehicle operator with the optical sensor include executable instructions that when executed by the processor cause the computer system to:

monitor vehicle operator mirror checking;

monitor vehicle operator head rotations;

monitor vehicle operator scanning frequency; and

monitor vehicle operator gaze fixation.

15. A computer system comprising:

a processor;

one or more optical sensors;

a program memory storing executable instructions that when executed by the processor cause the computer system to;

gather data about potential vehicle operator impairment by;

monitor a vehicle operator with an optical sensor,

monitor the environment ahead of the vehicle with an optical sensor, and

monitor force in one or more directions with an accelerometer,

wherein the data about potential vehicle operator impairment is stored on a computer-readable medium;

send to one of one or more computers the data about potential vehicle operator impairment;

receive a determination from the one of one or more computers whether the vehicle operator is impaired, wherein the determination from the one of one or more computers is based on comparing an impairment score to a threshold value,

the impairment score being generated by performing a mathematical operation on a plurality of scores,

each of the plurality of scores being based on the data about potential operator impairment, and

each of the plurality of scores corresponding to a respective impairment indicator;

when a determination that the vehicle operator is impaired is received, alert the vehicle operator using at least one or more of an audible alert, a visual alert, and a tactile alert;

receive recommendations to reduce operator impairment; and

communicate the recommendations to reduce operator impairment to the vehicle operator.

16. The computer system of claim 15, wherein the executable instructions that when executed by the processor cause the computer system to monitor a vehicle operator with the optical sensor include executable instructions that when executed by the processor cause the computer system to:

monitor vehicle operator blinks, including monitoring one or more of a number, a frequency, and a duration of blinks;

monitor vehicle operator head nods;

monitor vehicle operator scanning frequency; and

monitor vehicle operator gaze fixation.

17. The computer system of claim 15, wherein the executable instructions that when executed by the processor cause the computer system to monitor a vehicle operator with the optical sensor include executable instructions that when executed by the processor cause the computer system to:

monitor vehicle operator mirror checking;

monitor vehicle operator head rotations;

monitor vehicle operator scanning frequency; and

monitor vehicle operator gaze fixation.

18. A non-transitory, computer-readable medium storing instructions that when executed by a process of a computer system cause the computer system to:

gather data about potential vehicle operator impairment by;

monitor a vehicle operator with an optical sensor,

monitor the environment ahead of the vehicle with an optical sensor, and

monitor force in one or more directions with an accelerometer,

wherein the data about potential vehicle operator impairment is stored on a computer-readable medium;

assign a plurality of scores based on the data about potential operator impairment, wherein each of the plurality of scores corresponds to a respective impairment indicator;

generate an impairment score by performing a mathematical operation on the plurality of scores, wherein the impairment score is stored on a computer-readable medium;

determine whether the vehicle operator is impaired by comparing the impairment score generated by performing the mathematical operation on the plurality of scores to a threshold value;

when it is determined that the vehicle operator is impaired, alert the vehicle operator using at least one or more of an audible alert, a visual alert, and a tactile alert;

analyze, with the processor of the computer system, the data about potential vehicle operator impairment to identify recommendations to reduce operator impairment; and

communicate the recommendations to reduce operator impairment to the vehicle operator.

19. The non-transitory, computer-readable medium of claim 18, wherein the instructions that when executed by the processor cause the computer system to monitor a vehicle operator with the optical sensor include instructions that when executed by the processor cause the computer system to:

monitor vehicle operator blinks, including monitoring one or more of a number, a frequency, and a duration of blinks;

monitor vehicle operator head nods;

monitor vehicle operator scanning frequency;

or

monitor vehicle operator gaze fixation.

20. The non-transitory, computer-readable medium of claim 18, wherein the instructions that when executed by the processor cause the computer system to monitor a vehicle operator with the optical sensor include instructions that when executed by the processor cause the computer system to:

monitor vehicle operator mirror checking;

monitor vehicle operator head rotations;

monitor vehicle operator scanning frequency; and

monitor vehicle operator gaze fixation.