System and method to monitor and alert vehicle operator of impairment

• US 10,569,650 B1
• Filed: 10/03/2018
• Issued: 02/25/2020
• Est. Priority Date: 05/05/2014
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Riley, Sr., Matthew Eric, Carter, Gregory Dwayne, Freeman, James M., Pillai, Vaidya Balasubramanian, Chang, Jason, Thurber, David W.
• Primary Examiner(s): Sherwin, Ryan W
• Application Number:
  US16/150,780
• Time in Patent Office: 510 Days
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/02055 Simultaneously evaluating both cardiovascular condition and temperature
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/0533 Measuring galvanic skin response
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/6802 Sensor mounted on worn items
• CPC Class Codes: A61B 5/7455 characterised by tactile indication, e.g. vibration or electrical stimulation
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 2040/0827 due to sleepiness
• CPC Class Codes: B60W 2540/221 Physiology, e.g. weight, heartbeat, health or special needs
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: B60W 50/16 Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
• CPC Class Codes: B60Y 2400/902 giving haptic or tactile signals
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

The method, system, and computer-readable medium facilitates monitoring a vehicle operator during the course of vehicle operation to determine whether the vehicle operator is impaired (e.g., distracted, drowsy, intoxicated) and alerting the vehicle operator using a haptic alert delivered by a wearable computing device worn by the vehicle operator when impairment is detected. The method, system, and computer-readable medium may monitor the vehicle operator, the environment surrounding the vehicle, and/or forces acting on the vehicle using a variety of sensors, including optical sensors or accelerometers. In particular, optical sensors may monitor the vehicle operator to detect eye blinks, head nods, head rotations, and/or gaze fixation. Optical sensors may also monitor the road ahead of the vehicle to detect lane deviation, lane centering, and time to collision. Accelerometers may detect acceleration in the direction of vehicle travel and/or lateral acceleration.

Claims

1. A computer-implemented method for alerting a vehicle operator of operator impairment using a wearable computing device worn by the vehicle operator, comprising:

determining, by a mobile computing device associated with the vehicle operator, that the mobile computing device is within a threshold distance of a vehicle, wherein the vehicle is a particular vehicle;

generating, by the mobile computing device, a command to begin monitoring the vehicle operator based upon the determination that the mobile computing device is within the threshold distance;

monitoring, by the mobile computing device, the vehicle operator by;

causing one or more sensors to generate sensor data regarding the vehicle operator, wherein the one or more sensors are disposed within one or more of the following;

the vehicle, the mobile computing device, or the wearable computing device;

receiving the generated sensor data at one or more processors of the mobile computing device;

processing the received sensor data to determine one or more impairment scores indicating one or more impairment levels of the vehicle operator;

determining that the vehicle operator is impaired when at least one of the one or more impairment scores is outside a range of values indicative of an unimpaired state;

selecting an alert to warn the vehicle operator based upon the determination that the vehicle operator is impaired, wherein the alert includes a haptic alert; and

communicating the alert to the wearable computing device via a wireless communication link; and

activating a haptic output unit of the wearable computing device to generate the alert.

2. The computer-implemented method of claim 1, wherein the mobile computing device determines it is within the threshold distance of the vehicle using global positioning system (GPS) data from a GPS unit within the mobile computing device.

3. The computer-implemented method of claim 1, wherein the mobile computing device receives sensor data from an on-board computer of the vehicle via a wireless communication link.

4. The computer-implemented method of claim 1, wherein the sensor data regarding the vehicle operator indicates one or more of the following:

a vehicle operator head nod, a vehicle operator scanning frequency, a vehicle operator gaze fixation, a vehicle operator mirror checking, a vehicle operator head rotation, a vehicle operator arm movement, a vehicle operator skin conductivity, a vehicle operator temperature, or a vehicle operator pulse rate.

5. The computer-implemented method of claim 1, wherein the selected alert further includes a flashing image, and further comprising controlling a display of the wearable computing device to display the flashing image.

6. The computer-implemented method of claim 1, wherein the wearable computing device is a smart watch.

7. The computer-implemented method of claim 1, further comprising:

monitoring movement of the wearable computing device using accelerometer data from an accelerometer array within the wearable computing device while the alert is being generated;

determining that the wearable computing device is being shaken based upon the accelerometer data; and

disabling the alert in response to the determination that the wearable computing device is being shaken.

8. The computer-implemented method of claim 1, wherein monitoring the vehicle operator further comprises calibrating at least one of the one or more sensors.

9. The computer-implemented method of claim 8, wherein calibration includes adjusting the at least one of the one or more sensors to account for one or more of the following:

ambient lighting conditions, time of day, or weather conditions.

10. A computer system for alerting a vehicle operator of operator impairment comprising:

a mobile computing device having one or more processors and a program memory;

one or more wearable computing devices communicatively connected to the mobile computing device; and

one or more sensors communicatively connected to the mobile computing device;

wherein the program memory stores executable instructions that when executed by the one or more processors cause the mobile computing device to;

determine that the mobile computing device is within a threshold distance of a vehicle, wherein the vehicle is a particular vehicle;

generate a command to begin monitoring the vehicle operator based upon the determination that the mobile computing device is within the threshold distance;

monitor the vehicle operator by the following;

causing the one or more sensors to generate sensor data regarding the vehicle operator, wherein the one or more sensors are disposed within one or more of the following;

the vehicle, the mobile computing device, or the wearable computing device;

receiving the generated sensor data at the one or more processors of the mobile computing device;

processing the received sensor data to determine one or more impairment scores indicating one or more impairment levels of the vehicle operator;

determining that the vehicle operator is impaired when at least one of the one or more impairment scores is outside a range of values indicative of an unimpaired state; and

selecting an alert to warn the vehicle operator based upon the determination that the vehicle operator is impaired, wherein the alert includes a haptic alert; and

cause the wearable computing device to activate a haptic output unit of the wearable computing device to generate the alert by communicating the alert to the wearable computing device via a wireless communication link.

11. The computer system of claim 10, wherein the mobile computing device further includes a global positioning system (GPS) unit, and the mobile computing device determines it is within the threshold distance of the vehicle using GPS data from the GPS unit.

12. The computer system of claim 10, wherein the mobile computing device receives sensor data from an on-board computer of the vehicle via a wireless communication link.

13. The computer system of claim 10, wherein the wearable computing device is a smart watch.

14. The computer system of claim 10, wherein the sensor data regarding the vehicle operator indicates one or more of the following:

a vehicle operator head nod, a vehicle operator scanning frequency, a vehicle operator gaze fixation, a vehicle operator mirror checking, a vehicle operator head rotation, a vehicle operator arm movement, a vehicle operator skin conductivity, a vehicle operator temperature, or a vehicle operator pulse rate.

15. The computer system of claim 10, wherein:

the selected alert further includes a flashing image, and

the program memory further stores executable instructions that when executed by the one or more processors cause the mobile computing device to cause a display of the wearable computing device to display the flashing image.

16. A tangible, non-transitory computer-readable medium storing instructions for alerting a vehicle operator of operator impairment using a wearable computing device worn by the vehicle operator that, when executed by one or more processors of a computer system, cause the computer system to:

determine that a mobile computing device associated with the vehicle operator is within a threshold distance of a vehicle, wherein the vehicle is a particular vehicle;

generate a command to begin monitoring the vehicle operator based upon the determination that the mobile computing device is within the threshold distance;

monitor the vehicle operator by;

causing one or more sensors to generate sensor data regarding the vehicle operator, wherein the one or more sensors are disposed within one or more of the following;

the vehicle, the mobile computing device, or the wearable computing device;

receiving the generated sensor data at the one or more processors;

processing the received sensor data to determine one or more impairment scores indicating one or more impairment levels of the vehicle operator;

determining that the vehicle operator is impaired when at least one of the one or more impairment scores is outside a range of values indicative of an unimpaired state;

selecting an alert to warn the vehicle operator based upon the determination that the vehicle operator is impaired, wherein the alert includes a haptic alert; and

communicating the alert to the wearable computing device via a wireless communication link; and

activating a haptic output unit of the wearable computing device to generate the alert.

17. The tangible, non-transitory computer-readable medium of claim 16, further comprising executable instructions that when executed by the one or more processors cause the computer system to determine the mobile computing device is within the threshold distance of the vehicle using global positioning system (GPS) data from a GPS unit within the mobile computing device.

18. The tangible, non-transitory computer-readable medium of claim 16, further comprising executable instructions that when executed by the one or more processors cause the computer system to receive sensor data from an on-board computer of the vehicle via a wireless communication link.

19. The tangible, non-transitory computer-readable medium of claim 16, wherein:

the selected alert further includes a flashing image, and

further comprising executable instructions that when executed by the one or more processors cause the computer system to control a display of the wearable computing device to display the flashing image.

20. The tangible, non-transitory computer-readable medium of claim 16, wherein the wearable computing device is a smart watch.