System and method to monitor and alert vehicle operator of impairment

• US 9,283,847 B2
• Filed: 05/05/2014
• Issued: 03/15/2016
• Est. Priority Date: 05/05/2014
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Riley, Matthew Eric Sr., Carter, Gregory Dwayne, Freeman, James M., Pillai, Vaidya Balasubramanian, Chang, Jason, Thurber, David W.
• Primary Examiner(s): Lim, Steven
• Assistant Examiner(s): SHERWIN, RYAN W
• Application Number:
  US14/269,490
• Publication Number:
  US 20150314681A1
• Time in Patent Office: 680 Days
• Field of Search: 340/573.1, 340/575, 340/576, 340/691.1, 340/691.7
• US Class Current: 1/1
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/02055 Simultaneously evaluating both cardiovascular condition and temperature
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/0533 Measuring galvanic skin response
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/6802 Sensor mounted on worn items
• CPC Class Codes: A61B 5/7455 characterised by tactile indication, e.g. vibration or electrical stimulation
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 2040/0827 due to sleepiness
• CPC Class Codes: B60W 2540/221 Physiology, e.g. weight, heartbeat, health or special needs
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: B60W 50/16 Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
• CPC Class Codes: B60Y 2400/902 giving haptic or tactile signals
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

The method, system, and computer-readable medium facilitates monitoring a vehicle operator during the course of vehicle operation to determine whether the vehicle operator is impaired (e.g., distracted, drowsy, intoxicated) and alerting the vehicle operator using a haptic alert delivered by a wearable computing device worn by the vehicle operator when impairment is detected. The method, system, and computer-readable medium may monitor the vehicle operator, the environment surrounding the vehicle, and/or forces acting on the vehicle using a variety of sensors, including optical sensors or accelerometers. In particular, optical sensors may monitor the vehicle operator to detect eye blinks, head nods, head rotations, and/or gaze fixation. Optical sensors may also monitor the road ahead of the vehicle to detect lane deviation, lane centering, and time to collision. Accelerometers may detect acceleration in the direction of vehicle travel and/or lateral acceleration.

Claims

1. A computer-implemented method for alerting a vehicle operator of operator impairment comprising:

receiving sensor data regarding the potential impairment of the vehicle operator from one or more sensors;

determining, using one or more processors, whether the sensor data indicates the vehicle operator is impaired by (i) calculating a plurality of impairment indicator scores using the received sensor data, (ii) calculating an impairment score associated with a type of impairment based upon the plurality of impairment indicator scores, and (iii) comparing the impairment score to a threshold value to determine whether the vehicle operator is impaired;

selecting, using one or more processors, one or more impairment alerts from a database of potential responses based on the sensor data when the vehicle operator is determined to be impaired, wherein selecting the one or more impairment alerts includes selecting a haptic alert from a plurality of haptic alerts from the database of potential responses; and

presenting the one or more selected impairment alerts to the vehicle operator, wherein the haptic alert is presented using a wearable computing device worn by the vehicle operator and wherein presenting the one or more impairment alerts include automatically preventing presentation of a subsequent alert within a predetermined reset period following presentation of the one or more impairment alerts.

2. The method of claim 1, further comprising presenting the one or more impairment alerts using a smart watch.

3. The method of claim 2, further comprising causing the smart watch to provide sensor data regarding the vehicle operator to a control device within the vehicle and wherein the control device uses one or more processors to receive the sensor data from the watch and one or more other sensors, determine whether the vehicle operator is impaired, and instruct the watch to present the haptic alert to the vehicle operator.

4. The method of claim 2, further comprising terminating the one or more impairment alerts by performing a physical movement of the wrist.

5. The method of claim 1, further comprising causing a control device within the vehicle to receive the sensor data, determine whether the sensor data indicates the vehicle operator is impaired, and control the presentation of impairment alerts by the wearable computing device worn by the vehicle operator.

6. The method of claim 5, further comprising receiving at the control device sensor data regarding one or more of the following:

a vehicle operator head nod, a vehicle operator scanning frequency, a vehicle operator gaze fixation, a vehicle operator mirror checking, a vehicle operator head rotation, a vehicle operator arm movement, a vehicle operator skin conductivity, a vehicle operator temperature, a vehicle operator pulse rate, a vehicle lane deviation, a vehicle swerving, a vehicle lane centering, a vehicle acceleration along a single axis or multiple axis, or a vehicle distance to other objects.

7. The method of claim 5, wherein the control device is a mobile telephone.

8. The method of claim 5, further comprising causing the control device to transmit the sensor data to one or more servers through a network and determining, with the one or more servers, whether the sensor data indicates the vehicle operator is impaired.

9. The method of claim 1, further comprising terminating the one or more impairment alerts with a first timer element after a specified time-out period.

10. The method of claim 1, further comprising:

receiving a first set of data from the one or more sensors in the vehicle;

determining one or more calibration adjustments to the received first set of data to delineate features of the vehicle operator or the vehicle environment; and

applying the one or more calibration adjustments to the received sensor data prior to determining whether the sensor data indicates the vehicle operator is impaired.

11. A computer system for alerting a vehicle operator of operator impairment comprising:

one or more processors;

one or more databases storing information regarding potential responses to impairments;

one or more sensors;

one or more wearable computing devices; and

a program memory storing executable instructions that when executed by the one or more processors cause the computer system to;

receive sensor data regarding the vehicle operator from the one or more sensors;

determine whether the vehicle operator is impaired based on the received sensor data by (i) calculating a plurality of impairment indicator scores using the received sensor data, (ii) calculating an impairment score associated with a type of impairment based upon the plurality of impairment indicator scores, and (iii) comparing the impairment score to a threshold value to determine whether the vehicle operator is impaired;

select one or more impairment alerts from the one or more databases based on the sensor data when the vehicle operator is determined to be impaired, wherein selecting the one or more impairment alerts includes selecting a haptic alert from a plurality of haptic alerts from the database of potential responses; and

deliver the one or more selected impairment alerts to the vehicle operator, wherein the haptic alert is delivered to the vehicle operator using the one or more wearable computing devices and wherein delivering the haptic alert includes automatically preventing delivery of a subsequent alert within a predetermined reset period following delivery of the haptic alert.

12. The computer system of claim 11, wherein the computer system further includes a control device communicatively coupled with the wearable computing device that executes the executable instructions stored in the program memory using the one or more processors to:

receive the sensor data;

determine whether the vehicle operator is impaired based on the sensor data; and

cause the wearable computing device to deliver the haptic alert to the vehicle operator when the vehicle operator is determined to be impaired.

13. The computer system of claim 12, wherein the control device is connected to one or more networked servers and wherein the networked servers determine, using one or more processors, whether the vehicle operator is impaired based on the sensor data.

14. The computer system of claim 11, wherein the wearable computing device used to deliver the haptic alert is a smart watch.

15. The computer system of claim 14, wherein the watch provides sensor data regarding the vehicle operator, including one or more of the following:

accelerometer data, pulse data, skin resistivity data, thermal data, and optical data.

16. A tangible, non-transitory computer-readable medium storing instructions that when executed by one or more processors of a computer system cause the computer system to:

monitor a vehicle operator using sensor data from one or more sensors;

determine whether the vehicle operator is impaired based on the received sensor data by (i) calculating a plurality of impairment indicator scores using the received sensor data, (ii) calculating an impairment score associated with a type of impairment based upon the plurality of impairment indicator scores, and (iii) comparing the impairment score to a threshold value to determine whether the vehicle operator is impaired;

select one or more impairment alerts from a database of potential responses based on the received sensor data when the vehicle operator is determined to be impaired, wherein selecting the one or more impairment alerts includes selecting a haptic alert from a plurality of haptic alerts from the database of potential responses; and

cause the selected one or more impairment alerts to be provided to the vehicle operator, wherein one or more wearable computing devices are caused to provide the haptic alert to the vehicle operator and wherein the one or more wearable computing devices are automatically prevented from providing a subsequent alert within a predetermined reset period following provision of the haptic alert.

17. The tangible, non-transitory computer-readable medium of claim 16, further comprising executable instructions that when executed by the one or more processors cause the haptic alert to be provided to the vehicle operator by a smart watch.

18. The tangible, non-transitory computer-readable medium of claim 17, wherein the executable instructions that when executed by the one or more processors cause the computer system to monitor the vehicle operator and determine whether the vehicle operator is impaired are executed by one or more processors of a control device within the vehicle.

19. The tangible, non-transitory computer-readable medium of claim 17, wherein the executable instructions that when executed by the one or more processors cause the computer system to monitor the vehicle operator and determine whether the vehicle operator is impaired are executed by one or more processors of one or more networked servers.

20. The tangible, non-transitory computer-readable medium of claim 17, wherein the executable instructions that when executed by the one or more processors cause the computer system to monitor the vehicle operator include executable instructions that when executed by the one or more processors cause the computer system to monitor the vehicle operator using one or more of the following sensor data from the watch:

accelerometer data, pulse data, skin resistivity data, thermal data, or optical data.