System for fatigue detection using a suite of physiological measurement devices

• US 9,771,081 B2
• Filed: 09/29/2014
• Issued: 09/26/2017
• Est. Priority Date: 09/29/2014
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Grube, Robert William, Thomas, Lisa C., Craig, Kimberly Marie, Gast, Christopher Marc
• Primary Examiner(s): Bousono, Orlando
• Application Number:
  US14/500,264
• Publication Number:
  US 20160090097A1
• Time in Patent Office: 1,093 Days
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: B60W 2040/0827 due to sleepiness
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

A system, method, and computer program product for detecting, in real time, a fatigue level of a vehicle operator. Multiple physiological indicators of an individual vehicle operator can be monitored at different instances when the operator'"'"'s level of fatigue is known and can be quantified. A statistical model of the vehicle operator'"'"'s fatigue can be developed from the data from the multiple instances. In future instances, the statistical model can be applied to real-time physiological data collected from the operator during operation of the vehicle to determine a fatigue level. A warning can be provided when the statistically-calculated fatigue level exceeds a threshold level.

Claims

1. A method for monitoring vehicle operator fatigue, the method comprising:

receiving a plurality of different biometric data related to a vehicle operator from a plurality of biometric sensors during real time operation of a vehicle;

calculating a fatigue level for the vehicle operator by applying the received biometric data to a statistical model of fatigue, wherein the statistical model of fatigue outputs an indication of the calculated fatigue level for the vehicle operator, wherein the statistical model of fatigue is created by;

recording biometric data about the vehicle operator from the plurality of biometric sensors during a plurality of previous operations of the vehicle, wherein each of the plurality of previous operations corresponds to a respective fatigue level of a plurality of corresponding fatigue levels of the vehicle operator relative to a predefined numerical scale, each of the plurality of corresponding fatigue levels being at least one of;

known, assumed, and self-reported by the vehicle operator; and

performing statistical analysis on the recorded biometric data to determine the statistical model of fatigue by ascertaining which statistical model of fatigue results in a best fit of the recorded biometric data to the plurality of corresponding fatigue levels for the plurality of previous operations.

2. The method of claim 1, wherein the received biometric data comprises at least one of:

electroencephalography (EEG) data;

electrooculography (EOG) data;

electromyography (EMG) data;

electrocardiography (EKG) data;

heart rate data;

blink rate data;

voice data;

head posture data;

body posture data;

eye movement data;

eye closure data;

body temperature data;

galvanic skin response data; and

video data.

3. The method of claim 1, further comprising, upon determining that the indication of the calculated fatigue level for the vehicle operator output by the statistical model of fatigue exceeds a threshold level of fatigue, outputting a first signal.

4. The method of claim 1, wherein the statistical model of fatigue includes an uncertainty interval for the indication of the calculated fatigue level for the vehicle operator output by the statistical model of fatigue;

and further comprising;

upon the indication of the calculated fatigue level for the vehicle operator output by the statistical model of fatigue exceeding a threshold level of fatigue, outputting a first signal to the vehicle operator; and

upon a lower end of the uncertainty interval of the indication of the calculated fatigue level for the vehicle operator output by the statistical model of fatigue exceeding the threshold level of fatigue, outputting a second signal to the vehicle operator.

5. The method of claim 1, further comprising outputting the indication of the calculated fatigue level for the vehicle operator output by the statistical model of fatigue to a computer system.

6. A system for monitoring fatigue of a vehicle operator in real time, the system comprising:

a plurality of biometric sensors arranged relative to controls for a vehicle that are operated by a vehicle operator, wherein the plurality of biometric sensors detect a plurality of different biometric data for the vehicle operator in real time;

memory on board the vehicle, the memory storing executable code and a statistical model of fatigue for outputting an indication of a calculated fatigue level for the vehicle operator; and

a processor which, when programmed by the executable code, applies the detected biometric data to the statistical model of fatigue to determine the calculated fatigue level for the vehicle operator, wherein the statistical model of fatigue is created by;

recording biometric data about the vehicle operator during a plurality of previous operations of the vehicle, wherein each of the plurality of previous operations corresponds to a respective fatigue level of a plurality of corresponding fatigue levels of the vehicle operator relative to a predefined numerical scale, each of the plurality of corresponding fatigue levels being at least one of;

known, assumed, and self-reported by the vehicle operator; and

performing statistical analysis on the recorded biometric data to determine the statistical model of fatigue by ascertaining which statistical model of fatigue results in a best fit of the recorded biometric data to the plurality of corresponding fatigue levels for the plurality of previous operations.

7. The system of claim 6, wherein the plurality of biometric sensors comprises at least two of:

an electroencephalography (EEG) sensor;

an electrooculography (EOG) sensor;

an electromyography (EMG) sensor;

an electrocardiography (EKG) sensor;

a heart rate sensor;

a blink rate sensor;

a voice sensor;

a head posture sensor;

a body posture sensor;

an eye movement sensor;

an eye closure sensor;

a body temperature sensor;

a galvanic skin response sensor; and

a video sensor.

8. The system of claim 6, wherein the statistical model of fatigue comprises a plurality of coefficients, based on the recorded biometric data, wherein the plurality of coefficients are set to values that result in the best fit of the recorded biometric data to the plurality of corresponding fatigue levels for the plurality of previous operations.

9. The system of claim 6, further comprising a warning indicator arranged relative to the controls and configured to display at least one warning indication;

and

wherein the processor is further configured to output the at least one warning indication to the warning indicator upon determining that the indication of the calculated fatigue level for the vehicle operator output by the statistical model of fatigue exceeds a threshold level of fatigue.

10. The system of claim 6, further comprising a warning indicator arranged relative to the controls, wherein the warning indicator is configured to display a first warning indication and a second warning indication;

wherein the memory further stores an uncertainty interval for the indication of the calculated fatigue level for the vehicle operator output by the statistical model of fatigue;

wherein the processor is further configured to;

output a first warning signal to the warning indicator upon determining that the indication of the calculated fatigue level for the vehicle operator output by the statistical model of fatigue exceeds a threshold level of fatigue, wherein the warning indicator displays the first warning indication in response to the first warning signal; and

output a second warning signal to the warning indicator upon determining that a lower end of the uncertainty interval of the indication of the calculated fatigue level for the vehicle operator output by the statistical model of fatigue exceeds the threshold level of fatigue, wherein the warning indicator displays the second warning indication in response to the second warning signal.

11. The system of claim 6, further comprising a transmitter in communication with the processor, wherein the processor is configured to output the indication of the calculated fatigue level for the vehicle operator output by the statistical model of fatigue to the transmitter for transmission to a computer system.

12. A computer-program product for monitoring, in real time, vehicle operator fatigue, the computer program product comprising:

a non-transitory computer-readable medium having computer-readable program code embodied therewith, the computer-readable program code configured to;

receive a plurality of different biometric data related to a vehicle operator from a plurality of biometric sensors during real time operation of a vehicle;

calculate a fatigue level for the vehicle operator by applying the received biometric data to a statistical model of fatigue, wherein the statistical model of fatigue outputs an indication of the calculated fatigue level for the vehicle operator, wherein the statistical model of fatigue is created by;

recording biometric data about the vehicle operator from the plurality of biometric sensors during a plurality of previous operations of the vehicle, wherein each of the plurality of previous operations corresponds to a respective fatigue level of a plurality of corresponding fatigue levels of the vehicle operator relative to a predefined numerical scale, each of the plurality of corresponding fatigue levels being at least one of;

known, assumed, and self-reported by the vehicle operator; and

performing statistical analysis on the recorded biometric data to determine the statistical model of fatigue by ascertaining which statistical model of fatigue results in a best fit of the recorded biometric data to the plurality of corresponding fatigue levels for the plurality of previous operations.

13. The computer-program product of claim 12, further comprising computer-readable program code configured to create the statistical model of fatigue, wherein performing statistical analysis on the recorded biometric data comprises determining coefficients related to data from each of the plurality of biometric sensors that result in the best fit of the recorded biometric data to the plurality of corresponding fatigue levels for the plurality of previous operations.

14. The computer-program product of claim 13, wherein the computer-readable program code is further configured to calculate an uncertainty interval for the indication of the calculated fatigue level for the vehicle operator output by the statistical model of fatigue.