IMAGING-BASED MONITORING OF STRESS AND FATIGUE

• US 20140375785A1
• Filed: 06/19/2013
• Published: 12/25/2014
• Est. Priority Date: 06/19/2013
• Status: Abandoned Application

• Current Assignee: Raytheon Company (Rtx Corporation)
• Original Assignee: Raytheon Company (Rtx Corporation)
• Inventors: Kogut, John A., Berte, Marc
• Application Number:
  US13/921,310
• Publication Number:
  US 20140375785A1
• Time in Patent Office: Days
• US Class Current: 348/77
• CPC Class Codes: A61B 2576/02 specially adapted for a particular organ or body part
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/0082 adapted for particular medical purposes
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: A61B 5/02405 Determining heart rate variability
• CPC Class Codes: A61B 5/02433 for infrared radiation
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/1103 Detecting eye twinkling
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 2540/26 Incapacity
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

An example system can monitor the stress and fatigue of a subject. The system can include a light source configured to direct illuminating light onto a face of the subject. The illuminating light can reflect off the face of the subject to form reflected light. The system can include collection optics that collect a portion of the reflected light and produce video-rate images of the face of the subject. The system can include an image processor configured to locate an eye in the video-rate images, extract fatigue signatures from the located eye, and determine a fatigue level of the subject, in part, from the fatigue signatures. The image processor can also be configured to locate a facial region away from the eye in the video-rate images, extract stress signatures from the located facial region, and determine a stress level of the subject from the stress signatures.

Claims

1. A system for monitoring stress and fatigue of a subject, the system comprising:

collection optics that collect a portion of light reflected from a face of the subject and produce video-rate images of the face of the subject; and

an image processor configured to;

locate an eye in the video-rate images;

extract fatigue signatures from the located eye;

determine a fatigue level of the subject, in part, from the fatigue signatures;

locate a facial region away from the eye in the video-rate images;

extract stress signatures from the located facial region; and

determine a stress level of the subject from the stress signatures.

2. The system of claim 1, further comprising a light source configured to direct illuminating light onto the face of the subject, the illuminating light reflecting off the face of the subject to form the reflected light.

3. The system of claim 2,

wherein the illuminating light has a spectrum that includes a first wavelength;

and

wherein the collection optics produce the video-rate images at the first wavelength.

4. The system of claim 3, wherein the collection optics comprise:

a spectral filter that transmits wavelengths in a wavelength band that includes the first wavelength and blocks wavelengths outside the transmitted wavelength band;

a lens configured to form an image of the face of the subject; and

a detector configured to detect the image of the face of the subject at the first wavelength.

5. The system of claim 3, wherein the first wavelength is in the infrared portion of the spectrum.

6. The system of claim 2,

wherein the illuminating light has a spectrum that includes first and second wavelengths;

and

wherein the collection optics produce first video-rate images at the first wavelength and produce second video-rate images at the second wavelength.

7. The system of claim 6, wherein the collection optics comprise:

a spectrally-sensitive beamsplitter that transmits wavelengths in a first wavelength band that includes the first wavelength, and reflects wavelengths in a second wavelength band that includes the second wavelength;

a lens configured to form a first image of the face of the subject at the first wavelength and a second image of the face of the subject at the second wavelength;

a first detector configured to detect the image of the face of the subject at the first wavelength; and

a second detector configured to detect the image of the face of the subject at the second wavelength.

8. The system of claim 7, wherein the image processor is configured to locate the eye in one of the first and second video-rate images and locate the facial region in the other of the first and second video-rate images.

9. The system of claim 6, wherein the first and second wavelengths are in the infrared portion of the spectrum.

10. The system of claim 6, wherein one of the first and second wavelengths is in the infrared portion of the spectrum, and the other of the first and second wavelengths is in the visible portion of the spectrum.

11. The system of claim 1, wherein the fatigue signatures comprise at least one of eye gaze, eye blinks, and eye closure rate.

12. The system of claim 11, wherein the fatigue signatures further comprise at least one of yawn detection and micronods, the at least one of yawn detection and micronods being extracted from the video-rate images.

13. The system of claim 1, wherein the stress signatures comprise at least one of heart rate (HR), heart rate variability (HRV), and respiration rate (RR).

14. The system of claim 1, wherein the light source comprises a plurality of light-emitting diodes, at least two of the light-emitting diodes producing light having different emission spectra.

15. The system of claim 1, wherein the light source and the collection optics are spaced apart from the subject by a distance between 0.5 meters and 1.5 meters.

16. A system for monitoring stress and fatigue of a subject, the system comprising:

a light source configured to direct illuminating light onto a face of the subject, the light source comprising at least one infrared light emitting diode, the illuminating light having a spectrum that includes a first wavelength, the illuminating light reflecting off the face of the subject to form reflected light;

collection optics that collect a portion of the reflected light and produce video-rate images of the face of the subject at the first wavelength, the collection optics comprising;

a spectral filter that transmits wavelengths in a wavelength band that includes the first wavelength and blocks wavelengths outside the transmitted wavelength band;

a lens configured to form an image of the face of the subject; and

a detector configured to detect the image of the face of the subject at the first wavelength; and

an image processor configured to;

locate an eye in the video-rate images;

extract fatigue signatures from the located eye, the fatigue signatures comprising at least one of eye behavior gaze, eye blinks, and eye closure rate;

determine a fatigue level of the subject, in part, from the fatigue signatures;

locate a facial region away from the eye in the video-rate images;

extract stress signatures from the located facial region, the stress signatures comprising at least one of heart rate (HR), heart rate variability (HRV), and respiration rate (RR); and

determine a stress level of the subject from the stress signatures.

17. The system of claim 16, wherein the collection optics and the light source are spaced apart from the subject by a distance between 0.5 meters and 1.5 meters.

18. A method for monitoring stress and fatigue of a subject, the method comprising:

receiving video-rate images of a face of the subject;

locating an eye in the video-rate images;

extracting fatigue signatures from the located eye;

determining a fatigue level of the subject, in part, from the fatigue signatures;

locating a facial region away from the eye in the video-rate images;

extracting stress signatures from the located facial region; and

determining a stress level of the subject from the stress signatures.

19. The method of claim 18, further comprising:

directing illuminating light onto a face of the subject, the illuminating light reflecting off the face of the subject to form reflected light;

collecting a portion of the reflected light; and

producing the video-rate images from the collected light.

20. The method of claim 18,

wherein the fatigue signatures comprise at least one of eye behavior gaze, eye blinks, and eye closure rate;

and

wherein the stress signatures comprise at least one of heart rate (HR), heart rate variability (HRV), and respiration rate (RR).