Method and device for the detection of microsleep events

• US 8,570,176 B2
• Filed: 12/19/2008
• Issued: 10/29/2013
• Est. Priority Date: 05/28/2008
• Status: Active Grant

• Current Assignee: 7352867 Canada Inc.
• Original Assignee: 7352867 Canada Inc.
• Inventors: Farbos, Bruno
• Primary Examiner(s): SZMAL, BRIAN SCOTT
• Application Number:
  US12/340,017
• Publication Number:
  US 20090299209A1
• Time in Patent Office: 1,775 Days
• Field of Search: 600/300, 600/301, 600/558, 340/575, 340/576, 340/143, 340/117, 340/206, 340/211, 340/573, 340/573.1, 348/77, 348/143, 250/336.1
• US Class Current: 340/575
• CPC Class Codes: A61B 2503/22 Motor vehicles operators, e.g. drivers, pilots, captains
• CPC Class Codes: A61B 3/10 Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
• CPC Class Codes: A61B 5/1103 Detecting eye twinkling
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/369 Electroencephalography [EEG] devices for psychotechnics A61B5/16
• CPC Class Codes: A61B 5/398 Electrooculography [EOG], e.g. detecting nystagmus; Electroretinography [ERG]
• CPC Class Codes: A61B 5/4809 Sleep detection, i.e. determining whether a subject is asleep or not
• CPC Class Codes: A61B 5/6821 Eye
• CPC Class Codes: B60K 28/06 responsive to incapacity of driver

Abstract

Disclosed herein is a method of detecting a microsleep event in a subject. The method includes determining a number of eye openness factors by measuring a number of distances between an upper eyelid and a lower eyelid of at least one eye over a time period. Graphical representations of the eye openness factors are then generated. Changes in the eye openness factors over the time period are correlated with a reference eye closure pattern indicative of the microsleep event. Also disclosed is a microsleep event detection device.

Claims

1. A method of alerting a vehicle operator to a microsleep event, the method comprising:

determining a plurality of eye openness factors by measuring a plurality of distances between an upper eyelid and a lower eyelid of at least one eye over a time period;

generating graphical representations of the eye openness factors; and

correlating changes in the eye openness factors over the time period with a reference eye closure pattern indicative of the microsleep event; and

triggering an alarm so as to alert the operator when the eye openness factors, compared to a reference eye closure pattern, are indicative of the microsleep event.

2. The method, according to claim 1, further comprising:

illuminating the face of the vehicle operator; and

recording a facial image.

3. The method, according to claim 2, in which a digital camera having an infra-red source is used to illuminate the face and to record the facial image.

4. The method, according to claim 3, in which images of the face are sampled at a frequency of between 10 Hz and 60 Hz.

5. The method, according to claim 1, further comprising:

verifying the presence of microsleep characteristic eye openness factor levels by measuring the eye openness factors as a function of time for a blink cycle of the eye.

6. The method, according to claim 5, in which the eye openness factors levels include at least one eye openness level.

7. The method, according to claim 5, in which the eye openness factors include one to five eye openness levels.

8. The method, according to claim 5, in which the eye openness factors include five eye openness levels.

9. The method, according to claim 5, in which the eye openness levels are associated with an open eye, the closure of the eyelids, partial or closed eye, and opening of the eyelids.

10. The method, according to claim 5, further comprising:

a sub-process for detecting microsleep characteristic eye openness factor levels at an image sampling frequency of 20 Hz.

11. The method, according to claim 10, in which the sub-process comprises:

verifying that a first level is detected by confirming the presence of a series six or more successive eye openness factors corresponding to an open eye.

12. The method, according to claim 11, further comprising:

verifying that a second level is detected by confirming the presence of a series four or more successive decreasing eye openness factors.

13. The method, according to claim 12, further comprising:

verifying that a third level is detected by confirming the presence of a series of a minimum of five and a maximum of one-hundred and twenty successive eye openness factors.

14. The method, according to claim 13, further comprising:

verifying that a fourth level is detected by confirming the presence of a series of at least four successive eye openness factors.

15. The method, according to claim 14, further comprising:

verifying that a fifth level is detected by confirming the presence of a series of a minimum of six successive eye openness factors corresponding to the open eye.

16. The method, according to claim 1, in which the eye openness factors include five successive eye openness levels, the sequential detection of the five levels being indicative of microsleep characteristics.

17. The method, according to claim 16, further comprising:

determining additional eye openness factors if less than five successive eye openness levels are detected.

18. The method, according to claim 5, further comprising:

computing eye opening and eye closure representative curves.

19. The method, according to claim 18, in which the eye closure representative curves are computed using a negative slope and a second order polynomial regression applied to the eye openness factors of first and second eye openness factor levels.

20. The method, according to claim 18, in which the eye opening factors are computed using a positive slope and a second order polynomial regression applied to the eye openness factors of fourth and fifth eye openness factor levels.

21. The method, according to claim 18, further comprising:

verifying the presence of microsleep eye opening and closing representative curves by computing the Pearson coefficient of the eye closure representative curves with regard to first and second eye openness factor levels and the eye opening representative curves with regard to fourth and fifth eye openness factor levels.

22. The method, according to claim 21, in which the vehicle operator is informed when the Pearson coefficients are greater than or equal to a predetermined threshold value.

23. The method, according to claim 1, further comprising:

alerting the vehicle operator to the presence of the microsleep event.

24. A microsleep event detection device, the device comprising:

a facial image sampler for sampling facial images over time of a subject, the sampler having an infra red source for illuminating one or more eyes of the subject and for measuring the distance between upper eyelids and lower eyelids of the subject;

a microprocessor connected to the sampler for receiving the sampled facial images, and for converting the sampled facial images to graphical representations of eye openness factors;

a non-transitory memory electronically connected to the microprocessor, the memory having stored therein a plurality of reference eye closure patterns; and

an alarm connected to the microprocessor for alerting the subject when the eye openness factors, compared to a reference eye closure pattern, are indicative of the microsleep event.

25. The device, according to claim 24, further comprising an alert connected to the microprocessor for alerting the subject to the microsleep event.

26. A computer-implemented method of detecting a microsleep event in a subject, the method comprising:

receiving a signal from an infra red source at one or more eyes of the subject;

determining a plurality of eye openness factors by measuring a plurality of distances between an upper eyelid and a lower eyelid of at least one eye over a time period;

generating graphical representations of the eye openness factors; and

triggering an alarm to alert the subject when the eye openness factors, compared to a reference eye closure pattern, are indicative of the microsleep event.