Drowsiness detection method and system for determining the degree of eye opening and closure

• US 10,123,686 B2
• Filed: 10/27/2015
• Issued: 11/13/2018
• Est. Priority Date: 04/22/2015
• Status: Active Grant

• Current Assignee: Wistron Corporation
• Original Assignee: Wistron Corporation
• Inventors: Li, Chong-Wei, Liao, Chih-Pin, Kuo, Che-You
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): Beg, Samah A
• Application Number:
  US14/923,451
• Publication Number:
  US 20160310060A1
• Time in Patent Office: 1,113 Days
• Field of Search: None
• CPC Class Codes: A61B 2503/22 Motor vehicles operators, e.g. drivers, pilots, captains
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 3/0008 provided with illuminating means A61B3/117 takes precedence; goniolenses used for laser treatment A61F9/009; illuminating means for optical instruments G02B27/00
• CPC Class Codes: A61B 3/0025 characterised by electronic signal processing, e.g. eye models
• CPC Class Codes: A61B 3/10 Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 3/14 Arrangements specially adapted for eye photography apparatus or arrangements for taking photographs per se G03B
• CPC Class Codes: A61B 5/1103 Detecting eye twinkling
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/4809 Sleep detection, i.e. determining whether a subject is asleep or not
• CPC Class Codes: A61B 5/6893 Cars
• CPC Class Codes: A61B 5/7203 for noise prevention, reduction or removal
• CPC Class Codes: A61B 5/7405 using sound
• CPC Class Codes: A61B 5/7485 Automatic selection of region of interest
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

An eye detection method for detecting a degree of eye opening and closure of a testee includes detecting a location of an eye pupil, a location of an inner eye corner and a location of an outer eye corner of an eye of the testee; calculating an eye width according to the location of the inner eye corner and the location of the outer eye corner; multiplying the eye width by a specific ratio to obtain an eye height; generating a region of interest with a center on the location of the eye pupil, a length equal to the eye width, and a width equal to the eye height; and determining a ratio of the eye occupying a detection area in the region of interest, and determining the degree of eye opening and closure of the testee accordingly.

Claims

1. An eye detection method for detecting a degree of eye opening and closure of a testee, comprising:

detecting a location of an eye pupil, a location of an inner eye corner and a location of an outer eye corner of an eye of the testee;

calculating an eye width according to the location of the inner eye corner and the location of the outer eye corner;

multiplying the eye width by a specific ratio smaller than 1 to obtain an eye height;

generating a rectangular region of interest with a center on the location of the eye pupil, a length equal to the eye width, and a width equal to the eye height; and

determining a ratio of the eye occupying a detection area in the region of interest, and determining the degree of eye opening and closure of the testee accordingly;

wherein the step of determining the ratio of the eye occupying the detection area in the region of interest and determining the degree of eye opening and closure of the testee accordingly comprises;

performing an image binarization on an eye image within the region of interest, to generate a binarized image having an eye region and a non-eye region;

performing a filling process on the binarized image to eliminate a noise in the eye region determined to be the non-eye region;

detecting an area of the eye region within the detection area after the filling process is accomplished; and

determining that the eye of the testee is closed when a number of times the ratio of the eye region occupying the detection area smaller than a threshold value is detected in a specific period of time reaches a specific number.

2. The eye detection method of claim 1, further comprising:

sending a warning signal when the testee is determined to be sleepy according to the degree of eye opening and closure of the testee.

3. The eye detection method of claim 1, further comprising:

performing supplementary illumination on the eye, to prevent an image of the eye from being influenced by an external light source.

4. The eye detection method of claim 1, wherein the step of calculating the eye width according to the location of the inner eye corner and the location of the outer eye corner comprises:

setting the eye width to be equal to a distance between the location of the inner eye corner and the location of the outer eye corner.

5. The eye detection method of claim 1, wherein the filling process comprises:

determining whether a middle point of a row of pixels in the region of interest is in the eye region or the non-eye region;

scanning from a first middle point of a first row of pixels toward a direction of the eye width when the first middle point is in the non-eye region, and after scanning to the eye region, filling the non-eye region between the first middle point and the scanned eye region in the first row of pixels to be the eye region; and

scanning from a second middle point of a second row of pixels toward the direction of the eye width when the second middle point is in the eye region, and after scanning to the non-eye region and then scanning to the eye region, filling the scanned non-eye region in the second row of pixels to be the eye region.

6. The eye detection method of claim 5, further comprising:

performing the filling process on each row of pixels in the region of interest in order.

7. The eye detection method of claim 1, wherein the filling process comprises:

determining whether a middle point of a column of pixels in the region of interest is in the eye region or the non-eye region;

scanning from a first middle point of a first column of pixels toward a direction of the eye height when the first middle point is in the non-eye region, and after scanning to the eye region, filling the non-eye region between the first middle point and the scanned eye region in the first column of pixels to be the eye region; and

scanning from a second middle point of a second column of pixels toward the direction of the eye height when the second middle point is in the eye region, and after scanning to the non-eye region and then scanning to the eye region, filling the scanned non-eye region in the second column of pixels to be the eye region.

8. The eye detection method of claim 1, further comprising:

adjusting the threshold value according to a magnitude of the eye region detected during a period of time.

9. An eye detection system for detecting a degree of eye opening and closure of a testee, comprising:

a photography device, for detecting a location of an eye pupil, a location of an inner eye corner and a location of an outer eye corner of an eye of the testee; and

a processor, coupled to the photography device, for performing the following steps;

calculating an eye width according to the location of the inner eye corner and the location of the outer eye corner;

multiplying the eye width by a specific ratio smaller than 1 to obtain an eye height;

generating a rectangular region of interest with a center on the location of the eye pupil, a length equal to the eye width, and a width equal to the eye height; and

determining a ratio of the eye occupying a detection area in the region of interest, and determining the degree of eye opening and closure of the testee accordingly;

wherein the processor further performs the following steps to determine the ratio of the eye occupying the detection area in the region of interest and determine the degree of eye opening and closure of the testee accordingly;

performing an image binarization on an eye image within the region of interest, to generate a binarized image having an eye region and a non-eye region;

performing a filling process on the binarized image to eliminate a noise in the eye region determined to be the non-eye region;

detecting an area of the eye region within the detection area after the filling process is accomplished; and

determining that the eye of the testee is closed when a number of times the ratio of the eye region occupying the detection area smaller than a threshold value is detected in a specific period reaches a specific number.

10. The eye detection system of claim 9, further comprising:

a warning device, coupled to the processor, for sending a warning signal when the testee is determined to be sleepy according to the degree of eye opening and closure of the testee.

11. The eye detection system of claim 9, further comprising:

an infrared ray emitter, coupled to the processor, for performing supplementary illumination on the eye, to prevent an image of the eye from being influenced by an external light source.

12. The eye detection system of claim 9, wherein the processor sets the eye width to be equal to a distance between the location of the inner eye corner and the location of the outer eye corner.

13. The eye detection system of claim 9, wherein the filling process comprises:

determining whether a middle point of a row of pixels in the region of interest is in the eye region or the non-eye region;

scanning from a first middle point of a first row of pixels toward a direction of the eye width when the first middle point is in the non-eye region, and after scanning to the eye region, filling the non-eye region between the first middle point and the scanned eye region in the first row of pixels to be the eye region; and

scanning from a second middle point of a second row of pixels toward the direction of the eye width when the second middle point is in the eye region, and after scanning to the non-eye region and then scanning to the eye region, filling the scanned non-eye region in the second row of pixels to be the eye region.

14. The eye detection system of claim 13, wherein the processor performs the filling process on each row of pixels in the region of interest in order.

15. The eye detection system of claim 9, wherein the filling process comprising:

determining whether a middle point of a column of pixels in the region of interest is in the eye region or the non-eye region;

scanning from a first middle point of a first column of pixels toward a direction of the eye height when the first middle point is in the non-eye region, and after scanning to the eye region, filling the non-eye region between the first middle point and the scanned eye region in the first column of pixels to be the eye region; and

scanning from a second middle point of a second column of pixels toward the direction of the eye height when the second middle point is in the eye region, and after scanning to the non-eye region and then scanning to the eye region, filling the scanned non-eye region in the second column of pixels to be the eye region.

16. The eye detection system of claim 9, wherein the processor further adjusts the threshold value according to a magnitude of the eye region detected during a period of time.

17. An eye detection method for detecting a degree of eye opening and closure of a testee, comprising:

performing an image binarization on an eye image within a region of interest, to generate a binarized image having an eye region and a non-eye region;

performing a filling process on the binarized image to eliminate a noise in the eye region determined to be the non-eye region;

detecting an area of the eye region within a detection area in the region of interest after the filling process is accomplished; and

determining that the eye of the testee is closed when a number of times the ratio of the eye region occupying the detection area smaller than a threshold value is detected in a specific period of time reaches a specific number;

wherein the filling process comprises;

determining whether a middle point of a row of pixels in the region of interest is in the eye region or the non-eye region;

scanning from a first middle point of a first row of pixels toward a direction of the eye width when the first middle point is in the non-eye region, and after scanning to the eye region, filling the non-eye region between the first middle point and the scanned eye region in the first row of pixels to be the eye region; and

scanning from a second middle point of a second row of pixels toward the direction of the eye width when the second middle point is in the eye region, and after scanning to the non-eye region and then scanning to the eye region, filling the scanned non-eye region in the second row of pixels to be the eye region.