Detection of the open/closed state of eyes based on analysis of relation between eye and eyebrow images in input face images

• US 5,878,156 A
• Filed: 02/12/1996
• Issued: 03/02/1999
• Est. Priority Date: 07/28/1995
• Status: Expired due to Term

• Current Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Inventors: Okumura, Tomoko
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): Werner, Brian P.
• Application Number:
  US08/597,715
• Time in Patent Office: 1,114 Days
• Field of Search: 382/117, 382/118, 382/104, 382/173, 382/291, 348/78, 351/210, 364/413.02, 364/424.055, 128/745, 128/782, 340/575, 340/576
• US Class Current: 382/118
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

A face image processing apparatus for use in a sleep-inhibition security system for car drivers is disclosed. The apparatus includes an input section for acquiring an input face image of a target person, an eye extraction region setter for determining an eye extraction region within the input face image, and an extractor for extracting from the eye extraction region first and second image components corresponding to an eyebrow image and an eye image respectively. A first detector is adapted to detect relative positions of the first and second image components extracted. A second detector is provided for detecting the open or closed state of at least one eye image by analyzing variations in the relative positions detected.

Claims

1. A face image processing system operable to detect when a target person is in the process of falling asleep, said system comprising:

input means for acquiring an input image representing at least a portion of a face of said target person;

eye extraction region setter means for determining an eye extraction region within the input face image;

extractor means for extracting first and second image portions from the eye extraction region, said first and second image portions corresponding to an eyebrow image and an eye image respectively;

first detector means for detecting relative positions of said first and second image portions extracted; and

second detector means for detecting when said target person is in the process of falling asleep by determining an open and closed state of the eye image through analysis of variations in the relative positions detected.

2. The system according to claim 1, wherein said first detector means detects variations in the relative positions of the eyebrow image and the eye image based on variations in distance therebetween.

3. The system according to claim 1, wherein said first detector means comprises:

reference position setter means for setting a reference position of the eyebrow image along a length of the face;

eye characteristic position detector means for detecting a midway point of the eye image along the length of the face and a position of at least one representation point of the eye image along the face length; and

distance measurement means for measuring a distance between the reference position and the representation point.

4. The system according to claim 1, wherein said first detector means comprises:

centroid detector means for detecting respective centers of the eyebrow image and of the eye image along a width of the face; and

distance measurement means for measuring a distance between the centers of the eyebrow image and the eye image thus detected.

5. The apparatus according to claim 1, wherein said eye extraction region setter means includes:

digitizer means for digitizing the input face image to provide a corresponding binary image, and for determining an eye extraction region with said binary image.

6. A face image processing system operable to detect when a monitored person is in the process of falling asleep, said system comprising:

face imaging input means for acquiring an input face image of at least a portion of the face of said monitored person;

eye extraction region setter means for setting an eye extraction region within an input face image;

face image characteristics extractor means for extracting from said eye extraction region first and second image portions corresponding to an eyebrow image and an eye image;

area detector means for detecting areas of the eyebrow image and of the eye image extracted;

area ratio calculator means for using said areas to calculate an area ratio between the eyebrow image and the eye image; and

detector means for detecting when said person is in the process of falling asleep by determining a degree of an open or closed state of an eye based on variations in said area ratio calculated by said area ratio calculator means.

7. The apparatus according to claim 6, further comprising:

means for converting each of the eyebrow image and the eye image into corresponding block portions each having a frame of a fixed area and for providing eyebrow image blocks and eye image blocks; and

said area detector means detecting the area of each of the eyebrow image blocks and the eye image blocks.

8. The apparatus according to claim 6, wherein said eye extraction region setter means includes:

digitizer means for digitizing the input face image to provide a corresponding binary image, and for determining an eye extraction region within said binary image.

9. A face image processing apparatus comprising:

face image input means for acquiring an input face image of a face being monitored;

eye extraction region setter means for setting an eye extraction region within the input face image;

face image characteristic extractor means for extracting an eyebrow image and an eye image from the eye extraction region;

measurement means for measuring a width of a certain portions of each of the eyebrow and eye images along a length of the face;

calculator means for calculating a width ratio between the width of the eyebrow image and that of the eye image; and

detector means for detecting an open or closed state of an eye based on variations in the width ratio calculated by said calculator means.

10. The apparatus according to claim 9, further comprising:

means for converting each of the eyebrow image and the eye image into corresponding block portions each having a frame of a fixed area and for providing eyebrow image blocks and eye image blocks; and

width measurement means for measuring a width of each of the eyebrow image blocks and the eye image blocks along a length of the face.

11. The apparatus according to claim 9, wherein said eye extraction region setter means includes:

digitizer means for digitizing the input face image to provide a corresponding binary image, and for determining an eye extraction region within said binary image.

12. A system for detecting fatigue in a monitored person, said system comprising;

input means for inputting an image representing at least a portion of said monitored person'"'"'s face, said portion including at least one eye and at least one eyebrow;

a monitor for monitoring the relative positions of said eye and said eyebrow with respect to each other; and

a controller for determining when said monitored person is fatigued based on the relative positions of the eye and eyebrow with respect to each other.

13. The system for detecting fatigue according to claim 12, wherein said controller determines when said monitored person is fatigued based on the distance between said eye and eyebrow.