Sleepiness level determination device for driver

• US 8,094,883 B2
• Filed: 10/09/2008
• Issued: 01/10/2012
• Est. Priority Date: 10/11/2007
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Nagai, Fumiya, Omi, Takuhiro
• Primary Examiner(s): Bhatnagar, Anand
• Application Number:
  US12/285,580
• Publication Number:
  US 20090097701A1
• Time in Patent Office: 1,188 Days
• Field of Search: 382/100, 382/103, 382/106, 382/107, 382/118, 382/181, 382/190, 382/195, 382/203, 382/206, 340/575, 340/576, 340/573.1, 340/573.7, 351/205, 351/206, 351/209, 351/210
• US Class Current: 382/103
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

A sleepiness level determination device includes: a detector processing a face image of an user and for detecting an eye image of the user based on the face image; a characteristic value calculating unit calculating a characteristic value regarding the eye based on the eye image; a sleepiness level determining unit determining a sleepiness level based on the characteristic value; and a reliability calculating unit calculating reliability of the sleepiness level based on the characteristic value.

Claims

1. A sleepiness level determination apparatus comprising:

a detecting device for performing a predetermined image processing on a face image, which is shot at predetermined shooting time intervals and shows a face of a driver, and for detecting an eye image based on the face image;

a characteristic value calculating device for calculating a plurality of different characteristic values of eye based on the eye image detected by the detecting device, the characteristic values of eye including a period of time from a start to an end of one blink, a time interval of closing an eye, a closing speed of an eyelid, and an opening speed of the eyelid;

a sleepiness level determining device for determining a sleepiness level among a plurality of sleepiness levels, based on a predetermined estimation function having the plurality of different characteristic values of eye as a plurality of explaining variables calculated by the characteristic value calculating device;

a representative characteristic value determining device for obtaining a distribution of each characteristic value of eye with respect to every sleepiness level when the sleepiness level determining device determines the sleepiness level and for determining a representative characteristic value with respect to each distribution; and

a reliability calculating device for calculating reliability of the sleepiness level determined by the sleepiness level determining device according to proximity between each characteristic value of eye at a time when the sleepiness level is determined and a corresponding representative characteristic value determined by the representative characteristic value determining device with respect to the sleepiness level determined by the sleepiness level determining device.

2. The sleepiness level determination apparatus according to claim 1, further comprising:

a contribution ratio calculating device for calculating a contribution ratio of each characteristic value of eye when the sleepiness level determining device determines the sleepiness level,

wherein the reliability calculating device calculates the reliability of the sleepiness level based on the contribution ratio of each characteristic value of eye when the sleepiness level is determined.

3. The sleepiness level determination apparatus according to claim 2,

wherein the sleepiness level determining device determines an individual sleepiness level with respect to each characteristic value of eye,

the apparatus further comprising:

an inconsistence characteristic value specifying device for specifying an inconsistence characteristic value among the plurality of different characteristic values of eye,

wherein the inconsistence characteristic value has the individual sleepiness level determined by the sleepiness level determining device, which is not consistent with other individual sleepiness levels,

wherein the sleepiness level determining device finally determines the sleepiness level based on the plurality of characteristic values other than the inconsistence characteristic value specified by the inconsistence characteristic value specifying device, and

wherein the reliability calculating device finally calculates the reliability of the sleepiness level based on the contribution ratio of each characteristic value other than the inconsistence characteristic value specified by the inconsistence characteristic value specifying device.

4. The sleepiness level determination apparatus according to claim 1, further comprising:

a low reliability characteristic value specifying device for specifying a characteristic value of eye among the plurality of different characteristic values of eye, the characteristic value of eye having the proximity between each characteristic value of eye at a time when the sleepiness level is determined and a corresponding representative characteristic value, and the proximity being larger than a predetermined threshold proximity, which provides a high reliability threshold,

wherein the sleepiness level determining device finally determines the sleepiness level based on the plurality of characteristic values other than the characteristic value specified by the low reliability characteristic value specifying device.

5. The sleepiness level determination apparatus according to claim 4,

wherein the characteristic value specified by the low reliability characteristic value specifying device is a low reliability characteristic value, and

wherein, when the characteristic value has the proximity equal to or smaller than the predetermined threshold proximity, individual reliability corresponding to each characteristic value is equal to or higher than a predetermined individual reliability.

6. The sleepiness level determination apparatus according to claim 5,

wherein the representative characteristic value of each distribution is one of average, median and mode of a corresponding distribution,

wherein the distribution of each characteristic value has a dispersion, which is defined as σ

, and

wherein the proximity of each characteristic value is defined in such a manner that, the proximity is one when the characteristic value is disposed within ±

1σ

, the proximity is one-half when the characteristic value is disposed within ±

2σ

, the proximity is one-third when the characteristic value is disposed within ±

3σ

, and the proximity is zero when the characteristic value is not disposed within ±

3σ

.

7. The sleepiness level determination apparatus according to claim 1, further comprising:

a detection ratio calculating device for calculating a detection ratio of the eye image by the detecting device,

wherein the reliability calculating device calculates the reliability of the sleepiness level based on the characteristic value in addition to the detection ratio calculated by the detection ratio calculating device.

8. The sleepiness level determination apparatus according to claim 7, further comprising:

a camera shooting the face image, wherein the camera shoots a plurality of frames during a blink; and

a detection error calculating device detecting error of the image processing of the detector in each frame,

wherein, when the detection error calculating device does not detect error in a frame, the frame is defined as a detected frame, and

wherein the detection ratio is a ratio between the number of detected frames and the number of a whole frames.

9. The sleepiness level determination apparatus according to claim 1,

wherein the plurality of characteristic values are independent from each other, and

wherein the representative characteristic value represents a corresponding distribution.

10. The sleepiness level determination apparatus according to claim 1,

wherein the sleepiness level determining device determines the sleepiness level based on the characteristic value by using a regression equation.

11. The sleepiness level determination apparatus according to claim 1, further comprising:

a contribution ratio calculating device calculating a contribution ratio of each characteristic value with respect to the sleepiness level; and

a detection ratio calculating device calculating a detection ratio of the eye image,

wherein the number of the plurality of characteristic values is defined as n, the reliability is defined as R, the contribution of each characteristic value is in proportion to Ki, the proximity of each characteristic value is defined as Pi, and the detection ratio is defined as E, and

wherein the reliability calculating device calculates the reliability based on the proximity, the contribution ratio and the detection ratio of each characteristic value according to an equation of;

R={[K1×

P1+ . . . +Kn×

Pn]}×

E.