Method and apparatus for recognising microsleep in a driver of a vehicle

• US 10,748,404 B2
• Filed: 12/03/2015
• Issued: 08/18/2020
• Est. Priority Date: 01/19/2015
• Status: Active Grant

• Current Assignee: Robert Bosch GmbH
• Original Assignee: Robert Bosch GmbH
• Inventors: Stoltz, Achim, Wulf, Felix
• Primary Examiner(s): Phan, Hai
• Assistant Examiner(s): Afrifa-Kyei, Anthony D
• Application Number:
  US15/533,219
• Publication Number:
  US 20170345276A1
• Time in Patent Office: 1,720 Days
• Field of Search: 340575, 600544, 600588, 348 78
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

A method for recognizing microsleep on the part of a driver of a vehicle. The method includes at least a step of reading in an eye closure information item regarding an eye parameter of the driver, the eye closure information item representing a first eyelid position for a maximum eye opening level, and/or a second eyelid position for a minimum eye opening level, for the driver; a step of classifying a current eyelid position of an eyelid of the driver using the eye closure information item, in order to obtain an eye opening information item that represents an open state of the eyes or a closed state of the eyes; and a step of ascertaining a sleep recognition value that represents an indication of an occurrence of microsleep.

Claims

1. A method for recognizing microsleep on the part of a driver of a vehicle, the method comprising:

reading in an eye closure information item regarding an eye parameter of the driver, the eye closure information item representing at least one of a first eyelid position for a maximum eye opening level, and a second eyelid position for a minimum eye opening level, for the driver;

classifying a current eyelid position of an eyelid of the driver using the eye closure information item to obtain an eye opening information item that represents an open state of the eyes or a closed state of the eyes; and

ascertaining a risk potential value that represents a risk of an occurrence of the microsleep, the risk potential value being ascertained using at least one of the current eyelid position and the eye opening information item, the microsleep being recognized if the risk potential value during a predetermined time period exceeds a threshold value;

wherein before the reading in, the driver is identified in an identifying task, and the eye closure information item is furnished from a database using an information item regarding the identified driver,

wherein, in the ascertaining, the risk potential value is ascertained using lid closure dynamics and at least one of;

(i) lid closure acceleration, and (ii) lid opening dynamics, and

wherein, in the ascertaining, the risk potential value is ascertained using the lid closure dynamics and the at least one of;

(i) the lid closure acceleration, and (ii) the lid opening dynamics, and wherein the lid closure dynamics and the at least one of;

(i) the lid closure acceleration, and (ii) the lid opening dynamics is used to recognize characteristic features of the microsleep even as the eyes are closing.

3. The method as recited in claim 1, further comprising:

reading in an eye movement information item regarding an eye parameter of the driver, the eye movement information item representing an information item regarding an eye movement of the eyes of the driver; and

ascertaining a risk potential value that represents risk of an occurrence of the microsleep, the risk potential value being ascertained using at least one of an eye opening speed derived from the eye movement information item, and a gaze direction of the driver derived from the eye movement information item, the microsleep being recognized if the risk potential value during a predetermined time period exceeds a threshold value;

wherein, in the ascertaining, the risk potential value is ascertained using the lid closure dynamics and the lid closure acceleration.

4. The method as recited in claim 1, wherein, before the reading-in, in an evaluating task, the current eyelid position is evaluated over an initialization time span to determine the eye closure information item.

5. The method as recited in claim 1, further comprising:

performing a learning task, preceding the classifying, in which the eye closure information item is adapted using the current eyelid position.

6. The method as recited in claim 1, wherein, in the classifying, the eye opening information item is classified using a gaze direction signal to distinguish between open eyes and closed eyes.

7. The method as recited in claim 1, wherein, in the ascertaining, the risk potential value is ascertained using at least one of:

(i) a number of eye opening information items, representing closed eyes, for the driver during the time period, and (ii) a duration of an eye opening information item, representing closed eyes, during the time period.

11. The method as recited in claim 1, further comprising:

reading in an eye movement information item regarding an eye parameter of the driver, the eye movement information item representing an information item regarding an eye movement of the eyes of the driver;

wherein the risk potential value is further ascertained using an eye opening speed derived from the eye movement information item and a gaze direction of the driver derived from the eye movement information item, the microsleep being recognized if the risk potential value during a predetermined time period exceeds the threshold value, and

wherein drifting pupils are detected using the eye opening level and the gaze direction, where movement patterns of a gaze direction signal are used to detect the drifting pupils.

2. A method for recognizing microsleep on the part of a driver of a vehicle, the method comprising:

reading in an eye movement information item regarding an eye parameter of the driver, the eye movement information item representing an information item regarding an eye movement of the eyes of the driver; and

ascertaining a risk potential value that represents risk of an occurrence of the microsleep, the risk potential value being ascertained using an eye opening speed derived from the eye movement information item and a gaze direction of the driver derived from the eye movement information item, the microsleep being recognized if the risk potential value during a predetermined time period exceeds a threshold value;

wherein, in the ascertaining, the risk potential value is ascertained using lid closure dynamics and at least one of;

(i) lid closure acceleration, and (ii) lid opening dynamics, and

wherein, in the ascertaining, the risk potential value is ascertained using the lid closure dynamics and the at least one of;

(i) the lid closure acceleration, and (ii) the lid opening dynamics, and wherein the lid closure dynamics and the at least one of;

(i) the lid closure acceleration, and (ii) the lid opening dynamics is used to recognize characteristic features of the microsleep even as the eyes are closing.

12. The method as recited in claim 2, wherein drifting pupils are detected using the eye opening level and the gaze direction, where movement patterns of a gaze direction signal are used to detect the drifting pupils.

8. An apparatus for recognizing microsleep on the part of a driver of a vehicle, comprising:

a device configured to perform the following;

read in an eye closure information item regarding an eye parameter of the driver, the eye closure information item representing at least one of a first eyelid position for a maximum eye opening level, and a second eyelid position for a minimum eye opening level, for the driver;

classify a current eyelid position of an eyelid of the driver using the eye closure information item to obtain an eye opening information item that represents an open state of the eyes or a closed state of the eyes;

ascertain a risk potential value that represents a risk of an occurrence of the microsleep, the risk potential value being ascertained using at least one of the current eyelid position and the eye opening information item, the microsleep being recognized if the risk potential value during a predetermined time period exceeds a threshold value; and

before the reading in, identify the driver, the eye closure information item being furnished from a database using an information item regarding the identified driver;

wherein, in the ascertaining, the risk potential value is ascertained using lid closure dynamics and at least one of;

(i) lid closure acceleration, and (ii) lid opening dynamics, and

wherein, in the ascertaining, the risk potential value is ascertained using the lid closure dynamics and the at least one of;

(i) the lid closure acceleration, and (ii) the lid opening dynamics, and wherein the lid closure dynamics and the at least one of;

(i) the lid closure acceleration, and (ii) the lid opening dynamics is used to recognize characteristic features of the microsleep even as the eyes are closing.

13. The apparatus as recited in claim 8, further comprising:

reading in an eye movement information item regarding an eye parameter of the driver, the eye movement information item representing an information item regarding an eye movement of the eyes of the driver;

wherein the risk potential value is further ascertained using an eye opening speed derived from the eye movement information item and a gaze direction of the driver derived from the eye movement information item, the microsleep being recognized if the risk potential value during a predetermined time period exceeds the threshold value, and

wherein drifting pupils are detected using the eye opening level and the gaze direction, where movement patterns of a gaze direction signal are used to detect the drifting pupils.

9. A non-transitory machine-readable memory medium on which is stored a computer program, which is executable by a computer, comprising:

a program code arrangement having program code for recognizing microsleep on the part of a driver of a vehicle, by performing the following;

reading in an eye closure information item regarding an eye parameter of the driver, the eye closure information item representing at least one of a first eyelid position for a maximum eye opening level, and a second eyelid position for a minimum eye opening level, for the driver;

classifying a current eyelid position of an eyelid of the driver using the eye closure information item to obtain an eye opening information item that represents an open state of the eyes or a closed state of the eyes; and

ascertaining a risk potential value that represents a risk of an occurrence of the microsleep, the risk potential value being ascertained using at least one of the current eyelid position and the eye opening information item, the microsleep being recognized if the risk potential value during a predetermined time period exceeds a threshold value;

wherein before the reading in, the driver is identified in an identifying task, and the eye closure information item is furnished from a database using an information item regarding the identified driver,

wherein, in the ascertaining, the risk potential value is ascertained using lid closure dynamics and at least one of;

(i) lid closure acceleration, and (ii) lid opening dynamics, and

wherein, in the ascertaining, the risk potential value is ascertained using the lid closure dynamics and the at least one of;

(i) the lid closure acceleration, and (ii) the lid opening dynamics, and wherein the lid closure dynamics and the at least one of;

(i) the lid closure acceleration, and (ii) the lid opening dynamics is used to recognize characteristic features of the microsleep even as the eyes are closing.

10. The non-transitory machine-readable memory medium as recited in claim 9, further comprising:

reading in an eye movement information item regarding an eye parameter of the driver, the eye movement information item representing an information item regarding an eye movement of the eyes of the driver;

wherein the risk potential value is further ascertained using an eye opening speed derived from the eye movement information item and a gaze direction of the driver derived from the eye movement information item, the microsleep being recognized if the risk potential value during a predetermined time period exceeds the threshold value, and

wherein drifting pupils are detected using the eye opening level and the gaze direction, where movement patterns of a gaze direction signal are used to detect the drifting pupils.