Operator drowsiness detection in surface mines

• US 9,809,115 B2
• Filed: 06/26/2014
• Issued: 11/07/2017
• Est. Priority Date: 07/03/2013
• Status: Active Grant

• Current Assignee: SAFEmine AG (Hexagon AB)
• Original Assignee: SAFEmine AG (Hexagon AB)
• Inventors: Mder, Urban, Schlapbach, Andrea Boris, Spring, Peter
• Primary Examiner(s): Bousono, Orlando
• Application Number:
  US14/900,363
• Publication Number:
  US 20160137059A1
• Time in Patent Office: 1,230 Days
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60W 2040/0827 due to sleepiness
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

A method for deriving a fatigue score (100) indicative of a current drowsiness state of an operator (1) measures a plurality of signals (10, 20, 30, 40, 50, 60, 70, 80, 90) and derives a personalized fatigue score (100) for the operator using these signals (10, 20, 30, 40, 50, 60, 70, 80, 90) in a weighted and operator-personalized way.

Claims

1. A method for deriving a fatigue score which is indicative of a current drowsiness state of an operator, the method comprising steps of:

measuring, by means of a first sensor, at least one machine control signal which is indicative of a control input from said operator to a machine,

measuring, by means of a second sensor, at least one operator signal which is indicative of a physiological and/or a behavioral state of said operator,

determining an operator identification signal for uniquely identifying said operator and for taking personal characteristics of said operator into account by means of a personalized fatigue model for improving a precision of the derivation of said fatigue score,

loading from a local memory and/or receiving via a wirebound or wireless link from a remote database at least one generic signal which is indicative of said personalized fatigue model for said uniquely identified operator by means of said operator identification signal,

wherein the personalized fatigue model is indicative of how at least two of said machine control signal and operator signal contribute to the derivation of the fatigue score, and

deriving, using said machine control signal, said operator signal, and said operator identification signal, said fatigue score which is indicative of said current drowsiness state of said uniquely identified operator,

wherein said step of deriving said fatigue score comprises a personalized weighing of at least two of said machine control signal and said operator signal.

2. The method of claim 1 wherein said operator is an operator of a movable object, and wherein said method comprises a further step of:

measuring, by means of a third sensor, at least one navigation signal of said movable object, which navigation signal is indicative of at least one of the group of

a position,

a velocity,

an orientation, and

a yaw rate

of said movable object,

wherein said at least one navigation signal is used in said step of deriving said fatigue score.

3. The method of claim 1 comprising a further step of:

measuring, by means of a fourth sensor, at least one environment signal which is indicative of at least one of the group of

a light level,

a noise level,

a temperature, and

an air humidity level,

wherein said at least one environment signal is used in said step of deriving said fatigue score.

4. The method of claim 1 comprising a further step of:

determining a time of day,

wherein said determined time of day is used in said step of deriving said fatigue score.

5. The method of claim 1 comprising a further step of:

receiving, by means of a receiver, at least one collision avoidance signal from a collision avoidance system, which collision avoidance signal is indicative of at least one of the group of

a position of a fixed or of a movable object,

a velocity of a movable object,

a yaw rate of a movable object,

a dimension of a fixed or a movable object,

a type of a fixed or a movable object,

a distance between a first movable object and a second fixed or movable object,

an issuing of a collision warning by means of said collision avoidance system between a first movable object and a second fixed or movable object, and

an occurrence of a collision between a first movable object and a second fixed or movable object,

wherein said collision avoidance signal is used in said step of deriving said fatigue score.

6. The method of claim 1 wherein said step of determining said operator identification signal comprises:

a receiving of a wireless transmission from a badge which is personalized for said operator, and/or

a reading of a badge which is personalized for said operator and/or

an entering of a unique identification phrase for said operator and/or

a recognition of a physiological feature which is unique for said operator.

7. The method of claim 1 comprising a further step of:

loading or receiving at least one additional generic signal which is indicative of at least one of the group of

a static site map,

a shift schedule of said operator,

an age of said operator,

a sex of said operator,

a training level of said operator, in particular on a particular machine type,

a productivity level of said operator,

a productivity goal of said operator,

a productivity fulfillment level of said operator,

a personal trust level in said operator,

a circadian rhythm of said operator,

an excitement level signal, and

an operator history,

wherein said generic signal is used in said step of deriving said fatigue score.

8. The method of claim 1 wherein said operator is an operator of a movable object and wherein said at least one machine control signal comprises at least one of the group of:

a steering input angle,

a brake pedal position,

a gas pedal position,

a retarder activation state,

a turn indicator state, and

a horn activation state.

9. The method of claim 1 wherein said at least one operator signal is indicative of a physiological state of said operator and comprises at least one of the group of:

a pupil diameter,

a blood pressure,

a pulse rate,

a body temperature,

an electroencephalogram,

a skin conductance level, and

a blood glucose level

of said operator.

10. The method of claim 1 wherein said at least one operator signal is indicative of a behavioral state of said operator and comprises at least one of the group of:

an eyelid closing state,

a mouth closing state,

a viewing direction,

a head orientation,

a focal point position,

a body posture, and

a distraction level

of said operator.

11. The method of the claim 9, comprising at least a first operator signal and a second operator signal, wherein the first operator signal is indicative of a physiological state of said operator,

wherein the second operator signal is indicative of a behavioral state of said operator, and

wherein said first and said second operator signals are used in said step of deriving said fatigue score.

12. The method of claim 1 comprising a further step of:

deriving, using at least one acquired signal, at least one derived signal,

wherein said derived signal is used in said step of deriving said fatigue score.

13. The method of claim 12 wherein said at least one derived signal is selected from the group consisting of:

a percentage of eye closure (PERCLOS) signal,

a steering entropy signal,

a machine control correction rate signal,

a collision avoidance incident rate signal,

an operator reaction time signal, and

a behavioral peculiarity signal.

14. The method of claim 1 comprising a further step of:

at a first time storing an acquired and/or derived signal in a memory which signal at a later second time becomes a previously stored signal,

wherein said previously stored signal is used in said step of deriving said fatigue score at said second time.

15. The method of claim 1 further comprising a step of:

deriving at least one confidence level for an acquired and/or derived signal,

wherein said confidence level is used in said step of deriving said fatigue score.

16. The method of claim 1 wherein said step of deriving said fatigue score comprises a weighing of all acquired and/or derived signals.

17. The method of claim 1 comprising a further step of:

removing at least one outlier from at least one acquired and/or derived signal.

18. The method of claim 1 comprising a further step of:

issuing an acoustic and/or a tactile and/or a visual alarm signal when said fatigue score increases above a threshold.

19. The method of claim 1 comprising a further step of:

transmitting, by means of a transmitter, a warning signal, when said fatigue score increases above a threshold.

20. The method of claim 1 wherein said fatigue score is selected from a group of at least 10 distinct values, or from a group of at least 50 distinct values, or from a group of at least 100 distinct values.

21. The method of claim 1 wherein an acquired signal comprises at least one of the group of:

the machine control signal,

the operator signal,

the operator identification signal,

a navigation signal,

an environment signal,

a time of day,

a collision avoidance signal, and

a generic signal.

22. The method of claim 14, wherein said previously stored signal is used in addition to an acquired and/or derived signal at said second time in said step of deriving said fatigue score at said second time.

23. A drowsiness detection device for deriving a fatigue score indicative of a current drowsiness state of an operator comprising:

a first sensor for measuring at least one machine control signal which is indicative of a control input to a machine from said operator,

a second sensor for measuring at least one operator signal which is indicative of a physiological and/or a behavioral state of said operator, and

a control unit configured to;

determine an operator identification signal for uniquely identifying said operator and for taking personal characteristics of said operator into account by means of a personalized fatigue model for improving a precision of the derivation of said fatigue score,

load from a local memory and/or receiving via a wirebound or wireless link from a remote database at least one generic signal which is indicative of said personalized fatigue model for said uniquely identified operator by means of said operator identification signal,

wherein the personalized fatigue model is indicative of how at least two of said machine control signal and operator signal contribute to the derivation of the fatigue score, and

derive, using said machine control signal, said operator signal, and said operator identification signal, said fatigue score which is indicative of said current drowsiness state of said uniquely identified operator,

wherein said step of deriving said fatigue score comprises a personalized weighing of at least two of said machine control signal and said operator signal.

24. The drowsiness detection device of claim 23 further comprising an override switch for temporarily disabling the device.

25. A non-transitory computer program element comprising computer program code for performing, when executed on a control unit, the steps of:

measuring, by means of a first sensor, at least one machine control signal which is indicative of a control input from said operator to a machine,

measuring, by means of a second sensor, at least one operator signal which is indicative of a physiological and/or a behavioral state of said operator,

determining an operator identification signal for uniquely identifying said operator and for taking personal characteristics of said operator into account by means of a personalized fatigue model for improving a precision of the derivation of said fatigue score,

loading from a local memory and/or receiving via a wirebound or wireless link from a remote database at least one generic signal which is indicative of said personalized fatigue model for said uniquely identified operator by means of said operator identification signal,

wherein the personalized fatigue model is indicative of how at least two of said machine control signal and operator signal contribute to the derivation of the fatigue score, and

deriving, using said machine control signal, said operator signal, and said operator identification signal, said fatigue score which is indicative of said current drowsiness state of said uniquely identified operator,

wherein said step of deriving said fatigue score comprises a personalized weighing of at least two of said machine control signal and said operator signal.