Method for determining the operational state of a driver

• US 9,848,813 B2
• Filed: 08/14/2012
• Issued: 12/26/2017
• Est. Priority Date: 08/14/2012
• Status: Active Grant

• Current Assignee: Volvo Lastvagnar AB (AB Volvo)
• Original Assignee: Volvo Lastvagnar AB (AB Volvo)
• Inventors: Kronberg, Peter Tobias, Sandblom, Fredrik
• Primary Examiner(s): Lieu, Julie
• Application Number:
  US14/420,668
• Publication Number:
  US 20150216466A1
• Time in Patent Office: 1,960 Days
• Field of Search: None
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/11 Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb for measuring pulse A61B5/02 ;A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/18 for vehicle drivers or machine operators
• CPC Class Codes: A61B 5/4809 Sleep detection, i.e. determining whether a subject is asleep or not
• CPC Class Codes: A61B 5/6893 Cars
• CPC Class Codes: A61B 5/7246 using correlation, e.g. template matching or determination of similarity
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: A61B 5/7275 Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 2540/223 Posture, e.g. hand, foot, or seat position, turned or inclined
• CPC Class Codes: B60W 2540/225 Direction of gaze
• CPC Class Codes: B60W 2540/229 Attention level, e.g. attentive to driving, reading or sleeping
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2554/802 Longitudinal distance
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: G08B 29/186 : Fuzzy logic; neural networks
• CPC Class Codes: G08B 29/188 : Data fusion; cooperative systems, e.g. voting among different detectors
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/30 : for calculating health indices; for individual health risk assessment

Abstract

A method for determining an operational state of a driver of a vehicle uses an awareness detection arrangement. The awareness detection arrangement includes at least a first and a second source for generating data relating to the behavior of the driver. The method includes receiving, from the first and the second source, data relating to at least one of physiological data of the driver, the operation of the vehicle, and a model of the driver operating the vehicle, comparing the data from the first and the second source with a driver state model defining a plurality of predefined driver states for each of the first and the second source, respectively, determining based on the comparison, for each of the first and the second source, a state probability for each of the plurality of predefined driver states, and weighing the determined driver states for the first and the second source with each other for determining an overall operational state probability for the driver.

Claims

1. A method for determining an operational state of a driver of a vehicle using an awareness detection arrangement, the awareness detection arrangement comprising at least a first and a second source for generating data relating to the behavior of the driver, the method comprising:

receiving, from the first and the second source, data relating to at least one of physiological data of the driver, the operation of the vehicle, and a model of the driver operating the vehicle;

comparing the data from the first and the second source with a driver state model defining a plurality of predefined driver states for each of the first and the second source, respectively;

determining based on the comparison, for each of the first and the second source, a state probability for each of the plurality of predefined driver states including determining one of a probability mass function for each of the plurality of predefined driver states; and

weighing the determined driver states for the first, and the second source with each other for determining an overall operational state probability for the driver.

2. Method according to claim 1, wherein the physiological data of the driver comprises information relating to at least one of eye, face, head, arm and body motion of the operator.

3. Method according to claim 1, wherein the operation of the vehicle comprises information relating to at least one of time to line crossing, distance to a further vehicle arrange in front of the vehicle, steering and/or wheel operation pattern.

4. Method according to claim 1, wherein the model of the driver operating the vehicle comprises a first component relating to at least one of the time of the day and the operational time of the vehicle and a second component relating to the drowsiness level of the driver.

5. Method according to claim 4, wherein the second component relating to the drowsiness level of the driver is based on at least one of a model of sleep latency, time of day, time on task, a circadian rhythm, and a sleep/wake homeostatic process.

6. Method according to claim 1, wherein the plurality of predefined driver states comprises at least two states defined as drowsy and alert, respectively, preferably four states defined as very alert, alert, drowsy and fighting sleep, respectively.

7. Method according to claim 1, wherein the determined state of the driver is provided to a vehicle system configured to implement a vehicle control functionality, the vehicle system adjusting the vehicle control functionality based on the driver state.

8. Method according to claim 1, wherein the deter lined state of the driver is provided to a drowsy driver detection system for generating a warning to the driver state indicate that the driver is drowsy.

9. A control system for determining an operational state of a driver of a vehicle, the control system comprising a control unit, the control unit connected to an, awareness detection arrangement comprising at least a first and a second source for generating data relating to the behavior of the driver, wherein the control unit is configured to:

receive, from the first and the second source, data relating to at least one of physiological data of the driver, the operation of the vehicle, and a model of the driver operating the vehicle;

compare the data from the first and the second source with a driver state model defining a plurality of predefined driver states for each of the first and the second source, respectively;

determine based on the comparison, for each of the first and the second source, a state probability for each of the plurality of predefined driver states including determining one of a probability mass function for each of the plurality of predefined driver states; and

weigh the determined driver states for the first and the second source with each other for determining air overall operational state probability for the driver.

10. Control system according to claim 9, wherein the control unit is further configure to provide the determined state of the driver to a vehicle system configured to implement a vehicle control functionality, the vehicle system adjusting the vehicle control functionality based on the driver state.

11. A vehicle system, comprising

a control system for determining an operational state of a driver of a vehicle, and

an awareness detection arrangement,

the control system comprising a control unit, the control unit being connected to the awareness detection arrangement,

the awareness detection arrangement comprising at east a first and a second source for generating data relating to the behavior of the driver,

wherein the control unit is configured to:

receive, from the first and the second source, data relating to at least one of physiological data of the driver, the operation of the vehicle, and a model of the driver operating the vehicle;

compare the data from the first and the second source with a driver state model defining a plurality of predefined driver states for each of the first and the second source, respectively;

determine based on the comparison, for each of the first and the second source, a state probability for each of the plurality of predefined driver states including determining one of a probability mass function for each of the plurality of predefined driver states; and

weigh the determined driver states for the first and the second source with each other for determining an overall operational state probability for the driver.

12. Vehicle system according to claim 11, wherein at least one of the first and the second source is configured to generate physiological data of the driver comprising information relating to at least one of eye, face, head, arm and body motion of the operator.

13. Vehicle system according to claim 11, wherein at least one of the first and the second source is an image capturing device.

14. Vehicle system according to claim 11, wherein at least one of the first and the second source is configured to generate operational data of the vehicle comprising information relating to at least one of time to line crossing, distance to a further vehicle arrange in front of the vehicle, steering and/or wheel operation pattern.

15. A non-transitory computer readable medium embodying a computer program product for determining an operational state of a driver of a vehicle using an awareness detection arrangement, the awareness detection arrangement comprising at least a first and a second source for generating data relating to the behavior of the driver, the computer program product comprising code configured to, when executed by a processor:

receive, from the first and the second source, data relating to at least one of physiological data of the driver, the operation of the vehicle, and a model of the driver operating the vehicle;

compare the data from the first and the second source with a driver state model defining a plurality of predefined driver states for each of the first and the second source, respectively;

determine based on the comparison, for each of the first and the second source, a state probability for each of the plurality of predefined driver states including determining one of a probability mass function for each of the plurality of predefined driver states; and

weigh the determined driver states for the first and the second source with each other for determining an overall operational state probability for the driver.