Method, system, and computer program product for monitoring a driver of a vehicle

• US 9,744,971 B2
• Filed: 04/28/2016
• Issued: 08/29/2017
• Est. Priority Date: 04/29/2015
• Status: Active Grant

• Current Assignee: Bayerische Motoren Werke AG
• Original Assignee: Bayerische Motoren Werke AG
• Inventors: Bigdelou, Ali
• Primary Examiner(s): NGUYEN, LEON VIET Q
• Application Number:
  US15/141,177
• Publication Number:
  US 20160318520A1
• Time in Patent Office: 488 Days
• Field of Search: None
• CPC Class Codes: B60W 2540/043 Identity of occupants
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 2540/221 Physiology, e.g. weight, heartbeat, health or special needs
• CPC Class Codes: B60W 2554/4041 Position
• CPC Class Codes: B60W 2554/804 Relative longitudinal speed
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: B60W 50/08 Interaction between the driver and the control system
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0053 from vehicle to occupant
• CPC Class Codes: G08B 21/02 Alarms for ensuring the safety of persons

Abstract

A method is provided for monitoring a driver of a vehicle, the method receives a state of the driver; receives one or more objects of a real environment model of the vehicle; generates one or more objects of an envisioned environment model of the driver based on the one or more objects of the real environment model of the vehicle and the state of the driver, wherein, if the state of the driver indicates that at least one sense of the driver sensed a particular object of the real environment model, the particular object of the real environment model is added to the envisioned environment model; and updates the one or more objects of the envisioned environment model based on the state of the driver, wherein the one or more objects of the envisioned environment model are updated by a physics engine if the state of the driver indicates that the driver is prevented from sensing the one or more objects of the real environment model.

Claims

1. A method for monitoring a driver of a vehicle, the method comprising the acts of:

receiving a state of the driver;

receiving one or more objects of a real environment model of the vehicle;

generating one or more objects of an envisioned environment model of the driver based on the one or more objects of the real environment model of the vehicle and the state of the driver, wherein, if the state of the driver indicates that at least one sense of the driver sensed a particular object of the real environment model, the particular object of the real environment model is added to the envisioned environment model;

updating the one or more objects of the envisioned environment model based on the state of the driver, wherein the one or more objects of the envisioned environment model are updated by a physics engine if the state of the driver indicates that the driver is prevented from sensing the one or more objects of the real environment model;

computing a deviation factor between the particular object of the real environment model and the corresponding object of the envisioned environment model;

calculating a situation awareness factor based on the deviation factor for the particular object of the real environment model and the corresponding object of the envisioned environment model; and

calculating a take-over ability time based on a function of the situation awareness factor.

2. The method according to claim 1, wherein the generating of the one or more objects of the envisioned environment model further comprises the acts of:

finding a corresponding object of the particular object in the envisioned environment model;

determining whether the driver of the vehicle has received information about the particular object via a subset of senses of the driver, the subset of senses being determined based on the state of the driver;

updating the corresponding object of the envisioned environment model with data of the particular object based on the subset of senses, if the corresponding object exists in the envisioned environment model and the driver has received information about the particular object via the subset of senses.

3. The method according to claim 1, wherein the deviation factor comprises a deviation of at least one of position, speed, and acceleration between the particular object of the real environment model and the corresponding object of the envisioned environment model.

4. The method according to claim 3, wherein the deviation of the position comprises a weighted deviation of the position, wherein the deviation of the speed comprises a weighted deviation of the speed, and/or wherein the deviation of the acceleration comprises a weighted deviation of the acceleration.

5. The method according to claim 1, the method further comprising the act of:

providing the calculated situation awareness factor and/or the calculated take-over ability time to at least one of a warning system and a driving assistant system of the vehicle so that the warning system and/or the driving assistant system may adapt to a new situation as indicated by the calculated situation awareness factor and/or the calculated take-over ability time.

6. A system for monitoring a driver of a vehicle, the system comprising:

a processor,

a memory,

instructions stored within the memory, wherein the instructions, when executed on the processor, cause the system to;

receive a state of the driver;

receive one or more objects of a real environment model of the vehicle;

generate one or more objects of an envisioned environment model of the driver based on the one or more objects of the real environment model of the vehicle and the state of the driver, wherein, if the state of the driver indicates that at least one sense of the driver sensed a particular object of the real environment model, the particular object of the real environment model is added to the envisioned environment model;

update the one or more objects of the envisioned environment model based on the state of the driver, wherein the one or more objects of the envisioned environment model are updated by a physics engine if the state of the driver indicates that the driver is prevented from sensing the one or more objects of the real environment model;

compute a deviation factor between the particular object of the real environment model and the corresponding object of the envisioned environment model;

calculate a situation awareness factor based on the deviation factor for the particular object of the real environment model and the corresponding object of the envisioned environment model; and

calculate a take-over ability time based on a function of the situation awareness factor.

7. The system according to claim 6, wherein the instructions that generate the one or more objects of the envisioned environment model further comprise instructions which, when executed, cause the system to:

find a corresponding object of the particular object in the envisioned environment model;

determine whether the driver of the vehicle has received information about the particular object via a subset of senses of the driver, the subset of senses being determined based on the state of the driver;

update the corresponding object of the envisioned environment model with data of the particular object based on the subset of senses, if the corresponding object exists in the envisioned environment model and the driver has received information about the particular object via the subset of senses.

8. The system according to claim 6, wherein the deviation factor comprises a deviation of at least one of position, speed, and acceleration between the particular object of the real environment model and the corresponding object of the envisioned environment model.

9. The system according to claim 8, wherein the deviation of the position comprises a weighted deviation of the position, wherein the deviation of the speed comprises a weighted deviation of the speed, and/or wherein the deviation of the acceleration comprises a weighted deviation of the acceleration.

10. A computer product for monitoring a driver of a vehicle, the computer product comprising a non-transitory computer readable medium having stored thereon program instructions that:

receive a state of the driver;

receive one or more objects of a real environment model of the vehicle;

generate one or more objects of an envisioned environment model of the driver based on the one or more obj ects of the real environment model of the vehicle and the state of the driver, wherein, if the state of the driver indicates that at least one sense of the driver sensed a particular object of the real environment model, the particular object of the real environment model is added to the envisioned environment model; and

update the one or more objects of the envisioned environment model based on the state of the driver, wherein the one or more objects of the envisioned environment model are updated by a physics engine if the state of the driver indicates that the driver is prevented from sensing the one or more objects of the real environment model;

compute a deviation factor between the particular object of the real environment model and the corresponding object of the envisioned environment model;

calculate a situation awareness factor based on the deviation factor for the particular object of the real environment model and the corresponding object of the envisioned environment model; and

calculate a take-over ability time based on a function of the situation awareness factor.

11. The computer product according to claim 10, further comprising, for generating the one or more objects of an envisioned environment model, program instructions that:

find a corresponding object of the particular object in the envisioned environment model;

determine whether the driver of the vehicle has received information about the particular object via a subset of senses of the driver, the subset of senses being determined based on the state of the driver;

update the corresponding object of the envisioned environment model with data of the particular object based on the subset of senses, if the corresponding object exists in the envisioned environment model and the driver has received information about the particular object via the subset of senses.