Apparatus and method for controlling a user situation awareness modification of a user of a vehicle, and a user situation awareness modification processing system

• US 10,363,945 B2
• Filed: 10/19/2017
• Issued: 07/30/2019
• Est. Priority Date: 04/20/2015
• Status: Active Grant

• Current Assignee: Bayerische Motoren Werke AG
• Original Assignee: Bayerische Motoren Werke AG
• Inventors: Huang, Guangbin, Sourina, Olga, Yang, Yan, Koenig, Alexander, Schumann, Josef, Decke, Ralf, Denk, Cornelia, Kerschbaum, Philipp
• Primary Examiner(s): Nguyen, Leon Viet Q
• Application Number:
  US15/788,093
• Publication Number:
  US 20180099679A1
• Time in Patent Office: 649 Days
• Field of Search: None
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 2540/221 Physiology, e.g. weight, heartbeat, health or special needs
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 50/16 Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal

Abstract

An apparatus configured to control a user situation awareness modification of a user of a vehicle, which includes input module configured to repeatedly receive a first event detection signal comprising information related to a detected event outside a vehicle and to repeatedly receive a first user detection signal comprising information related to a reaction or a missing reaction of a user to the detected event outside the vehicle. The apparatus further includes a processing module configured to determine a user situation awareness level based on the information related to the detected event outside the vehicle and the information related to the reaction of the user to the detected event, and to generate a control signal configured to control a user situation awareness modification module based on the determined user situation awareness level.

Claims

1. An apparatus configured to control a user situation awareness modification of a user of a vehicle, the apparatus comprising:

an input module configured to repeatedly receive a first event detection signal comprising information related to a detected event outside a vehicle and to repeatedly receive a first user detection signal comprising information related to a reaction or a missing reaction of a user to the detected event outside the vehicle; and

a processing module configured to determine a user situation awareness level based on the information related to the detected event outside the vehicle and the information related to the reaction of the user to the detected event, and to generate a control signal configured to control a user situation awareness modification module based on the determined user situation awareness level,

wherein the processing module is configured to determine a desired time-till-takeover based on the detected event outside the vehicle,

wherein the processing module is configured to generate the control signal to trigger the user situation awareness modification module if the determined user situation awareness level indicates that a predicted time-till-takeover is larger than the desired time-till-takeover.

2. The apparatus according to claim 1, wherein the processing module is configured to determine the user situation awareness level based on a reaction time between the detected event and a reaction or missing reaction of the user to the detected event.

3. The apparatus according to claim 1, wherein the input module is configured to repeatedly receive the first event detection signal at regular or predefined detection time intervals from a first detection module.

4. The apparatus according to claim 1, wherein the processing module is configured to repeatedly determine the situation awareness level of the user at regular or predetermined determination time intervals based on the first user detection signal received from a user detection module at regular or predefined time intervals.

5. The apparatus according to claim 1, wherein the processing module is configured to generate the control signal to repeatedly or continuously trigger a user situation awareness modification by the user situation awareness modification module until a desired user situation awareness level is reached.

6. The apparatus according to claim 1, wherein the input module is configured to receive the first user detection signal comprising information related to a reaction or a missing reaction of a user to the detected event outside the vehicle from a first user detection module and a second user detection signal comprising information related to information related to a reaction or a missing reaction of a user to the detected event outside the vehicle from a second user detection module,

wherein the processing module is configured to determine a user situation awareness level based on the information related to the reaction or the missing reaction of the user to the detected event from the first user detection signal and the information related to the reaction or the missing reaction of the user to the detected event from the second user detection signal.

7. The apparatus according to claim 1, wherein the information related to the detected event outside the vehicle is information related to a position or a movement of an object outside the vehicle and/or a time of detection of the event outside the vehicle.

8. The apparatus according to claim 1, wherein the input module is configured to receive the first user detection signal from at least one sensor for measuring a pulse rate of the user, a skin conductance of the user, a gaze direction of the user, a gaze time of the user, a saccadic speed of the user, or image data related to the user.

9. The apparatus according to claim 1, wherein the first user detection signal comprises speed variation of the vehicle, acceleration variation of the vehicle or jerk information of the vehicle related to the reaction or a missing reaction of the user to the detected event outside the vehicle.

10. The apparatus according to claim 1, wherein the processing module is configured to generate the control signal comprising information for selecting at least one user situation awareness modification parameter to be varied by the user situation awareness modification module from a plurality of controllable user situation awareness modification parameters.

11. The apparatus according to claim 10, wherein the at least one user situation awareness modification parameter is selected based on a previously selected situation awareness modification parameter.

12. The apparatus according to claim 10, wherein the processing module is configured to select a first user situation awareness modification parameter during a first time interval and to select a second user situation awareness modification parameter during a subsequent second time interval if a modification of the current user situation awareness levels towards the currently necessary situation awareness level is too slow.

13. The apparatus according to claim 1, wherein the plurality of user situation awareness modification parameters comprises a light level, a sound level, an odor, a temperature level, a degree of air flow, an oxygen level, or a haptic communication level.

14. The apparatus according to claim 1, wherein the processing module is configured to determine a currently necessary situation awareness level by selecting one of a plurality of predefined situation awareness levels based on the information related to a detected event from the first event detection signal outside the vehicle, and to generate the control signal to control the user situation awareness modification module based on the currently necessary situation awareness level and the user situation awareness level.

15. The apparatus according to claim 14, wherein the processing module is configured to determine a currently necessary situation awareness level based on a driving situation parameter signal and the first event detection signal, wherein the driving situation parameter signals comprises information related to a weather condition, a period of the day, a traffic condition, or a selectable level of autonomy of the user.

16. The apparatus according to claim 14, wherein the processing module is configured to generate the control signal to control the user situation awareness modification module such that the situation awareness level of the user is modified towards the currently necessary situation awareness level.

17. A partly-autonomous or highly autonomous vehicle comprising an apparatus configured to control a user situation awareness modification of a user of a vehicle, the apparatus comprising:

an input module configured to repeatedly receive a first event detection signal comprising information related to a detected event outside a vehicle and to repeatedly receive a first user detection signal comprising information related to a reaction or a missing reaction of a user to the detected event outside the vehicle; and

a processing module configured to determine a user situation awareness level based on the information related to the detected event outside the vehicle and the information related to the reaction of the user to the detected event, and to generate a control signal configured to control a user situation awareness modification module based on the determined user situation awareness level,

wherein the processing module is configured to determine a desired time-till-takeover based on the detected event outside the vehicle,

wherein the processing module is configured to generate the control signal to trigger the user situation awareness modification module if the determined user situation awareness level indicates that a predicted time-till-takeover is larger than the desired time-till-takeover.

18. A method for controlling a user situation awareness modification of a user of a vehicle, the method comprising:

repeatedly receiving a first event detection signal comprising information related to a detected event outside a vehicle;

repeatedly receiving a first user detection signal comprising information related to a reaction or a missing reaction of a user to the detected event outside the vehicle;

determining a user situation awareness level based on the information related to the detected event outside the vehicle and the information related to the reaction of the user to the detected event;

determining a desired time-till-takeover based on the detected event outside the vehicle; and

generating a control signal configured to control a user situation awareness modification module based on the determined user situation awareness level,

wherein generating the control signal comprises generating the control signal to trigger the user situation awareness modification module if the determined user situation awareness level indicates that a predicted time-till-takeover is larger than the desired time-till-takeover.