Combining driver alertness with advanced driver assistance systems (ADAS)

• US 10,137,893 B2
• Filed: 09/25/2017
• Issued: 11/27/2018
• Est. Priority Date: 09/26/2016
• Status: Active Grant

• Current Assignee: Knapp Investment Company Limited
• Original Assignee: Keith J. Hanna
• Inventors: Hanna, Keith J.
• Primary Examiner(s): To, Tuan C
• Application Number:
  US15/714,727
• Publication Number:
  US 20180086339A1
• Time in Patent Office: 428 Days
• Field of Search: 701301
• CPC Class Codes: B60W 2540/00 Input parameters relating to occupants
• CPC Class Codes: B60W 2540/225 Direction of gaze
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0953 the prediction being responsive to vehicle dynamic parameters
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

The disclosure is directed to systems and methods for operating or controlling an ADAS mechanism. A first sensor of an ADAS mechanism can determine a potential obstacle to a vehicle, and a position of the potential obstacle relative to the first sensor. A second sensor can determine a gaze angle of a user of the vehicle. An activation engine in communication with the first sensor and the second sensor can determine a proximity of the gaze angle of the user to the determined position of the potential obstacle. The activation engine can control, in response to the determined proximity, an operation of the ADAS mechanism for responding to the potential obstacle.

Claims

1. A method of operating an advanced driver assistance system (ADAS) mechanism, the method comprising:

determining, by a first sensor of an ADAS mechanism, a potential obstacle to a vehicle, and a position of the potential obstacle relative to the first sensor;

determining, by a second sensor, a gaze angle of a user of the vehicle;

determining, by an activation engine in communication with the first sensor and the second sensor, that a proximity of the gaze angle of the user to the determined position of the potential obstacle comprises a value that is within a first range of a plurality of defined ranges, each of the plurality of defined ranges having an ADAS action preconfigured for each action of the user defined for the respective defined range; and

in response to the determined proximity being within the first range and to detecting a first action of the user defined for the first range, performing a first ADAS action preconfigured for the first action of the user defined for the first range, the first ADAS action comprising at least one of;

controlling an operation of the ADAS mechanism for responding to the potential obstacle, or

providing an alert to at least one of;

the user regarding the potential obstacle, or the potential obstacle.

2. The method of claim 1, further comprising responsive to determining that a proximity of the gaze angle of the user to a position of a second potential obstacle is within the first range, and to detecting a second action of the user defined for the first range, performing, by the ADAS mechanism, a second ADAS action different from the first ADAS action, and preconfigured for the second action of the user defined for the first range.

3. The method of claim 1, comprising:

determining, by the activation engine, that the determined proximity of the gaze angle of the user to the determined position at a first time instance is within the first range, and that the user appears to gaze further away from the potential obstacle at a subsequent time instance; and

increasing a threshold for the ADAS mechanism to initiate collision avoidance responsive to the determination that the determined proximity is within the first range and that the user appears to gaze further away from the potential obstacle at a subsequent time instance.

4. The method of claim 1, comprising:

determining, by the activation engine, that the determined proximity of the gaze angle of the user to the determined position is within a second range of the plurality of defined ranges; and

at least one of;

providing the alert to the at least one of;

the user regarding the potential obstacle, or the potential obstacle;

or

maintaining or decreasing a threshold for the ADAS mechanism to initiate collision avoidance.

5. The method of claim 1, comprising:

determining, by the activation engine, that the determined proximity of the gaze angle of the user to the determined position is within the first range, at a first time instance and at a subsequent time instance; and

increasing a threshold for the ADAS mechanism to initiate collision avoidance.

6. The method of claim 1, comprising:

determining, by the activation engine, that the determined proximity of the gaze angle of the user to the determined position is within the first range, and that the user appears to respond to the potential obstacle in a sudden or immediate manner; and

decreasing or maintaining a threshold for the ADAS mechanism to initiate collision avoidance.

7. The method of claim 1, comprising:

determining, by the activation engine, that the determined proximity of the gaze angle of the user to the determined position is within the first range, and that the user'"'"'s facial or eye expression corresponds to shock or distress; and

decreasing a threshold for the ADAS mechanism to initiate collision avoidance.

8. The method of claim 1, further comprising calibrating the second sensor for determining the gaze angle of the user according to a movement of the user relative to the second sensor.

9. The method of claim 1, wherein the calibrating comprises determining an offset between the gaze angle of the user determined by the second sensor, and a direction or the location of the potential obstacle relative to the second sensor according to the first sensor.

10. The method of claim 1, wherein controlling the operation of the ADAS mechanism comprises modifying a predetermined operation of the ADAS mechanism for responding to the potential obstacle.

11. The method of claim 1, comprising:

comparing a rate of change of the gaze angle and a rate of change of an angle of the potential obstacle relative to the first or second sensor; and

controlling the operation of the ADAS mechanism for responding to the potential obstacle, according to the comparison.

12. A system for operating an advanced driver assistance system (ADAS) mechanism, the system comprising:

a first sensor of an ADAS mechanism, the first sensor configured to determine a potential obstacle to a vehicle, and a position of the potential obstacle relative to the first sensor;

a second sensor configured to determine a gaze angle of a user of the vehicle; and

an activation engine in communication with the first sensor and the second sensor, the activation engine configured to;

determine that a proximity of the gaze angle of the user to the determined position of the potential obstacle comprises a value that is within a first range of a plurality of defined ranges, each of the plurality of defined ranges having an ADAS action preconfigured for each action of the user defined for the respective defined range; and

in response to the determined proximity being within the first range and to detecting a first action of the user defined for the first range, performing a first ADAS action preconfigured for the first action of the user defined for the first range, the first ADAS action comprising at least one of;

controlling an operation of the ADAS mechanism for responding to the potential obstacle, or

providing an alert to at least one of;

the user regarding the potential obstacle, or the potential obstacle.

13. The system of claim 12, wherein the activation engine is configured to, responsive to determining that a proximity of the gaze angle of the user to a position of a second potential obstacle is within the first range, and to detecting a second action of the user defined for the first range, performing a second ADAS action different from the first ADAS action, and preconfigured for the second action of the user defined for the first range.

14. The system of claim 12, wherein the activation engine is configured to:

determine that the determined proximity of the gaze angle of the user to the determined position at a first time instance is within the first range, and that the user appears to gaze further away from the potential obstacle at a subsequent time instance; and

increase a threshold for the ADAS mechanism to initiate collision avoidance responsive to the determination that the determined proximity is within the first range and that the user appears to gaze further away from the potential obstacle at a subsequent time instance.

15. The system of claim 12, wherein the activation engine is configured to:

determine that the determined proximity of the gaze angle of the user to the determined position is within a second range of the plurality of defined ranges; and

at least one of;

provide the alert to the at least one of;

the user regarding the potential obstacle, or the potential obstacle;

or

maintain or decrease a threshold for the ADAS mechanism to initiate collision avoidance.

16. The system of claim 12, wherein the activation engine is configured to:

determine that the determined proximity of the gaze angle of the user to the determined position is within the first range, at a first time instance and at a subsequent time instance; and

increase a threshold for the ADAS mechanism to initiate collision avoidance.

17. The system of claim 12, wherein the activation engine is configured to:

determine that the determined proximity of the gaze angle of the user to the determined position is within the first range, and that the user appears to respond to the potential obstacle in a sudden or immediate manner; and

decrease or maintain a threshold for the ADAS mechanism to initiate collision avoidance.

18. The system of claim 12, wherein the activation engine is configured to:

determine that the determined proximity of the gaze angle of the user to the determined position is within the first range, and that the user'"'"'s facial or eye expression corresponds to shock or distress; and

decrease a threshold for the ADAS mechanism to initiate collision avoidance.

19. The system of claim 12, wherein the activation engine is configured to calibrate the second sensor for determining the gaze angle of the user, according to a movement of the user relative to the second sensor.

20. The system of claim 12, wherein the calibrating comprises determining an offset between the gaze angle of the user determined by the second sensor, and a direction or the location of the potential obstacle relative to the second sensor according to the first sensor.