Predictive human-machine interface using eye gaze technology, blind spot indicators and driver experience

• US 10,336,323 B2
• Filed: 04/11/2017
• Issued: 07/02/2019
• Est. Priority Date: 12/18/2009
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Feit, Steven, Scheufler, Nicholas A., Miller, Ross C., Kosterich, Michelle, Lin, Jason, Choi, Dave Jaeyeong, Grebe, Cody D.
• Primary Examiner(s): Mustafa, Imran K
• Application Number:
  US15/484,300
• Publication Number:
  US 20170247031A1
• Time in Patent Office: 812 Days
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/30 including control of auxiliary equipment, e.g. air-conditioning compressors or oil pumps
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2540/225 Direction of gaze
• CPC Class Codes: B60W 2540/30 Driving style
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

A system for preventing accidents includes an eye gaze detector for receiving an eye gaze vector from a driver. The system additionally comprises a proximity sensor for detecting locations of nearby objects and their velocities. A risk is determined based on the nearby object location and velocity. Additionally, the driver'"'"'s knowledge of vehicle circumstances is determined based on the nearby object location and velocity as well as the eye gaze location. Responsive to the driver'"'"'s knowledge and the risk, an alert is activated.

Claims

1. A method of providing an alert in a vehicle comprising:

detecting a location of at least one lane marker, a location of an adjacent vehicle, and a location of the vehicle relative to the at least one lane marker based on a proximity sensor;

determining an eye gaze vector based on an eye gaze detector;

determining, using a processor, a metric of driver awareness of the vehicle location with respect to the location of the at least one lane marker using the eye gaze vector;

gathering, using the eye gaze vector determined from the eye gaze detector, driver experience information based on an amount of time a driver has spent driving in a current weather condition and whether the driver looks for an upcoming turn rather than focusing on another vehicle;

determining, using the processor, a risk based on the relative location of the vehicle and the at least one lane marker; and

activating the alert, using the processor, the alert based on the risk, the driver experience level, whether the driver is looking at the upcoming turn rather than the other vehicle, and the metric of the driver awareness of the vehicle location with respect to the location of the at least one lane marker.

2. The method of claim 1, comprising activating a vehicle control function based on the risk, the driver experience level, and the metric of the driver awareness of the vehicle location with respect to the location of the at least one lane marker.

3. The method of claim 2, wherein activating the vehicle control function includes increasing seatbelt tension using an electronic pretensioner.

4. The method of claim 1, wherein the alert is activated based on a traffic condition.

5. The method of claim 1, wherein determining the eye gaze vector includes analyzing an amount of time the driver is looking to the sides.

6. The method of claim 1, wherein determining the eye gaze vector includes analyzing an amount of time the driver is using peripheral vision.

7. A system of providing an alert in a vehicle comprising:

an eye gaze detector receiving an eye gaze vector from a driver and determining a driver experience level based on the eye gaze vector and whether the driver looks for an upcoming turn rather than focusing on another vehicle;

a proximity sensor detecting a location of at least one lane marker and a location of the vehicle relative to the at least one lane marker; and

a lane keep assist system (LKAS) control unit activating the alert based on the driver experience level, whether the driver is looking at the upcoming turn rather than the other vehicle, and the vehicle location with respect to the location of the at least one lane marker.

8. The system of claim 7, wherein the LKAS control unit activates the alert at a first intensity level when the eye gaze vector indicates that the driver is looking at a road.

9. The system of claim 8, wherein the alert at the first intensity level includes a visual alert.

10. The system of claim 7, wherein the LKAS control unit activates the alert at a second intensity level when the eye gaze vector indicates that the driver is not looking at a road.

11. The system of claim 10, wherein the alert at the second intensity level includes a visual alert and an audio alert.

12. The system of claim 7, wherein the LKAS control unit activates the alert based on a velocity of another vehicle.

13. The system of claim 7, wherein the LKAS control unit activates a vehicle control function based on the driver experience level and the vehicle location with respect to the location of the at least one lane marker.

14. The system of claim 13, wherein activating the vehicle control function includes braking the vehicle.

15. A method of providing an alert in a vehicle comprising:

detecting a location of the vehicle relative to at least one lane marker or an adjacent vehicle based on a proximity sensor;

determining an eye gaze vector based on an eye gaze detector;

gathering, using the eye gaze vector determined from the eye gaze detector, driver experience information based on whether a driver is looking at an upcoming turn;

determining, using a processor, a risk based on the relative location of the vehicle and the at least one lane marker or the adjacent vehicle; and

activating the alert, using the processor, the alert based on the risk, whether the driver is looking at the upcoming turn rather than looking at another vehicle, and the driver experience level.

16. The method of claim 15, comprising activating a vehicle control function based on the risk and the driver experience level.

17. The method of claim 16, wherein activating the vehicle control function includes increasing seatbelt tension using an electronic pretensioner.

18. The method of claim 15, wherein the alert is activated based on a traffic condition.

19. The method of claim 15, wherein determining the eye gaze vector includes analyzing an amount of time the driver is looking to the sides.

20. The method of claim 15, wherein determining the eye gaze vector includes analyzing an amount of time the driver is using peripheral vision.