Predictive human-machine interface using eye gaze technology, blind spot indicators and driver experience

• US 9,650,041 B2
• Filed: 12/09/2010
• Issued: 05/16/2017
• Est. Priority Date: 12/18/2009
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Choi, Dave, Kosterich, Michelle, Grebe, Cody, Lin, Jason, Feit, Steven, Scheufler, Nick, Miller, Ross
• Primary Examiner(s): MUSTAFA, IMRAN K
• Application Number:
  US13/515,249
• Publication Number:
  US 20120271484A1
• Time in Patent Office: 2,350 Days
• Field of Search: 701 1, 701301
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/30 including control of auxiliary equipment, e.g. air-conditioning compressors or oil pumps
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2540/225 Direction of gaze
• CPC Class Codes: B60W 2540/30 Driving style
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

A system and method are disclosed for preventing accidents in a vehicle. The system comprises an eye gaze detector for receiving an eye gaze vector from a driver. The system additionally comprises a proximity sensor for detecting locations of nearby objects and their velocities. A vehicle accident risk is determined based on the nearby object location and velocity. Additionally, the driver'"'"'s knowledge of vehicle circumstances is determined based on the nearby object location and velocity as well as the eye gaze location. Responsive to the driver'"'"'s knowledge and the vehicle accident risk, a vehicle alert is activated.

Claims

1. A method of providing an accident warning alert in a vehicle comprising:

detecting a location of at least one lane marker, a location of an adjacent vehicle, and a location of the vehicle relative to the at least one lane marker based on a proximity sensor;

determining an eye gaze vector based on an eye gaze detector and determining a current pupil size of a driver;

gathering, using the eye gaze vector determined from the eye gaze detector, driver experience information based on an amount of time the driver uses peripheral vision and the driver'"'"'s reaction time to the adjacent vehicle, wherein the driver experience information is further based on an amount of time the driver has spent driving in a current weather condition;

determining, using a processor, a metric of driver awareness of the vehicle location with respect to the location of the at least one lane marker using the eye gaze vector;

determining, using the processor, a vehicle accident risk based on the relative location of the vehicle and the at least one lane marker;

locking out a vehicle infotainment system, using the processor, the locking out based on the vehicle accident risk, the driver experience level, the current weather condition, a comparison of the current pupil size of the driver to a baseline pupil size for the driver, and the metric of the driver awareness of the vehicle location with respect to the location of the at least one lane marker and the adjacent vehicle; and

activating a vehicle alert, using the processor, the vehicle alert based on the vehicle accident risk, the driver experience level, and the metric of the driver awareness of the vehicle location with respect to the location of the at least one lane marker.

2. The method of claim 1 further comprising:

activating a car control function based on the vehicle accident risk, the driver experience level, and the driver'"'"'s awareness of the vehicle location.

3. The method of claim 2 wherein the car control function is at least one of a vehicle braking function, a steering wheel function, or an infotainment control function.

4. The method of claim 1 wherein determining the vehicle accident risk further comprises:

determining a location of a nearby object; and

determining whether the location of the nearby object is within a field of view of the eye gaze vector.

5. The method of claim 1 wherein the vehicle alert comprises a first visual alert when the metric of driver awareness of the vehicle location and the vehicle accident risk are above a first threshold.

6. The method of claim 5 wherein the vehicle alert comprises a second visual alert when the metric of driver awareness of the vehicle location and the vehicle accident risk are above a second threshold.

7. The method of claim 6 wherein the vehicle alert comprises an auditory alert when the metric of driver awareness of the vehicle location and the vehicle accident risk are above a third threshold.

8. The method of claim 4 wherein the nearby object is at least one of a second vehicle, or a fixed object.

9. The method of claim 8 further comprising detecting the nearby object velocity using the proximity sensor, and wherein determining the vehicle accident risk is further based on the nearby object velocity.

19. The method of claim 1 wherein the driver experience is determined based on the amount of time the driver uses peripheral vision in relation to an upcoming road feature.

10. A system of providing an accident warning alert in a vehicle comprising:

an eye gaze detector configured to receive an eye gaze vector from a driver and a current pupil size of the driver, the eye gaze detector further configured to determine driver experience based on an amount of time the driver uses peripheral vision and the driver'"'"'s reaction time to an adjacent vehicle, wherein the driver experience is further based on an amount of time the driver has spent driving in a current weather condition;

a proximity sensor configured to detect a location of at least one lane marker, a location of the adjacent vehicle, and a location of the vehicle relative to the at least one lane marker;

an electronic control unit configured to determine a vehicle accident risk based on the relative location of the vehicle and the at least one lane marker;

the electronic control unit further configured to lock out a vehicle infotainment system based on the vehicle accident risk, the driver experience level, the current weather condition, a comparison of the current pupil size of the driver to a baseline pupil size for the driver, and the metric of the driver awareness of the vehicle location with respect to the location of the at least one lane marker and the adjacent vehicle; and

the electronic control unit further configured to activate a vehicle alert based on the vehicle accident risk, the driver experience level, and a metric of driver awareness of the vehicle location with respect to the location of the at least one lane marker determined using the eye gaze vector.

11. The system of claim 10 wherein the electronic control unit is further configured to activate a car control function based on the vehicle accident risk, the driver experience level, and the metric of driver awareness of the vehicle location.

12. The system of claim 11 wherein the car control function is at least one of a vehicle braking function, a steering wheel function, or an infotainment control function.

13. The system of claim 10 wherein the determination of the vehicle accident risk is further based on determining a location of a nearby object and determining whether the location of the nearby object is within a field of view of the eye gaze vector.

14. The system of claim 10 wherein the vehicle alert comprises a first visual alert when the metric of driver awareness of the vehicle location and the vehicle accident risk are above a first threshold.

15. The system of claim 14 wherein the vehicle alert comprises a second visual alert when the metric of driver awareness of the vehicle location and the vehicle accident risk are above a second threshold.

16. The system of claim 15 wherein the vehicle alert comprises an auditory alert when the metric of driver awareness of the vehicle location and the vehicle accident risk are above a third threshold.

17. The system of claim 13 wherein the nearby object is at least one of a second vehicle, or a fixed object.

18. The system of claim 17 wherein the proximity sensor is further configured to detect a nearby object velocity, and wherein the vehicle accident risk and is further based on the nearby object velocity.

20. A method of providing an accident warning alert in a vehicle comprising:

detecting a location of at least one lane marker, a location of an adjacent vehicle, and a location of the vehicle relative to the at least one lane marker based on a proximity sensor;

determining an eye gaze vector based on an eye gaze detector and determining a current pupil size of a driver;

gathering, using the eye gaze vector determined from the eye gaze detector, driver experience information based on an amount of time the driver uses peripheral vision, the driver'"'"'s reaction time to the adjacent vehicle, and an amount of time the driver has spent driving in a current weather condition;

determining, using a processor, a metric of driver awareness of the vehicle location with respect to the location of the at least one lane marker using the eye gaze vector;

determining, using the processor, a vehicle accident risk based on the relative location of the vehicle and the at least one lane marker;

locking out a vehicle infotainment system, using the processor, the locking out based on the vehicle accident risk, the driver experience level, the current weather condition, a comparison of the current pupil size of the driver to a baseline pupil size for the driver, and the metric of the driver awareness of the vehicle location with respect to the location of the at least one lane marker and the adjacent vehicle; and

activating a vehicle alert, using the processor, the vehicle alert based on the vehicle accident risk, the driver experience level, and the metric of the driver awareness of the vehicle location with respect to the location of the at least one lane marker,

wherein the vehicle alert comprises a first visual alert when the metric of driver awareness of the vehicle location and the vehicle accident risk are above a first threshold.