COMBINING DRIVER AND ENVIRONMENT SENSING FOR VEHICULAR SAFETY SYSTEMS

• US 20110169625A1
• Filed: 01/14/2010
• Published: 07/14/2011
• Est. Priority Date: 01/14/2010
• Status: Active Grant

• Current Assignee: Toyota Motor Corporation
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Samples, Michael Edward, James, Michael Robert, Kalik, Steven F.
• Granted Patent: US 8,384,534 B2
• Time in Patent Office: Days
• US Class Current: 340/439
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2540/00 Input parameters relating to occupants
• CPC Class Codes: B60W 2540/225 Direction of gaze
• CPC Class Codes: B60W 30/095 Predicting travel path or likelihood of collision
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G01S 13/89 for mapping or imaging
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

An apparatus for assisting safe operation of a vehicle includes an environment sensor system detecting hazards within the vehicle environment, a driver monitor providing driver awareness data (such as a gaze track), and an attention-evaluation module identifying hazards as sufficiently or insufficiently sensed by the driver by comparing the hazard data and the gaze track. An alert signal relating to the unperceived hazards can be provided.

Claims

1. An apparatus for assisting safe operation of a vehicle, the vehicle having a vehicle environment, the apparatus comprising:

an environment sensor system, operable to detect hazards within the vehicle environment and to provide hazard data;

a driver monitor, operable to provide driver awareness data, the driver awareness data including a gaze track; and

an attention-evaluation module, identifying perceived hazards and unperceived hazards by comparing the hazard data and the gaze track,

the attention-evaluation module providing an alert signal relating to the unperceived hazards.

2. The apparatus of claim 1, further comprising a hazard alarm providing an alert on receiving the alert signal.

3. The apparatus of claim 1, the hazard data including hazard location data,

an unperceived hazard having a hazard location outside a predetermined distance from the gaze track.

4. The apparatus of claim 3, wherein the predetermined distance is a predetermined angular distance from the gaze track.

5. The apparatus of claim 3, the hazard location data corresponding to hazard locations over a first predetermined time period,

the driver awareness data corresponding to driver awareness over a second predetermined time period,

the first and second predetermined time periods each having a duration of between approximately 0.01 seconds and approximately 10 seconds.

6. The apparatus of claim 1, the driver awareness data further including eye fixation time data,

at least some hazards requiring an appreciable eye fixation time to be identified as perceived hazards.

7. The apparatus of claim 1, the environment sensor system including a radar sensor.

8. The apparatus of claim 1, the hazards including other vehicles in the vehicle environment.

9. The apparatus of claim 1, the driver monitor including:

at least one image sensor, providing image data;

a processing module operable to determine a gaze direction from the image data.

10. The apparatus of claim 1, further comprising a vehicle control module operable to adjust vehicle operation in response to unperceived hazards.

11. The apparatus of claim 1, the attention-evaluation module including an electronic circuit having a processor,

the processor operable to execute an algorithm comparing hazard location data with the gaze track,

the electronic circuit being further operable to estimate driver awareness using a model of driver memory.

12. An apparatus for assisting safe operation of a vehicle, the vehicle having a vehicle environment, the apparatus comprising:

an environment sensor system, operable to detect hazards within the vehicle environment and provide hazard data,

hazard data including hazard location data,

the hazards including other vehicles in the vehicle environment;

a driver monitor, the driver monitor including a gaze tracker and providing gaze track data including a gaze track;

an attention-evaluation module, operable to receive the hazard data and the gaze track data, and to identify unperceived hazards by comparing the hazard locations and the gaze track; and

a hazard alarm providing alerts to the driver relating to the unperceived hazards.

13. The apparatus of claim 12,

the environment sensor system including radar sensors supported by the vehicle,

the driver monitor including a computer vision system operable to determine a gaze track from the image data relating to the driver,

the vehicle being a land vehicle.

14. The apparatus of claim 12, the attention-evaluation module identifying perceived hazards and unperceived hazards by comparing the hazard locations and the gaze track over a predetermined time period,

perceived hazards being within a predetermined distance from the gaze track during the predetermined time period,

the predetermined distance being a predetermined angular separation.

15. The apparatus of claim 12, the gaze track data including eye fixation data relating to the driver,

the attention-evaluation module being operable to identify unperceived hazards by comparing the hazard locations and the eye fixation data.

16. A method for assisting safe operation of a vehicle by a driver, the method comprising:

monitoring the vehicle environment using environment sensors supported by the vehicle, so as to obtain hazard data;

monitoring the driver, so as to obtain driver awareness data, the driver awareness data including gaze direction data;

comparing the hazard data and the driver awareness data so as to identify unperceived hazards; and

providing alerts to the driver relating to the unperceived hazards.

17. The method of claim 16, wherein the gaze direction data includes a gaze track over a predetermined time period,

the gaze track fixating the perceived hazards during the predetermined time period.

18. The method of claim 16, wherein the gaze direction data includes a fixation data,

the method including comparing the hazard location data and the fixation data to identify unperceived hazards.