LANE DEPARTURE WARNING/ASSISTANCE METHOD AND SYSTEM HAVING A THRESHOLD ADJUSTED BASED ON DRIVER IMPAIRMENT DETERMINATION USING PUPIL SIZE AND DRIVING PATTERNS

• US 20130335213A1
• Filed: 08/16/2013
• Published: 12/19/2013
• Est. Priority Date: 02/16/2011
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Sherony, Rini, Hada, Hideki
• Granted Patent: US 9,542,847 B2
• Time in Patent Office: Days
• US Class Current: 340/439
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2540/043 Identity of occupants
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 2540/26 Incapacity
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A method/system for lane departure warning/assistance that warns the driver that the vehicle is about to leave a current lane and enter an adjacent lane. The driver is identified, and a corresponding profile is accessed. The driver'"'"'s pupils may be measured and compared to pupil size baseline data stored in the accessed profile. If the difference in pupil size exceeds a pupil size baseline by more than a deviation level, the method/system may adjust a lane departure warning/assistance threshold of a lane departure detector that warns the driver each time the vehicle is getting too close to an adjacent lane, thus alerting the driver that the vehicle may drift into the next lane. Driving patterns, such as steering angles and braking force, may also be used to adjust the lane departure warning/assistance threshold and determine whether the driver may benefit from lane departure warning/assistance.

Claims

1. A method comprising:

identifying a driver of a vehicle;

determining that a completed profile is available corresponding to the driver;

retrieving the completed profile from a corresponding vehicle memory;

collecting impairment indication data;

comparing the impairment indication data to data corresponding to the retrieved completed profile;

determining, based on the comparison of the impairment indication data to the data corresponding to the completed profile, that the driver is to be treated as an impaired driver;

adjusting a lane departure warning/assistance threshold of a lane departure detector based on the comparison of the impairment indication data to the data corresponding to the retrieved completed profile data; and

outputting a warning or providing lane departure assistance based on the adjusted lane departure warning/assistance threshold.

2. The method of claim 1 further comprising:

in response to identifying the driver as a new driver, inviting the driver to create a profile;

in response to receiving input that the driver accepts the invitation to create a new profile, creating a driver identification profile;

collecting and storing data related to a size of a pupil of the driver; and

collecting and storing data related to the driver'"'"'s operation of the vehicle.

3. The method of claim 1 further comprising:

in response to identifying the driver as a known driver with an incomplete profile, retrieving existing profile data corresponding to the incomplete profile from the vehicle memory;

determine data needed to complete the incomplete profile; and

collect and store the needed data to complete the incomplete profile.

4. The method of claim 1 wherein the step of outputting the warning includes one of:

displaying on a vehicle display a message indicating that the vehicle is encroaching an edge of a lane of the road, or playing an audible message indicating that the vehicle is encroaching the edge of the lane of the road.

5. The method of claim 1 wherein the step of outputting the warning or providing the lane departure assistance further comprises:

identifying a point of reference corresponding to the vehicle;

identifying a point of reference corresponding to a first lane marker;

identifying a point of reference corresponding to a second, adjacent lane maker;

calculating a first distance between the point of reference corresponding to the vehicle and the first lane marker;

calculating a second distance between the point of reference corresponding to the vehicle and the point of reference corresponding to the second, adjacent lane marker; and

determining that the vehicle is encroaching an edge of a lane of a road when at least one of the first distance and second distance drops at or below a distance corresponding to the adjusted lane departure warning/assistance threshold.

6. The method of claim 5 wherein the step of identifying the point of reference corresponding to the first lane marker is performed by a camera and wherein the step of identifying the point of reference corresponding to the second, adjacent lane marker is performed by a camera.

7. The method of claim 1 further comprising determining a current time-to-lane-crossing corresponding to a predicted time period remaining before the vehicle crosses a lane marking, wherein:

the lane departure warning/assistance threshold is a time-to-lane-crossing threshold,

the step of adjusting the time-to-lane-crossing threshold is performed in response to determining that the driver is to be treated as an impaired driver, and

the step of outputting the warning or providing the lane departure assistance is performed when the current time-to-lane-crossing is less than or equal to the time-to-lane-crossing threshold.

8. The method of claim 1 wherein the impairment indication data comprises braking data and steering data.

9. The method of claim 8 wherein the step of determining that the driver is to be treated as an impaired driver includes at least one of:

determining that a size of a pupil of the driver corresponding to the impairment indication data is different than a size of a pupil of the driver corresponding to the completed profile;

determining that the braking data of the impairment indication data is different than braking data corresponding to the completed profile; and

determining that the steering data of the impairment indication data is different than steering data corresponding to the completed profile.

10. The method of claim 9 wherein the step of determining that the driver is to be treated as an impaired driver further comprises determining that a diameter of the pupil of the driver corresponding to the impairment indication data deviates from a pupil diameter baseline of the driver corresponding to the completed profile by more than a pupil size deviation level.

11. The method of claim 9 wherein the step of determining that the driver is to be treated as an impaired driver further comprises:

determining that a diameter of the pupil of the driver corresponding to the impairment indication data does not deviate from a pupil diameter baseline of the driver corresponding to the completed profile by more than a pupil size deviation level, and that an average braking force applied to a set of vehicle brakes corresponding to the impairment indication data deviates from a braking force baseline corresponding to the completed profile by more than a braking deviation level, and that an average steering angle of the vehicle corresponding to the impairment indication data deviates from a steering angle baseline of the vehicle corresponding to the completed profile by more than a steering deviation level.

12. The method of claim 1 wherein a size of a pupil of the driver corresponding to the completed profile is obtained during a driving session different than a driving session during which the size of the pupil of the driver corresponding to the impairment indication data is collected.

13. The method of claim 12 wherein the size of the pupil of the driver corresponding to the completed profile is obtained using a camera located within an interior of the vehicle, and the size of the pupil of the driver corresponding to the impairment indication data is collected using the camera located within the interior of the vehicle.

14. A device comprising:

a processor;

a memory coupled to the processor;

an image processing module coupled to the processor and configured to process an image to determine a pupil size; and

a pupil size comparator configured to receive the pupil size of the image and compare the pupil size of the image to a pupil size corresponding to a user'"'"'s profile stored in the memory, the pupil size comparator further configured to output a result of the pupil size comparison;

wherein the processor is configured to adjust a lane departure warning/assistance threshold of a lane departure detector based on the result of the pupil size comparison.

15. The device of claim 14 further comprising:

a steering data comparator configured to receive current steering data and compare the current steering data to steering baseline data stored in the memory, the steering data comparator further configured to output a result of the steering data comparison; and

a braking data comparator configured to receive current braking data and compare the current braking data to braking baseline data stored in the memory, the braking data comparator further configured to output a result of the braking data comparison;

wherein the processor is further configured to adjust the lane departure warning/assistance threshold of the lane departure detector based on the result of the pupil size comparison, the result of the steering data comparison and the result of the braking data comparison.

16. The device of claim 14 wherein the lane departure warning/assistance threshold is an encroachment threshold adjusted by the processor, the device further comprising:

a lane marker encroachment module configured to determine whether a lane of a road that a vehicle is traveling in based on a pair of lane markers, and to determine when a distance between the vehicle and one of the lane markers is shorter than the adjusted encroachment threshold, the lane marker encroachment module further configured to output a warning signal when the distance between the vehicle and one of the lane markers is shorter than the adjusted encroachment threshold.

17. A system comprising:

a first sensor or camera configured to capture an image of a driver'"'"'s pupil;

a second sensor or camera configured to determine a distance between a lane marker and a vehicle; and

a vehicle control unit configured to;

receive an image of the driver'"'"'s pupil and the distance between the lane marker and the vehicle, the vehicle control unit configured to determine if the driver is impaired based at least in part on the image of the driver'"'"'s pupil,

adjust an encroachment threshold of a lane departure detector when the driver is determined to be impaired,

determine that the vehicle is encroaching on the lane marker by comparing a distance between the lane marker and the vehicle to the adjusted encroachment threshold, and

output a warning signal when the driver is impaired and the vehicle is encroaching on the lane marker.

18. The system of claim 17 further comprising an audio control unit configured to receive the warning signal from the vehicle control unit and output an audio message.

19. The system of claim 17 further comprising a display control unit configured to receive the warning signal from the vehicle control unit and output a visual message.

20. The system of claim 17 wherein the vehicle control unit is further configured to determine if the driver is impaired based on steering and braking data.