Lane departure warning/assistance method and system having a threshold adjusted based on driver impairment determination using pupil size and driving patterns

• US 9,542,847 B2
• Filed: 08/16/2013
• Issued: 01/10/2017
• Est. Priority Date: 02/16/2011
• Status: Expired due to Fees

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Sherony, Rini, Hada, Hideki
• Primary Examiner(s): Mengistu, Amare
• Assistant Examiner(s): MATHEWS, CRYSTAL
• Application Number:
  US13/968,927
• Publication Number:
  US 20130335213A1
• Time in Patent Office: 1,243 Days
• US Class Current: 1/1
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2540/043 Identity of occupants
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 2540/26 Incapacity
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A method/system for lane departure warning/assistance that warns the driver that the vehicle is about to leave a current lane and enter an adjacent lane. The driver is identified, and a corresponding profile is accessed. The driver'"'"'s pupils may be measured and compared to pupil size baseline data stored in the accessed profile. If the difference in pupil size exceeds a pupil size baseline by more than a deviation level, the method/system may adjust a lane departure warning/assistance threshold of a lane departure detector that warns the driver each time the vehicle is getting too close to an adjacent lane, thus alerting the driver that the vehicle may drift into the next lane. Driving patterns, such as steering angles and braking force, may also be used to adjust the lane departure warning/assistance threshold and determine whether the driver may benefit from lane departure warning/assistance.

Claims

1. A method comprising:

identifying a driver of a vehicle;

storing, in a memory, a profile having a driver identification of the driver and profile data including pupil size baseline data corresponding to a stored pupil size of the driver during a previous driving session, braking baseline data corresponding to stored braking operations of the driver and steering baseline data including an average steering angle of stored steering operations of the driver;

detecting, using at least one sensor or camera, impairment indication data during a current driving session of the driver including pupil size data corresponding to a current pupil size of the driver during the current driving session, braking data for the current driving session of the driver, and steering data corresponding to the current driving session of the driver;

determining, by a vehicle control unit, that the driver is impaired by comparing the impairment indication data to the profile data including comparing the pupil size data to the pupil size baseline data, the braking data for the current driving session to the braking baseline data corresponding to the stored braking operations and an average steering angle from the steering data of the current driving session to the average steering angle included in the steering baseline data;

determining that a deviation between the braking data and the braking baseline data is greater than a first threshold deviation and that a deviation between the average steering angle of the current driving session and the average steering angle included in the steering baseline data is greater than a second threshold deviation; and

adjusting, by the vehicle control unit, a lane departure warning/assistance threshold in response to determining that the deviation between the braking data and the braking baseline data is greater than the first threshold deviation and that the deviation between the average steering angle of the current driving session and the average steering angle included in the steering baseline data is greater than the second threshold deviation.

2. The method of claim 1 further comprising:

determining that the driver is a new driver;

inviting the driver to create a new profile;

creating the new profile in response to receiving input indicating that the driver accepts the invitation;

detecting and storing the pupil size baseline data;

detecting and storing the braking baseline data; and

detecting and storing the steering baseline data.

3. The method of claim 1 further comprising:

determining that the driver is a known driver with an incomplete profile;

retrieving the profile data corresponding to the incomplete profile from the memory;

determining data needed to complete the incomplete profile; and

detecting and storing the needed data.

4. The method of claim 1 further comprising outputting, by an output device, a warning or providing, via the output device, lane departure assistance based on the adjusted lane departure warning/assistance threshold, wherein outputting the warning includes at least one of displaying, on a vehicle display, a message indicating that the vehicle is encroaching an edge of a lane of the road, or playing, via a speaker, an audible message indicating that the vehicle is encroaching the edge of the lane of the road.

5. The method of claim 1 further comprising outputting, by an output device, a warning or providing, via the output device, lane departure assistance based on the adjusted lane departure warning/assistance threshold, wherein outputting the warning or providing the lane departure assistance comprises:

identifying a vehicle point of reference corresponding to the vehicle;

identifying a first point of reference corresponding to a first lane marker delimiting a first boundary of a lane of the road;

identifying a second point of reference corresponding to a second lane maker delimiting a second boundary of the lane of a road;

calculating a first distance between the vehicle point of reference corresponding to the vehicle and the first point of reference corresponding to the first lane marker;

calculating a second distance between the vehicle point of reference corresponding to the vehicle and the second point of reference corresponding to the second lane marker; and

determining that the vehicle is encroaching the first boundary or the second boundary of the lane when at least one of the first distance or the second distance is less than or equal to a distance corresponding to the adjusted lane departure warning/assistance threshold.

6. The method of claim 5 wherein identifying the first point of reference corresponding to the first lane marker is performed by a first camera and identifying the second point of reference corresponding to the second lane marker is performed by at least one of the first camera or a second camera.

7. The method of claim 1 further comprising:

determining a current time-to-lane-crossing corresponding to a predicted time period remaining before the vehicle crosses a lane marker, wherein the lane departure warning/assistance threshold is a time-to-lane-crossing threshold; and

outputting a warning or providing lane departure assistance when a current time-to-lane-crossing is less than or equal to the time-to-lane-crossing threshold.

8. The method of claim 1 wherein determining that the driver is impaired further comprises determining that the detected pupil size data deviates from the stored pupil size baseline data by more than a pupil size deviation level.

9. The method of claim 1 wherein the pupil size baseline data and the pupil size data are each detected using the camera and wherein the camera is located within an interior of the vehicle.

10. A device for providing lane departure warning or assistance to a driver of a vehicle, the device comprising:

a memory configured to store profile data corresponding to the driver including pupil size baseline data corresponding to a stored pupil size of the driver during a previous driving session, braking baseline data corresponding to a previous braking operation by the driver, and steering baseline data corresponding to a previous steering operation by the driver; and

a processor connected to the memory and configured to;

determine a current pupil size of the driver during a current driving session based on image data,

compare the current pupil size of the driver during the current driving session to the pupil size baseline data that corresponds to the stored pupil size of the driver during the previous driving session,

compare current braking data corresponding to a current braking operation by the driver to the braking baseline data stored in the memory,

compare current steering data corresponding to a current steering operation by the driver to the steering baseline data stored in the memory, and

determine that the driver is impaired when at least one of the comparison of the current pupil size of the driver to the pupil size baseline data, the comparison of the current braking data to the braking baseline data stored in the memory, or the comparison of the current steering data to the steering baseline data stored in the memory indicates that the driver is impaired, until at least one of the vehicle is turned off or the comparison of the current pupil size to the pupil size baseline data, the comparison of the current braking data to the braking baseline data, or the comparison of the current steering data to the steering baseline data indicates that the driver is unimpaired, and

adjust a lane departure warning/assistance threshold corresponding to at least one of a threshold distance to a lane marker or a predicted time period remaining before the vehicle crosses the lane marker when the driver is determined to be impaired.

11. The device of claim 10 wherein the lane departure warning/assistance threshold is the threshold distance to a first lane marker or a second lane marker delimiting a current lane of a road, the device further comprising:

a lane marker encroachment module configured to;

determine the current lane of the road that the vehicle is traveling in based on the first lane marker and the second lane marker,

determine when a distance between the vehicle and one of the first lane marker or the second lane marker is shorter than the threshold distance, and

generate a warning signal when the distance between the vehicle and one of the first lane marker or the second lane marker is shorter than the threshold distance.

12. The system of claim 10 wherein the lane departure warning/assistance threshold is adjusted to allow the vehicle to encroach closer to the lane marker as a portion of a road narrows than when the portion of the road widens.

13. The system of claim 10 wherein the lane departure warning/assistance threshold is adjusted to allow the vehicle to encroach closer to the lane marker as a speed of the vehicle decreases.

14. A system for providing lane departure warning or assistance to a driver of a vehicle, the system comprising:

a memory configured to store pupil size baseline data corresponding to a pupil size of the driver during a previous driving session and braking baseline data corresponding to a previous braking operation by the driver;

a first camera configured to be repositioned to track a pupil of the driver when the driver changes position relative to the first camera and to detect image data corresponding to the pupil of the driver that is usable to determine a current pupil size of the pupil of the driver during a current driving session;

a second sensor or camera configured to determine a distance between the vehicle and a lane marker;

a third sensor configured to detect current braking data corresponding to a current braking operation by the driver; and

a vehicle control unit configured to;

compare the current pupil size of the driver during the current driving session to the pupil size baseline data,

compare the current braking data to the braking baseline data,

determine that the driver is impaired when at least one of the comparison of the current pupil size of the driver to the pupil size baseline data or the comparison of the current braking data to the braking baseline data indicate that the driver is impaired,

activate a lane departure warning/assistance feature having a lane departure warning/assistance threshold based on at least one of the comparison of the current pupil size of the driver to the pupil size baseline data and the current braking data to the braking baseline data, and

adjust a lane departure warning/assistance threshold corresponding to at least one of the distance between the vehicle and the lane marker or a predicted time period remaining before the vehicle crosses the lane marker when the driver is determined to be impaired.

15. The system of claim 14 further comprising an audio control unit configured to output an audio message based on the distance between the vehicle and the lane marker and based on the adjusted lane departure/warning assistance threshold.

16. The system of claim 14 further comprising a display control unit configured to output a visual message based on the distance between the vehicle and the lane marker and based on the adjusted lane departure/warning assistance threshold.

17. The system of claim 14 further comprising a fourth sensor configured to detect current steering data corresponding to a current steering operation by the driver,

wherein the memory is configured to store steering baseline data, the vehicle control unit is further configured to compare the current steering data to the steering baseline data, and determine that the driver is impaired further based on the comparison of the current steering data to the steering baseline data.

18. The system of claim 14 further comprising a steering wheel and wherein the first camera is positioned in proximity of the steering wheel.

19. The system of claim 14 wherein the first camera is configured to reposition itself to track the pupil of the driver.

20. The system of claim 14 wherein the vehicle control unit is further configured to start a timer in response to determining that the driver is impaired and to shut down the vehicle when the timer expires.