Auto turn signal initiation based on lane change and driver history

• US 10,011,216 B1
• Filed: 04/27/2017
• Issued: 07/03/2018
• Est. Priority Date: 04/27/2017
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Rovik, Christopher L.
• Primary Examiner(s): Casillashernandez, Omar
• Assistant Examiner(s): Casillas, Omar
• Application Number:
  US15/499,438
• Time in Patent Office: 432 Days
• Field of Search: 340465
• CPC Class Codes: B60Q 1/346 with automatic actuation
• CPC Class Codes: G01S 19/13 Receivers
• CPC Class Codes: G01S 19/42 Determining position
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

An auto turn signal system for activating a turn signal of a vehicle includes an electronic control unit, a processor, a non-transitory computer readable memory, a camera and a machine-readable instruction set. The camera automatically captures image data and transmits the image data to the electronic control unit. The machine-readable instruction set causes the processor to receive the image data from the camera, determine at least one of a driver head movement and a driver eye movement based on the image data, predict a change in a lane of travel from a current lane of travel to a predicted destination lane of travel, detect initiation of the change from the current lane of travel to the predicted destination lane of travel, and automatically activate the turn signal in response to detecting the initiation of the change from the current lane of travel to the predicted destination lane of travel.

Claims

1. An auto turn signal system for activating a turn signal of a vehicle, the auto turn signal system comprising:

an electronic control unit comprising a processor and a non-transitory computer readable memory;

one or more cameras communicatively coupled to the electronic control unit, wherein the one or more cameras automatically captures image data and transmits the image data to the electronic control unit;

a driver profile, stored in the non-transitory computer readable memory, comprising a characterization of at least one of a driver head movement and a driver eye movement as indicative of a driver planning to make a change in a lane of travel; and

a machine-readable instruction set stored in the non-transitory computer readable memory that causes the auto turn signal system to perform at least the following when executed by the processor;

receive the image data from the one or more cameras;

determine at least one of the driver head movement and the driver eye movement based on the image data;

compare at least one of the driver head movement and the driver eye movement determined from the image data to at least one of the driver head movement and the driver eye movement of the driver profile;

predict a change in the lane of travel from a current lane of travel to a predicted destination lane of travel based on the comparison;

detect initiation of the change from the current lane of travel to the predicted destination lane of travel;

automatically activate the turn signal in response to detecting the initiation of the change from the current lane of travel to the predicted destination lane of travel; and

update the driver profile, in response to automatic activation of the turn signal, such that the characterization of at least one of the driver head movement and at least one of the driver eye movement in the driver profile is refined based on at least one of the driver head movement and the driver eye movement determined from the image data.

2. The auto turn signal system of claim 1, further comprising a near-infrared light emitter focusing near-infrared light onto one or more eyes of the driver, wherein the one or more cameras includes an infrared camera positioned to capture the near-infrared light reflected from the one or more eyes of the driver.

3. The auto turn signal system of claim 1, wherein the machine-readable instruction set further causes the processor to:

identify the driver of the vehicle;

create the driver profile for an identified driver when the driver profile for the identified driver does not exists in the non-transitory computer readable memory; and

recall the driver profile for the identified driver when the driver profile for the identified driver exists in the non-transitory computer readable memory.

4. The auto turn signal system of claim 1, further comprising one or more environment sensors coupled to the electronic control unit, wherein the one or more environment sensors generate one or more signals indicative of one or more conditions around the driver, and wherein the machine-readable instruction set further causes the processor to:

receive the one or more signals indicative of the one or more conditions around the driver; and

update the driver profile with the one or more conditions around the driver leading up to the driver initiating the change from the current lane of travel to the predicted destination lane of travel when the driver failed to manually activate the turn signal.

5. The auto turn signal system of claim 1, wherein the machine-readable instruction set further causes the processor to:

adjust a relevance value of an element of the driver profile such that the auto turn signal system is more or less sensitive to the element of the driver profile when predicting the change in the lane of travel from the current lane of travel to the predicted destination lane of travel based on at least one of the driver head movement and the driver eye movement.

6. The auto turn signal system of claim 1, further comprising a lane departure system communicatively coupled to the electronic control unit, wherein the lane departure system comprises a forward looking camera capturing image data from in front of the vehicle for determining the lane of travel and the lane departure system provides one or more signals to the electronic control unit indicative of a position of the vehicle relative to the lane of travel.

7. The auto turn signal system of claim 6, wherein the lane departure system comprises one or more cameras for detecting a lane marking and determining when the vehicle travels over the lane marking.

8. An auto turn signal system for activating a turn signal of a vehicle, the auto turn signal system comprising:

an electronic control unit comprising a processor and a non-transitory computer readable memory;

one or more cameras communicatively coupled to the electronic control unit, wherein the one or more cameras automatically captures image data and transmits the image data to the electronic control unit;

a driver profile, stored in the non-transitory computer readable memory, comprising one or more elements corresponding to a characterization of one or more driver actions indicative of a driver planning to make a change in a lane of travel; and

a machine-readable instruction set stored in the non-transitory computer readable memory that causes the auto turn signal system to perform at least the following when executed by the processor;

receive the image data from the one or more cameras;

identify the driver of the vehicle;

determine the one or more driver actions based on the image data;

compare the one or more driver actions determined from the image data with the one or more elements of the driver profile corresponding to an identity of the driver of the vehicle;

predict a change in the lane of travel from a current lane of travel to a predicted destination lane of travel based on the comparison of the one or more driver actions determined from the image data with the one or more elements of the driver profile;

detect initiation of the change from the current lane of travel to the predicted destination lane of travel;

automatically activate the turn signal in response to detecting the initiation of the change from the current lane of travel to the predicted destination lane of travel; and

update the one or more elements of the driver profile, in response to the automatic activation of the turn signal, such that the characterization of the one or more driver actions in the driver profile is refined based on the one or more driver actions determined from the image data.

9. The auto turn signal system of claim 8, further comprising a near-infrared light emitter focusing near-infrared light onto one or more eyes of the driver, wherein the one or more cameras includes an infrared camera positioned to capture the near-infrared light reflected from the one or more eyes of the driver.

10. The auto turn signal system of claim 8, further comprising one or more environment sensors coupled to the electronic control unit, wherein the one or more environment sensors generate one or more signals indicative of one or more conditions around the driver and the machine-readable instruction set further causes the processor to:

receive the one or more signals indicative of the one or more conditions around the driver; and

update the driver profile with the one or more conditions around the driver leading up to the driver initiating the change from the current lane of travel to the predicted destination lane of travel when the driver failed to manually activate the turn signal.

11. The auto turn signal system of claim 10, wherein the one or more conditions around the driver comprises at least one of:

traffic conditions;

weather conditions;

time of day;

location;

duration of current travel;

status of hands-free calling link;

noise level;

status of navigation system;

number of occupants in the vehicle; and

status of occupants.

12. The auto turn signal system of claim 8, wherein the one or more driver actions comprises at least one of:

eye movement;

head movement;

shoulder movement;

torso movement;

change in grip on a steering wheel;

change in weight distribution in a driver seat; and

pause in conversation.

13. The auto turn signal system of claim 8, wherein the machine-readable instruction set further causes the processor to:

adjust one or more relevance values of the one or more elements of the driver profile such that the auto turn signal system is more or less sensitive to the one or more elements of the driver profile when predicting the change in the lane of travel from the current lane of travel to the predicted destination lane of travel based on the one or more driver actions.

14. A method for automatically activing a turn signal of a vehicle, the method comprising:

receiving image data from one or more cameras;

retrieving, from a non-transitory computer readable memory, a driver profile comprising a characterization of at least one of a driver head movement and a driver eye movement as indicative of a driver planning to make a change in a lane of travel;

determining at least one of the driver head movement and the driver eye movement based on the image data;

comparing the at least one of the driver head movement and the driver eye movement determined from the image data to the at least one of the driver head movement and the driver eye movement of the driver profile;

predicting a change in the lane of travel from a current lane of travel to a predicted destination lane of travel based on the comparison;

detecting initiation of the change from the current lane of travel to the predicted destination lane of travel;

activating the turn signal in response to detecting the initiation of the change from the current lane of travel to the predicted destination lane of travel is detected; and

updating, in response to the automatic activation of the turn signal, the driver profile such that the characterization of the at least one of the driver head movement and the at least one of the driver eye movement in the driver profile is refined based on at least one of the driver head movement and the driver eye movement determined from the image data.

15. The method of claim 14, further comprising:

identifying the driver of the vehicle;

creating the driver profile for the driver when the driver profile for the driver does not exists in the non-transitory computer readable memory; and

selecting the driver profile for the driver when the driver profile for the driver exists in the non-transitory computer readable memory.

16. The method of claim 14, further comprising:

receiving, from one or more environment sensors, one or more signals indicative of one or more conditions around the driver; and

updating the driver profile with the one or more conditions around the driver leading up to the driver initiating the change from the current lane of travel to the predicted destination lane of travel when the driver failed to manually activate the turn signal.

17. The method of claim 14, further comprising:

adjusting one or more relevance values of one or more elements of the driver profile such that the method for automatically activing the turn signal of the vehicle is more or less sensitive to the one or more elements of the driver profile when predicting the change in the lane of travel from the current lane of travel to the predicted destination lane of travel based on at least one of the driver head movement and the driver eye movement.