Vehicle turn signal detection

• US 10,800,455 B2
• Filed: 12/17/2015
• Issued: 10/13/2020
• Est. Priority Date: 12/17/2015
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Micks, Ashley Elizabeth, Banvait, Harpreetsingh, Jain, Jinesh J, Reiff, Brielle, Kadetotad, Sneha
• Primary Examiner(s): Berns, Michael A
• Application Number:
  US14/973,454
• Publication Number:
  US 20170174261A1
• Time in Patent Office: 1,762 Days
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0027 using trajectory prediction for other traffic participants
• CPC Class Codes: B62D 15/0255 Automatic changing of lane, e.g. for passing another vehicle
• CPC Class Codes: B62D 15/0265 Automatic obstacle avoidance by steering
• CPC Class Codes: B62D 15/029 Steering assistants using warnings or proposing actions to the driver without influencing the steering system
• CPC Class Codes: B62D 6/00 Arrangements for automatically controlling steering depending on driving conditions sensed and responded to, e.g. control circuits means for initiating a change in direction B62D1/00; steering valves B62D5/06; combined with means for inclining the vehicle body or wheels on bends B62D9/00
• CPC Class Codes: G05D 1/0276 using signals provided by a source external to the vehicle involving a plurality of vehicles G05D1/0287; automatically controlling vehicle speed responsive to externally generated signals B60K31/0058

Abstract

Systems, methods, and devices for detecting a vehicle'"'"'s turn signal status for collision avoidance during lane-switching maneuvers or otherwise. A method includes detecting, at a first vehicle, a presence of a second vehicle in an adjacent lane. The method includes identifying, in an image of the second vehicle, a sub-portion containing a turn signal indicator of the second vehicle. The method includes processing the sub-portion of the image to determine a state of the turn signal indicator. The method also includes notifying a driver or performing a driving maneuver, at the first vehicle, based on the state of the turn signal indicator.

Claims

1. A method comprising:

detecting, at a first vehicle, a presence of a second vehicle in an adjacent lane;

identifying, in an image of the second vehicle, a sub-portion of the image containing a turn signal indicator of the second vehicle, wherein identifying the sub-portion containing the turn signal indicator comprises processing the image of the second vehicle using a first neural network trained to recognize one or more turn signal regions of interest;

processing the sub-portion of the image to determine a state of the turn signal indicator using a second neural network trained to determine a state of one or more turn signal indicators;

processing the state of the turn signal indicator with a decision matrix to predict a future maneuver of the second vehicle; and

notifying a driver or performing a driving maneuver, at the first vehicle, based at least in part on the predicted future maneuver of the second vehicle.

2. The method of claim 1, wherein processing the state of the turn signal indicator with the decision matrix comprises providing the state of the turn signal indicator to a neural network configured to execute the decision matrix.

3. The method of claim 2, wherein the decision matrix is further configured to predict whether the first vehicle should perform one or more of a lane change or a change in velocity.

4. The method of claim 1, wherein notifying the driver comprises notifying the driver of the predicted future maneuver of the second vehicle.

5. The method of claim 1, wherein processing the state of the turn signal indicator comprises processing to predict whether the second vehicle is likely to make a lane change.

6. The method of claim 1, wherein processing the state of the turn signal indicator comprises processing to predict whether the second vehicle is likely to make a lane change from the adjacent lane into a same lane as the first vehicle.

7. The method of claim 1, wherein identifying the sub-portion of the image containing the turn signal indicator comprises determining a bounding box in the image, wherein the bounding box comprises one or more turn signal indicators.

8. The method of claim 1, wherein processing the sub-portion of the image to determine the state of the turn signal indicator comprises providing the sub-portion of the image to the second neural network, wherein the second neural network is configured to execute a computer vision algorithm to recognize whether the turn signal indicator is in use.

9. A driving control system for a vehicle, the system comprising:

one or more sensors for obtaining sensor data in a region near a first vehicle, the one or more sensors comprising a camera; and

one or more processors configured to execute instructions stored in non-transitory computer readable storage media, the instructions comprising;

detecting, at the first vehicle, a presence of a second vehicle in an adjacent lane;

identifying a sub-portion of an image containing a turn signal indicator of the second vehicle by processing the image of the second vehicle using a first neural network trained to recognize one or more turn signal regions of interest;

processing the sub-portion of the image to determine a state of the turn signal indicator using a second neural network trained to determine a state of one or more turn signal indicators;

processing the state of the turn signal indicator with a decision matrix to predict a future maneuver of the second vehicle; and

providing a notification to a driver or an automated driving system of the first vehicle based at least in part on the predicted future maneuver of the second vehicle.

10. The driving control system of claim 9, wherein the instructions further comprise determining a driving maneuver for the first vehicle based at least in part on the predicted future maneuver of the second vehicle.

11. The driving control system of claim 9, wherein the instructions further comprise determining that the second vehicle is within a risk zone with respect to the first vehicle.

12. The driving control system of claim 9, wherein the instructions further comprise determining a lane of one or more of the first vehicle or the second vehicle.

13. Non-transitory computer readable storage media storing instructions for execution by one or more processors, the instructions comprising:

detecting, at a first vehicle, a presence of a second vehicle in an adjacent lane;

identifying, in an image of the second vehicle, a sub-portion of the image containing a turn signal indicator of the second vehicle, wherein identifying the sub-portion containing the turn signal indicator in the image comprises processing the image using a first neural network trained to recognize one or more turn signal regions of interest;

processing the sub-portion of the image to determine a state of the turn signal indicator using a second neural network trained to determine a state of one or more turn signal indicators;

processing the state of the turn signal indicator with a decision matrix to predict a future maneuver of the second vehicle; and

notifying a driver or an automated driving system, at the first vehicle, of the predicted future maneuver of the second vehicle.

14. The non-transitory computer readable storage media of claim 13, wherein the instructions are such that processing the state of the turn signal indicator comprises processing to predict whether the second vehicle is likely to make a lane change from the adjacent lane into a same lane as the first vehicle.

15. The non-transitory computer readable storage media of claim 13, wherein the instructions further comprise determining whether or not to cause the first vehicle to perform a lane change based at least in part on the turn signal indicator.

16. The non-transitory computer readable storage media of claim 13, wherein the instructions further comprise determining whether the second vehicle is one or more of:

in a lane immediately adjacent a lane of the first vehicle;

driving in a similar direction along a roadway as the first vehicle;

or

within a risk zone of the first vehicle, wherein the risk zone corresponds to an area where there is a risk of collision between the first vehicle and the second vehicle.

17. The non-transitory computer readable storage media of claim 13, wherein the instructions further comprise determining whether the state of the turn signal indicator corresponds with a direction of movement of the first vehicle.