Robust Method for Detecting Traffic Signals and their Associated States

• US 20130253754A1
• Filed: 03/26/2012
• Published: 09/26/2013
• Est. Priority Date: 03/26/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ferguson, David I., Fairfield, Nathaniel, Levandowski, Anthony
• Granted Patent: US 9,145,140 B2
• Time in Patent Office: Days
• US Class Current: 701/28
• CPC Class Codes: B60W 2555/60 Traffic rules, e.g. speed limits or right of way
• CPC Class Codes: B60W 2556/20 Data confidence level
• CPC Class Codes: B60W 2556/50 of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 30/18154 Approaching an intersection
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0231 using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights

Abstract

Methods and devices for detecting traffic signals and their associated states are disclosed. In one embodiment, an example method includes a scanning a target area using one or more sensors of a vehicle to obtain target area information. The vehicle may be configured to operate in an autonomous mode, and the target area may be a type of area where traffic signals are typically located. The method may also include detecting a traffic signal in the target area information, determining a location of the traffic signal, and determining a state of the traffic signal. Also, a confidence in the traffic signal may be determined. For example, the location of the traffic signal may be compared to known locations of traffic signals. Based on the state of the traffic signal and the confidence in the traffic signal, the vehicle may be controlled in the autonomous mode.

Claims

1. A method comprising:

scanning a target area using one or more sensors of a vehicle to obtain target area information, wherein the vehicle is configured to operate in an autonomous mode and the target area is a type of area where traffic signals are typically located;

detecting a traffic signal in the target area information;

determining a location of the traffic signal;

determining a state of the traffic signal;

determining a confidence in the traffic signal, wherein determining the confidence in the traffic signal comprises comparing the location of the traffic signal to one or more known locations of traffic signals; and

controlling the vehicle in the autonomous mode based on the state of the traffic signal and the confidence in the traffic signal.

2. The method of claim 1, wherein the target area is in front of the vehicle and includes a range of heights that are expected for traffic signals.

3. The method of claim 1, wherein the target area is proximate to a traffic intersection.

4. The method of claim 1, wherein comparing the location of the traffic signal to one or more known locations of traffic signals comprises:

determining that the location of the traffic signal does not correspond to the one or more known locations of traffic signals.

5. The method of claim 1, wherein the target area information comprises an image obtained by one or more cameras, wherein detecting a traffic signal in the target area information comprises identifying in the image a pattern of red, yellow, and green objects that is expected for traffic signals.

6. The method of claim 1, wherein the target area information comprises a three-dimensional point cloud, and wherein detecting a traffic signal in the target area information comprises identifying an object in the three-dimensional point cloud having a shape that is expected for traffic signals.

7. The method of claim 1, wherein the target area information comprises geographic location and orientation information, and wherein the location of the traffic signal is determined based on the geographic location and orientation information.

8. The method of claim 7, wherein the location of the traffic signal is a three-dimensional location, and the method further comprising:

storing the three-dimensional location of the traffic signal in a memory.

9. The method of claim 1, wherein the one or more known locations of traffic signals are determined based on a map of three-dimensional locations of traffic signals and a geographic location of the vehicle.

10. A non-transitory computer readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

scanning a target area using one or more sensors of a vehicle to obtain target area information, wherein the vehicle is configured to be operated in an autonomous mode and the target area includes a type of area where traffic signals are typically located;

detecting a traffic signal in the target area information;

determining a location of the traffic signal;

determining a state of the traffic signal;

determining a confidence in the traffic signal, wherein determining the confidence in the traffic signal comprises comparing the location of the traffic signal to one or more known locations of traffic signals; and

controlling the vehicle in the autonomous mode based on the state of the traffic signal and the confidence in the traffic signal.

11. The non-transitory computer readable medium of claim 10, wherein the target area is in front of the vehicle and includes a range of heights that are expected for traffic signals.

12. The non-transitory computer readable medium of claim 10, wherein the target area is proximate to a traffic intersection.

13. The non-transitory computer readable medium of claim 10, wherein the target area information comprises an image obtained by one or more cameras, wherein detecting a traffic signal in the target area information comprises identifying in the image a pattern of red, yellow, and green objects that is expected for traffic signals.

14. The non-transitory computer readable medium of claim 10, further comprising instructions executable by the computing device to cause the computing device to perform functions comprising:

storing the location of the traffic signal in a memory, wherein the location of the traffic signal is a three-dimensional location.

15. The non-transitory computer readable medium of claim 10, wherein the one or more known locations of traffic signals are determined based on a map of three-dimensional locations of traffic signals and a geographic location of the vehicle.

16. A vehicle configured to be operated in autonomous mode, comprising:

one or more sensors;

a memory;

a processor; and

instructions stored in the memory and executable by the processor to;

cause the one or more sensors to scan a target area to obtain target area information, wherein the target area includes a type of area where traffic signals are typically located;

detect a traffic signal in the target area information;

determine a location of the traffic signal;

determine a state of the traffic signal;

determine a confidence in the traffic signal, wherein determining the confidence in the traffic signal comprises comparing the location of the traffic signal to one or more known locations of traffic signals; and

control the vehicle in the autonomous mode based on the state of the traffic signal and the confidence in the traffic signal.

17. The vehicle of claim 16, wherein the target area is in front of the vehicle and includes a range of heights that are expected for traffic signals.

18. The vehicle of claim 16, wherein the target area information comprises an image obtained by one or more cameras, wherein detecting a traffic signal in the target area information comprises identifying in the image a pattern of red, yellow, and green objects that is expected for traffic signals.

19. The vehicle of claim 16, further comprising instructions stored in the memory and executable by the processor to:

store the location of the traffic signal in the memory, wherein the location of the traffic signal is a three-dimensional location.

20. The vehicle of claim 16, wherein the one or more known locations of traffic signals are determined based on a map of three-dimensional locations of traffic signals and a geographic location of the vehicle.