Inferring state of traffic signal and other aspects of a vehicle's environment based on surrogate data

• US 9,804,601 B2
• Filed: 03/31/2016
• Issued: 10/31/2017
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Lombrozo, Peter, Teller, Eric, Templeton, Bradley
• Primary Examiner(s): Zanelli, Michael J
• Application Number:
  US15/087,390
• Publication Number:
  US 20160209844A1
• Time in Patent Office: 579 Days
• CPC Class Codes: B60W 2050/0075 Automatic parameter input, automatic initialising or calibrating means
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/406 Traffic density
• CPC Class Codes: B60W 2556/10 Historical data
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/16 Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
• CPC Class Codes: B60W 30/18159 Traversing an intersection
• CPC Class Codes: B60W 40/00 Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
• CPC Class Codes: B60W 60/001 Planning or execution of driving tasks
• CPC Class Codes: B60W 60/0017 of other traffic participants
• CPC Class Codes: B60W 60/0027 using trajectory prediction for other traffic participants
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: G01C 21/30 Map- or contour-matching
• CPC Class Codes: G01S 13/88 Radar or analogous systems specially adapted for specific applications electromagnetic prospecting or detecting of objects, e.g. near-field detection, G01V3/00
• CPC Class Codes: G01S 15/88 Sonar systems specially adapted for specific applications seismic or acoustic prospecting or detecting G01V1/00
• CPC Class Codes: G01S 17/88 Lidar systems specially adapted for specific applications
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights

Abstract

A vehicle configured to operate in an autonomous mode can obtain sensor data from one or more sensors observing one or more aspects of an environment of the vehicle. At least one aspect of the environment of the vehicle that is not observed by the one or more sensors could be inferred based on the sensor data. The vehicle could be controlled in the autonomous mode based on the at least one inferred aspect of the environment of the vehicle.

Claims

1. A method, comprising:

obtaining sensor data from one or more sensors observing an environment of a vehicle, wherein the one or more sensors are operationally associated with the vehicle, wherein the vehicle is configured to operate in an autonomous mode, wherein the environment includes a traffic signal that is not observed by the one or more sensors, and wherein the sensor data indicates a status of at least one other vehicle in the environment;

using an inference system to infer a state of the traffic signal based on the status of the at least one other vehicle in the environment; and

controlling the vehicle in the autonomous mode based on the inferred state of the traffic signal.

2. The method of claim 1, wherein the status of the at least one other vehicle in the environment comprises at least one vehicle that is stopped ahead of the vehicle, and wherein the inference system infers that the state of the traffic signal is red based at least in part on the at least one vehicle that is stopped ahead of the vehicle.

3. The method of claim 1, wherein the traffic signal is at an intersection of a first roadway and a second roadway, and wherein the vehicle is on the first roadway.

4. The method of claim 3, wherein the status of the at least one other vehicle in the environment comprises at least one vehicle on the first roadway that is stopped at the intersection, and wherein the inference system infers that the state of the traffic signal is red based at least in part on the at least one vehicle on the first roadway that is stopped at the intersection.

5. The method of claim 3, wherein the status of the at least one other vehicle in the environment comprises at least one vehicle on the second roadway proceeding through the intersection, and wherein the inference system infers that the state of the traffic signal is red based at least in part on the at least one vehicle on the second roadway proceeding through the intersection.

6. The method of claim 3, wherein controlling the vehicle in the autonomous mode based on the inferred state of the traffic signal comprises controlling the vehicle to stop at the intersection.

7. The method of claim 3, wherein controlling the vehicle in the autonomous mode based on the inferred state of the traffic signal comprises controlling the vehicle to proceed through the intersection.

8. The method of claim 1, further comprising:

obtaining subsequent sensor data from the one or more sensors, wherein the subsequent sensor data indicates a subsequent status of the at least one other vehicle in the environment;

using the inference system to make a subsequent inference regarding the state of the traffic signal based on the subsequent status of the at least one other vehicle in the environment; and

controlling the vehicle in the autonomous mode based on the subsequent inference regarding the state of the traffic signal.

9. The method of claim 1, wherein obtaining sensor data comprises using at least one of a camera, a radar system, or a lidar system.

10. The method of claim 1, wherein the status of the at least one other vehicle comprises a state of a brake light of the at least one other vehicle.

11. A vehicle, comprising:

one or more sensors, wherein the one or more sensors are configured to acquire sensor data regarding an environment of the vehicle, wherein the vehicle is configured to operate in an autonomous mode, wherein the environment includes a traffic signal that is not observed by the one or more sensors, and wherein the sensor data indicates a status of at least one other vehicle in the environment;

an inference system, wherein the inference system is configured to infer a state of a traffic signal based on the status of the at least one other vehicle in the environment; and

a control system, wherein the control system is configured to control the vehicle in the autonomous mode based on the inferred state of the traffic signal.

12. The vehicle of claim 11, wherein the one or more sensors comprise at least one of a camera, a radar system, or a lidar system.

13. The vehicle of claim 11, wherein the traffic signal is at an intersection of a first roadway and a second roadway, and wherein the vehicle is on the first roadway.

14. The vehicle of claim 13, wherein the status of the at least one other vehicle in the environment comprises at least one vehicle that is stopped on the first roadway at the intersection, and wherein the inference system infers that the state of the traffic signal is red based at least in part on the at least one vehicle that is stopped on the first roadway at the intersection.

15. The vehicle of claim 13, wherein the status of the at least one other vehicle in the environment comprises at least one vehicle on the second roadway proceeding through the intersection, and wherein the inference system infers that the state of the traffic signal is red based at least in part on the at least one vehicle on the second roadway proceeding through the intersection.

16. A non-transitory computer readable medium having stored therein instructions executable by a computer system to cause the computer system to perform functions, the functions comprising:

obtaining sensor data from one or more sensors observing an environment of a vehicle, wherein the one or more sensors are operationally associated with the vehicle, wherein the vehicle is configured to operate in an autonomous mode, wherein the environment includes a traffic signal that is not observed by the one or more sensors, and wherein the sensor data indicates a status of at least one other vehicle in the environment;

using an inference system to infer a state of the traffic signal based on the status of the at least one other vehicle in the environment; and

controlling the vehicle in the autonomous mode based on the inferred state of the traffic signal.

17. The non-transitory computer readable medium of claim 16, wherein the status of the at least one other vehicle in the environment comprises at least one vehicle that is stopped ahead of the vehicle, and wherein the inference system infers that the state of the traffic signal is red based at least in part on the at least one vehicle that is stopped ahead of the vehicle.

18. The non-transitory computer readable medium of claim 16, wherein the traffic signal is at an intersection of a first roadway and a second roadway, and wherein the vehicle is on the first roadway.

19. The non-transitory computer readable medium of claim 18, wherein the status of the at least one other vehicle in the environment comprises at least one vehicle on the first roadway that is stopped at the intersection, and wherein the inference system infers that the state of the traffic signal is red based at least in part on the at least one vehicle on the first roadway that is stopped at the intersection.

20. The non-transitory computer readable medium of claim 18, wherein the status of the at least one other vehicle in the environment comprises at least one vehicle on the second roadway proceeding through the intersection, and wherein the inference system infers that the state of the traffic signal is red based at least in part on the at least one vehicle on the second roadway proceeding through the intersection.