Inferring state of traffic signal and other aspects of a vehicle's environment based on surrogate data

• US 9,327,734 B2
• Filed: 06/18/2014
• Issued: 05/03/2016
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Lombrozo, Peter, Teller, Eric, Templeton, Bradley
• Primary Examiner(s): Zanelli, Michael J
• Application Number:
  US14/308,409
• Publication Number:
  US 20140303829A1
• Time in Patent Office: 685 Days
• US Class Current: 1/1
• CPC Class Codes: B60W 2050/0075 Automatic parameter input, automatic initialising or calibrating means
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/406 Traffic density
• CPC Class Codes: B60W 2556/10 Historical data
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/16 Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
• CPC Class Codes: B60W 30/18159 Traversing an intersection
• CPC Class Codes: B60W 40/00 Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
• CPC Class Codes: B60W 60/001 Planning or execution of driving tasks
• CPC Class Codes: B60W 60/0017 of other traffic participants
• CPC Class Codes: B60W 60/0027 using trajectory prediction for other traffic participants
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: G01C 21/30 Map- or contour-matching
• CPC Class Codes: G01S 13/88 Radar or analogous systems specially adapted for specific applications electromagnetic prospecting or detecting of objects, e.g. near-field detection, G01V3/00
• CPC Class Codes: G01S 15/88 Sonar systems specially adapted for specific applications seismic or acoustic prospecting or detecting G01V1/00
• CPC Class Codes: G01S 17/88 Lidar systems specially adapted for specific applications
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights

Abstract

A vehicle configured to operate in an autonomous mode can obtain sensor data from one or more sensors observing one or more aspects of an environment of the vehicle. At least one aspect of the environment of the vehicle that is not observed by the one or more sensors could be inferred based on the sensor data. The vehicle could be controlled in the autonomous mode based on the at least one inferred aspect of the environment of the vehicle.

Claims

1. A method, comprising:

obtaining sensor data from one or more sensors observing an environment of a vehicle, wherein the one or more sensors are operationally associated with the vehicle, wherein the vehicle is configured to operate in an autonomous mode, wherein the environment includes a traffic signal that is not observed by the one or more sensors, wherein the sensor data comprises vehicle-related sensor data obtained from the one or more sensors observing at least one other vehicle in the environment, and wherein the vehicle-related sensor data indicates a state of a brake light of the at least one other vehicle;

using an inference system to infer, based on the sensor data, a state of the traffic signal; and

controlling the vehicle in the autonomous mode based on the inferred state of the traffic signal.

2. The method of claim 1, wherein the traffic signal is obscured by a vehicle.

3. The method of claim 1, wherein obtaining sensor data comprises using at least one of a camera, a radar system, an acoustic-sensing system, an ultrasonic-sensing system, and a lidar system.

4. The method of claim 1, wherein the vehicle-related sensor data indicates that the brake light of the at least one other vehicle is on, and wherein the inference system infers that the state of traffic signal is red based at least in part on the brake light of the at least one other vehicle being on.

5. The method of claim 4, wherein controlling the vehicle in the autonomous mode based on the inferred state of the traffic signal comprises controlling the vehicle to stop.

6. The method of claim 1, wherein the vehicle-related sensor data indicates that the brake light of the at least one other vehicle is off, and wherein the inference system infers that the state of traffic signal is green based at least in part on the brake light of the at least one other vehicle being off.

7. The method of claim 6, wherein controlling the vehicle in the autonomous mode based on the inferred state of the traffic signal comprises controlling the vehicle to proceed.

8. The method of claim 1, wherein using an inference system comprises using at least one of a Bayesian network, a hidden Markov model, or a decision tree.

9. A vehicle, comprising:

one or more sensors, wherein the one or more sensors are configured to acquire sensor data, wherein the sensor data relates to an environment of a vehicle observed by the one or more sensors, wherein the vehicle is configured to operate in an autonomous mode, wherein the sensor data comprises vehicle-related sensor data obtained from the one or more sensors observing at least one other vehicle in the environment, and wherein the vehicle-related sensor data indicates a state of a brake light of the at least one other vehicle;

an inference system, wherein the inference system is configured to infer, based on the sensor data, a state of a traffic signal that is not observed by the one or more sensors, and wherein the inference is configured to infer the state of the traffic light based at least in part on the state of the brake light of the at least one other vehicle; and

a control system, wherein the control system is configured to control the vehicle in the autonomous mode based on the inferred state of the traffic signal.

10. The vehicle of claim 9, wherein the one or more sensors comprise at least one of a camera, a radar system, an acoustic-sensing system, an ultrasonic-sensing system, and a lidar system.

11. The vehicle of claim 9, wherein the inference system is configured to use at least one of a Bayesian network, a hidden Markov model, or a decision tree.

12. The method of claim 9, wherein the control system is configured to control the vehicle to stop based on the inferred state of the traffic light being red, and wherein the control system is configured to control the vehicle to proceed based on the inferred state of the traffic light being green.

13. A non-transitory computer readable medium having stored therein instructions executable by a computer system to cause the computer system to perform functions, the functions comprising:

obtaining sensor data from one or more sensors observing an environment of a vehicle, wherein the one or more sensors are operationally associated with the vehicle, wherein the vehicle is configured to operate in an autonomous mode, wherein the environment includes a traffic signal that is not observed by the one or more sensors, wherein the sensor data comprises vehicle-related sensor data obtained from the one or more sensors observing at least one other vehicle in the environment, and wherein the vehicle-related sensor data indicates a state of a brake light of the at least one other vehicle;

inferring, based on the sensor data, a state of the traffic signal; and

controlling the vehicle in the autonomous mode based on the inferred state of the traffic signal.

14. The non-transitory computer readable medium of claim 13, wherein the vehicle-related sensor data indicates that the brake light of the at least one other vehicle is on, wherein the inferred state of traffic signal is red based at least in part on the brake light of the at least one other vehicle being on, and wherein controlling the vehicle in the autonomous mode based on the inferred state of the traffic signal comprises controlling the vehicle to stop.

15. The non-transitory computer readable medium of claim 13, wherein the vehicle-related sensor data indicates that the brake light of the at least one other vehicle is off, wherein the inference system infers that the state of traffic signal is green based at least in part on the brake light of the at least one other vehicle being off, and wherein controlling the vehicle in the autonomous mode based on the inferred state of the traffic signal comprises controlling the vehicle to proceed.