Predicting trajectories of objects based on contextual information

• US 10,421,453 B1
• Filed: 11/03/2017
• Issued: 09/24/2019
• Est. Priority Date: 10/02/2014
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Ferguson, David Ian Franklin, Silver, David Harrison, Ross, Stephane, Fairfield, Nathaniel, Sucan, Ioan-Alexandru
• Primary Examiner(s): Dunn, Alex C
• Application Number:
  US15/802,820
• Time in Patent Office: 690 Days
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/4044 Direction of movement, e.g. backwards
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2556/00 Input parameters relating to data
• CPC Class Codes: B60W 2720/24 Direction of travel
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0953 the prediction being responsive to vehicle dynamic parameters
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 30/18154 Approaching an intersection
• CPC Class Codes: B60W 50/0097 Predicting future conditions
• CPC Class Codes: B60W 60/0027 using trajectory prediction for other traffic participants
• CPC Class Codes: B60W 60/00274 considering possible movement changes
• CPC Class Codes: B60W 60/00276 for two or more other traffic participants

Abstract

Aspects of the disclosure relate to detecting and responding to objects in a vehicle'"'"'s environment. For example, an object may be identified in a vehicle'"'"'s environment, the object having a heading and location. A set of possible actions for the object may be generated using map information describing the vehicle'"'"'s environment and the heading and location of the object. A set of possible future trajectories of the object may be generated based on the set of possible actions. A likelihood value of each trajectory of the set of possible future trajectories may be determined based on contextual information including a status of the detected object. A final future trajectory is determined based on the determined likelihood value for each trajectory of the set of possible future trajectories. The vehicle is then maneuvered in order to avoid the final future trajectory and the object.

Claims

1. A computer-implemented method for maneuvering a vehicle, the method comprising:

receiving, by one or more computing devices, sensor data identifying a first object in a vehicle'"'"'s environment;

generating, by the one or more computing devices, a set of possible future trajectories of the first object based on the sensor data for the first object;

identifying a first plurality of waypoints based on each trajectory in the set of possible future trajectories, wherein a waypoint includes a position and a timestamp corresponding to a future time;

determining a trajectory of the vehicle, wherein the trajectory of the vehicle includes a second plurality of waypoints;

comparing the first plurality of waypoints to the second plurality of waypoints; and

maneuvering, by the one or more computing devices, the vehicle in order to avoid the first object based on the comparison.

2. The method of claim 1, wherein each given possible future trajectory of the set of possible future trajectories is associated with a likelihood value corresponding to a likelihood that the first object will follow the given possible future trajectory, and the method further comprises determining whether any likelihood values meet a threshold value, and wherein the comparing is performed only when none of the likelihood values meet the threshold value.

3. The method of claim 1, wherein comparing the first plurality of waypoints to the second plurality of waypoints include comparing positions and timestamps of the first plurality of waypoints to positions and timestamps of the second plurality of waypoints.

4. The method of claim 1, wherein each waypoint of the first plurality of waypoints further defines a projected future velocity for the first object, and wherein the comparing is further based on the projected future velocities of the waypoints of the first plurality of waypoints.

5. The method of claim 1, wherein the comparing further includes determining whether any waypoints of the first plurality of waypoints are within a predetermined distance of any of the waypoints of the second plurality of waypoints.

6. The method of claim 5, wherein the comparing further includes determining whether any waypoints of the first plurality of waypoints are within a predetermined distance of any of the waypoints of the second plurality of waypoints at a given point in time.

7. The method of claim 5, wherein the predetermined distance is no more than 6 meters.

8. The method of claim 5, wherein each given possible future trajectory of the set of possible future trajectories is associated with a likelihood value corresponding to a likelihood that the first object will follow the given possible future trajectory, and the method further comprises:

determining a final future trajectory based on the likelihood values for any trajectories of the set of possible future trajectories with corresponding waypoints of the first plurality of waypoints that are within the predetermined distance of any waypoints of the second plurality of waypoints, and

wherein maneuvering the vehicle is further based on the final future trajectory.

9. The method of claim 8, wherein determining the final future trajectory includes:

summing likelihood values for any trajectories of the set of possible future trajectories with corresponding waypoints of the first plurality of waypoints that are within the predetermined distance of any waypoints of the second plurality of waypoints; and

determining whether the sum meets a predetermine threshold value.

10. The method of claim 9, wherein when the sum is determined to meet the predetermined each of the trajectories of the set of possible future trajectories with corresponding waypoints of the first plurality of waypoints that are within the predetermined distance of any waypoints of the second plurality of waypoints is included in the final future trajectory.

11. The method of claim 5, further comprising determining a final future trajectory based on any trajectories of the set of possible future trajectories with corresponding waypoints of the first plurality of waypoints that are within the predetermined distance of any waypoints of the second plurality of waypoints is included in the final future trajectory, and wherein the maneuvering is further based on the final future trajectory.

12. The method of claim 5, further comprising determining a final future trajectory based on any waypoints of the first plurality of waypoints that are within the predetermined distance of any waypoints of the second plurality of waypoints.

13. A system for maneuvering a vehicle, the system comprising one or more computing devices configured to:

receive sensor data identifying a first object in a vehicle'"'"'s environment;

generate a set of possible future trajectories of the first object based on the sensor data for the first object;

identify a first plurality of waypoints based on each trajectory in the set of possible future trajectories, wherein a waypoint includes a position and a timestamp corresponding to a future time;

determine a trajectory of the vehicle, wherein the trajectory of the vehicle includes a second plurality of waypoints;

compare the first plurality of waypoints to the second plurality of waypoints; and

maneuver the vehicle in order to avoid the first object based on the comparison.

14. The system of claim 13, wherein each given possible future trajectory of the set of possible future trajectories is associated with a likelihood value corresponding to a likelihood that the first object will follow the given possible future trajectory, and the one or more computing devices are further configured to determine whether any likelihood values meet a threshold value, and wherein the comparing is performed only when none of the likelihood values meet the threshold value.

15. The system of claim 13, wherein the comparing further includes determining whether any waypoints of the first plurality of waypoints are within a predetermined distance of any of the waypoints of the second plurality of waypoints.

16. The system of claim 15, wherein each given possible future trajectory of the set of possible future trajectories is associated with a likelihood value corresponding to a likelihood that the first object will follow the given possible future trajectory, and the one or more computing devices are further configured to:

determine a final future trajectory based on the likelihood values for any trajectories of the set of possible future trajectories with corresponding waypoints of the first plurality of waypoints that are within the predetermined distance of any waypoints of the second plurality of waypoints, and

wherein maneuvering the vehicle is further based on the final future trajectory.

17. The system of claim 16, wherein determining the final future trajectory includes:

summing likelihood values for any trajectories of the set of possible future trajectories with corresponding waypoints of the first plurality of waypoints that are within the predetermined distance of any waypoints of the second plurality of waypoints; and

determining whether the sum meets a predetermine threshold value.

18. The system of claim 17, wherein when the sum is determined to meet the predetermined each of the trajectories of the set of possible future trajectories with corresponding waypoints of the first plurality of waypoints that are within the predetermined distance of any waypoints of the second plurality of waypoints is included in the final future trajectory.

19. The system of claim 15, further comprising determining a final future trajectory based on any trajectories of the set of possible future trajectories with corresponding waypoints of the first plurality of waypoints that are within the predetermined distance of any waypoints of the second plurality of waypoints is included in the final future trajectory, and wherein the maneuvering is further based on the final future trajectory.

20. The system of claim 13, further comprising the vehicle.