Predicting trajectories of objects based on contextual information

• US 9,248,834 B1
• Filed: 10/02/2014
• Issued: 02/02/2016
• Est. Priority Date: 10/02/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ferguson, David Ian Franklin, Fairfield, Nathaniel, Silver, David Harrison, Ross, Stphane, Sucan, Ioan-Alexandru
• Primary Examiner(s): Tarcza, Thomas
• Assistant Examiner(s): Dunn, Alex C
• Application Number:
  US14/505,007
• Time in Patent Office: 488 Days
• Field of Search: 701/301
• US Class Current: 1/1
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/4044 Direction of movement, e.g. backwards
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2556/00 Input parameters relating to data
• CPC Class Codes: B60W 2720/24 Direction of travel
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0953 the prediction being responsive to vehicle dynamic parameters
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 30/18154 Approaching an intersection
• CPC Class Codes: B60W 50/0097 Predicting future conditions
• CPC Class Codes: B60W 60/0027 using trajectory prediction for other traffic participants
• CPC Class Codes: B60W 60/00274 considering possible movement changes
• CPC Class Codes: B60W 60/00276 for two or more other traffic participants

Abstract

Aspects of the disclosure relate to detecting and responding to objects in a vehicle'"'"'s environment. For example, an object may be identified in a vehicle'"'"'s environment, the object having a heading and location. A set of possible actions for the object may be generated using map information describing the vehicle'"'"'s environment and the heading and location of the object. A set of possible future trajectories of the object may be generated based on the set of possible actions. A likelihood value of each trajectory of the set of possible future trajectories may be determined based on contextual information including a status of the detected object. A final future trajectory is determined based on the determined likelihood value for each trajectory of the set of possible future trajectories. The vehicle is then maneuvered in order to avoid the final future trajectory and the object.

Claims

1. A computer-implemented method comprising:

identifying, by one or more computing devices, an object in a vehicle'"'"'s environment, the object having a heading and location;

generating, by the one or more computing devices, a set of possible actions for the object using map information describing the vehicle'"'"'s environment and the heading and location of the object;

generating, by the one or more computing devices, a set of possible future trajectories of the object based on the set of possible actions;

receiving, by the one or more computing devices, contextual information including a status of the detected object;

determining, by the one or more computing devices, a likelihood value of each trajectory of the set of possible future trajectories based on the contextual information;

determining, by the one or more computing devices, a final future trajectory based on the determined likelihood value for each trajectory of the set of possible future trajectories; and

maneuvering, by the one or more computing devices, the vehicle in order to avoid the final future trajectory and the object.

2. The method of claim 1, wherein determining the final future trajectory includes:

comparing the likelihood value for each trajectory of the set of possible future trajectories to a threshold value; and

discarding a trajectory from the set of trajectories when the likelihood value of that trajectory does not meet the threshold value, wherein the likelihood value of the discarded trajectory is not used to determine the final future trajectory.

3. The method of claim 2, wherein determining the final future trajectory includes:

when none of the trajectories of the set of possible future trajectories meet the threshold value, identifying a plurality of waypoints for each trajectory in the set of trajectories, wherein a waypoint includes at least one of a position, a velocity, and a timestamp;

determining a trajectory of the vehicle, wherein the trajectory of the vehicle includes a plurality of waypoints; and

comparing, at a same timestamp, each of the waypoints to a waypoint associated with a trajectory of the vehicle in order to determine the final future trajectory.

4. The method of claim 2, wherein determining the final future trajectory includes:

identifying a situational relationship between the object and the vehicle;

comparing the likelihood value of the trajectories remaining in the set of possible future trajectories to a second threshold different from the first threshold value, and

discarding a second trajectory from the trajectories remaining in the set of possible future trajectories when the likelihood value of that second trajectory does not meet the second threshold value, wherein the likelihood value of the discarded second trajectory is not used to determine the final future trajectory.

5. The method of claim 4, wherein after discarding the second trajectory, the remaining trajectories of the set of possible future trajectories are each identified as final future trajectories, such that maneuvering the vehicle includes avoiding each of the remaining trajectories of the set of possible future trajectories.

6. The method of claim 1, wherein determining the final future trajectory includes selecting a trajectory of the set of possible future trajectories with a highest likelihood value as the final future trajectory.

7. The method of claim 1, wherein generating the set of possible actions includes discarding an action from the set of possible actions for failing to comply with a model of possible actions for the object.

8. The method of claim 1, wherein generating the set of possible actions is further based on a past trajectory of the object.

9. The method of claim 1, wherein the contextual information further describes a status of a second object in the vehicle'"'"'s environment.

10. A system comprising one or more computing devices configured to:

identify an object in a vehicle'"'"'s environment, the object having a heading and location;

generate a set of possible actions for the object using map information describing the vehicle'"'"'s environment and the heading and location of the object;

generate a set of possible future trajectories of the object based on the set of possible actions;

receive contextual information including a status of the detected object;

determine a likelihood value of each trajectory of the set of possible future trajectories based on the contextual information;

determine a final future trajectory based on the determined likelihood value for each trajectory of the set of possible future trajectories; and

maneuver the vehicle in order to avoid the final future trajectory and the object.

11. The system of claim 10, wherein the one or more computing devices are further configured to determine the final future trajectory by:

comparing the likelihood value for each trajectory of the set of possible future trajectories to a threshold value; and

discarding a trajectory from the set of trajectories when the likelihood value of that trajectory does not meet the threshold value, wherein the likelihood value of the discarded trajectory is not used to determine the final future trajectory.

12. The system of claim 11, wherein the one or more computing devices are further configured to determine the final future trajectory by:

when none of the trajectories of the set of possible future trajectories meet the threshold value, identifying a plurality of waypoints for each trajectory in the set of trajectories, wherein a waypoint includes at least one of a position, a velocity, and a timestamp;

determining a trajectory of the vehicle, wherein the trajectory of the vehicle includes a plurality of waypoints; and

comparing, at a same timestamp, each of the waypoints for each trajectory to a waypoint of the trajectory of the vehicle to determine the final future trajectory.

13. The system of claim 11, wherein the one or more computing devices are further configured to determine the final future trajectory by:

identifying a situational relationship between the object and the vehicle;

comparing the likelihood value of the trajectories remaining in the set of possible future trajectories to a second threshold different from the first threshold value, and

discarding a second trajectory from the trajectories remaining in the set of possible future trajectories when the likelihood value of that second trajectory does not meet the second threshold value, wherein the likelihood value of the discarded second trajectory is not used to determine the final future trajectory.

14. The system of claim 13, wherein the one or more computing devices are further configured to, after discarding the second trajectory, identify the remaining trajectories of the set of possible future trajectories as each being final future trajectories, such that maneuvering the vehicle includes avoiding each of the remaining trajectories of the set of possible future trajectories.

15. The system of claim 10, wherein the one or more computing devices are further configured to determine the final future trajectory by selecting a trajectory of the set of possible future trajectories with a highest likelihood value as the final future trajectory.

16. The system of claim 10, wherein the one or more computing devices are further configured to determine the final future trajectory by discarding an action from the set of possible actions for failing to comply with a model of possible actions for the object.

17. The system of claim 10, wherein the one or more computing devices are further configured to generate the set of possible actions further based on a past trajectory of the object.

18. The system of claim 10, wherein the contextual information further describes a status of a second object in the vehicle'"'"'s environment.

19. The system of claim 10, wherein the vehicle is an autonomous vehicle.

20. A non-transitory computer-readable medium on which instructions are stored, the instructions, when executed by one or more processors cause the one or more processors to perform a method, the method comprising:

identifying an object in a vehicle'"'"'s environment, the object having a heading and location;

generating a set of possible actions for the object using map information describing the vehicle'"'"'s environment and the heading and location of the object;

generating a set of possible future trajectories of the object based on the set of possible actions;

receiving contextual information including a status of the detected object;

determining a likelihood value of each trajectory of the set of possible future trajectories based on the contextual information;

determining a final future trajectory based on the determined likelihood value for each trajectory of the set of possible future trajectories; and

maneuvering the vehicle in order to avoid the final future trajectory and the object.