Data-driven prediction-based system and method for trajectory planning of autonomous vehicles

• US 10,649,458 B2
• Filed: 09/07/2017
• Issued: 05/12/2020
• Est. Priority Date: 09/07/2017
• Status: Active Grant

• Current Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Original Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Inventors: Sun, Xing, Lin, Wutu, Liu, Liu, Ma, Kai-Chieh, Xuan, Zijie, Zhao, Yufei
• Primary Examiner(s): Kiswanto, Nicholas
• Application Number:
  US15/698,607
• Publication Number:
  US 20190072973A1
• Time in Patent Office: 978 Days
• CPC Class Codes: B62D 15/025 Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation B62D1/28 takes precedence
• CPC Class Codes: B62D 15/0255 Automatic changing of lane, e.g. for passing another vehicle
• CPC Class Codes: B62D 15/026 combined with automatic distance control, i.e. electronic tow bar
• CPC Class Codes: B62D 15/0265 Automatic obstacle avoidance by steering
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0221 involving a learning process
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

A data-driven prediction-based system and method for trajectory planning of autonomous vehicles are disclosed. A particular embodiment includes: generating a first suggested trajectory for an autonomous vehicle; generating predicted resulting trajectories of proximate agents using a prediction module; scoring the first suggested trajectory based on the predicted resulting trajectories of the proximate agents; generating a second suggested trajectory for the autonomous vehicle and generating corresponding predicted resulting trajectories of proximate agents, if the score of the first suggested trajectory is below a minimum acceptable threshold; and outputting a suggested trajectory for the autonomous vehicle wherein the score corresponding to the suggested trajectory is at or above the minimum acceptable threshold.

Claims

1. A system comprising:

an electronic data processor;

one or more vehicle sensor subsystems to generate input object data related to objects detected in proximity to an autonomous vehicle, the input object data including speed and trajectory data corresponding to proximate agents, the proximate agents being vehicles in the vicinity of the autonomous vehicle and detectable by the vehicle sensor subsystems;

a vehicle control subsystem to control the autonomous vehicle; and

a trajectory planning module, executable by the data processor, the trajectory planning module being configured to perform a data-driven prediction-based trajectory planning operation for autonomous vehicles, the trajectory planning operation being configured to;

generate a first suggested trajectory for the autonomous vehicle, the first suggested trajectory configured to comply with pre-defined goals for the autonomous vehicle;

generate a first distribution of predicted resulting trajectories for each of the proximate agents using a prediction module, the first distributions of predicted resulting trajectories being based on the input object data corresponding to the proximate agents and the first suggested trajectory, the predicted resulting trajectories of each of the first distributions having an associated confidence level;

score the first suggested trajectory based on the first distributions of predicted resulting trajectories of the proximate agents, the score corresponding to a level to which the first suggested trajectory complies with the pre-defined goals;

generate a second suggested trajectory for the autonomous vehicle to comply with the pre-defined goals and generate a corresponding second distribution of predicted resulting trajectories for each of the proximate agents using the prediction module, if the score of the first suggested trajectory is below a minimum acceptable threshold, the second distributions of predicted resulting trajectories being based on the input object data corresponding to the proximate agents and the second suggested trajectory, the predicted resulting trajectories of each of the second distributions having an associated confidence level;

output a suggested trajectory for the autonomous vehicle wherein the score corresponding to the suggested trajectory is at or above the minimum acceptable threshold; and

cause the vehicle control subsystem to manipulate the autonomous vehicle to follow the output suggested trajectory.

2. The system of claim 1 wherein the data-driven prediction-based trajectory planning operation includes machine learnable components.

3. The system of claim 1 wherein the output suggested trajectory is dependent upon the predicted resulting trajectories of proximate agents.

4. The system of claim 1 wherein the data-driven prediction-based trajectory planning operation is configured to repeatedly generate distributions of predicted resulting trajectories and their confidence values, until the score corresponding to the suggested trajectory is at or above the minimum acceptable threshold or until a time period or iteration count is exceeded.

5. The system of claim 1 wherein the prediction module is configured to retain information corresponding to predicted trajectories of proximate agents as mathematical or data representations.

6. A method comprising:

using one or more vehicle sensor subsystems to generate input object data related to objects detected in proximity to an autonomous vehicle, the input object data including speed and trajectory data corresponding to proximate agents, the proximate agents being vehicles in the vicinity of the autonomous vehicle and detectable by the vehicle sensor subsystems;

generating, by use of an electronic data processor, a first suggested trajectory for the autonomous vehicle, the first suggested trajectory configured to comply with pre-defined goals for the autonomous vehicle;

generating, by use of the electronic data processor, a first distribution of predicted resulting trajectories for each of the proximate agents using a prediction module, the first distributions of predicted resulting trajectories being based on the input object data corresponding to the proximate agents and the first suggested trajectory, the predicted resulting trajectories of each of the first distributions having an associated confidence level;

scoring, by use of the electronic data processor, the first suggested trajectory based on the first distributions of predicted resulting trajectories of the proximate agents, the score corresponding to a level to which the first suggested trajectory complies with the pre-defined goals;

generating, by use of the electronic data processor, a second suggested trajectory for the autonomous vehicle to comply with the pre-defined goals and generating a corresponding second distribution of predicted resulting trajectories for each of the proximate agents using the prediction module, if the score of the first suggested trajectory is below a minimum acceptable threshold, the second distributions of predicted resulting trajectories being based on the input object data corresponding to the proximate agents and the second suggested trajectory, the predicted resulting trajectories of each of the second distributions having an associated confidence level;

outputting, by use of the electronic data processor, a suggested trajectory for the autonomous vehicle wherein the score corresponding to the suggested trajectory is at or above the minimum acceptable threshold; and

causing a vehicle control subsystem to manipulate the autonomous vehicle to follow the output suggested trajectory.

7. The method of claim 6 including causing a machine to implement a learnable prediction module.

8. The method of claim 6 wherein the output suggested trajectory is dependent upon the predicted resulting trajectories of proximate agents.

9. The method of claim 6 including repeatedly generating distributions of predicted resulting trajectories and their confidence values, until the score corresponding to the suggested trajectory is at or above the minimum acceptable threshold or until a time period or iteration count is exceeded.

10. The method of claim 6 including causing the prediction module to retain information corresponding to predicted trajectories of proximate agents as mathematical or data representations.

11. A non-transitory machine-useable storage medium embodying instructions which, when executed by an electronic data processor, cause the electronic data processor to:

use one or more vehicle sensor subsystems to generate input object data related to objects detected in proximity to an autonomous vehicle, the input object data including speed and trajectory data corresponding to proximate agents, the proximate agents being vehicles in the vicinity of the autonomous vehicle and detectable by the vehicle sensor subsystems;

generate a first suggested trajectory for the autonomous vehicle, the first suggested trajectory configured to comply with pre-defined goals for the autonomous vehicle;

generate a first distribution of predicted resulting trajectories for each of the proximate agents using a prediction module, the first distributions of predicted resulting trajectories being based on the input object data corresponding to the proximate agents and the first suggested trajectory, the predicted resulting trajectories of each of the first distributions having an associated confidence level;

score the first suggested trajectory based on the first distributions of predicted resulting trajectories of the proximate agents, the score corresponding to a level to which the first suggested trajectory complies with the pre-defined goals;

generate a second suggested trajectory for the autonomous vehicle to comply with the pre-defined goals and generate a corresponding second distribution of predicted resulting trajectories for each of the proximate agents, if the score of the first suggested trajectory is below a minimum acceptable threshold, the second distributions of predicted resulting trajectories being based on the input object data corresponding to the proximate agents and the second suggested trajectory, the predicted resulting trajectories of each of the second distributions having an associated confidence level;

output a suggested trajectory for the autonomous vehicle wherein the score corresponding to the suggested trajectory is at or above the minimum acceptable threshold; and

cause a vehicle control subsystem to manipulate the autonomous vehicle to follow the output suggested trajectory.

12. The non-transitory machine-useable storage medium of claim 11 wherein the instructions are further configured to enable machine learning.

13. The non-transitory machine-useable storage medium of claim 11 wherein the output suggested trajectory is dependent upon the predicted resulting trajectories of proximate agents.

14. The non-transitory machine-useable storage medium of claim 11 wherein the instructions are further configured to repeatedly generate distributions of predicted resulting trajectories and their confidence values, until the score corresponding to the suggested trajectory is at or above the minimum acceptable threshold or until a time period or iteration count is exceeded.

15. The non-transitory machine-useable storage medium of claim 11 wherein the prediction module is configured to retain information corresponding to predicted trajectories of proximate agents as mathematical or data representations.