Prediction-based system and method for trajectory planning of autonomous vehicles

• US 10,782,694 B2
• Filed: 11/07/2017
• Issued: 09/22/2020
• Est. Priority Date: 09/07/2017
• Status: Active Grant

• Current Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Original Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Inventors: Zhang, Xiaomin, Chen, Yilun, Li, Guangyu, Sun, Xing, Lin, Wutu, Liu, Liu, Ma, Kai-Chieh, Xuan, Zijie, Zhao, Yufei
• Primary Examiner(s): Kiswanto, Nicholas
• Application Number:
  US15/806,013
• Publication Number:
  US 20190072966A1
• Time in Patent Office: 1,050 Days
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G05D 1/228 Command input arrangements located on-board unmanned vehicles
• CPC Class Codes: G05D 1/646 Following a predefined trajectory, e.g. a line marked on the floor or a flight path
• CPC Class Codes: G08G 1/161 Decentralised systems, e.g. inter-vehicle communication
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A prediction-based system and method for trajectory planning of autonomous vehicles are disclosed. A particular embodiment is configured to: receive training data and ground truth data from a training data collection system, the training data including perception data and context data corresponding to human driving behaviors; perform a training phase for training a trajectory prediction module using the training data; receive perception data associated with a host vehicle; and perform an operational phase for extracting host vehicle feature data and proximate vehicle context data from the perception data, generating a proposed trajectory for the host vehicle, using the trained trajectory prediction module to generate predicted trajectories for each of one or more proximate vehicles near the host vehicle based on the proposed host vehicle trajectory, determining if the proposed trajectory for the host vehicle will conflict with any of the predicted trajectories of the proximate vehicles, and modifying the proposed trajectory for the host vehicle until conflicts are eliminated.

Claims

1. A system comprising:

a data processor; and

a prediction-based trajectory planning module, executable by the data processor, the prediction-based trajectory planning module being configured to perform a prediction-based trajectory planning operation for autonomous vehicles, the prediction-based trajectory planning operation being configured to;

receive training data and ground truth data from a training data collection system, the training data including perception data and context data corresponding to human driving behaviors;

perform a training phase to train a trajectory prediction module using the training data;

receive perception data associated with a host vehicle; and

perform an operational phase configured to extract host vehicle feature data and proximate vehicle context data from the perception data, generate a proposed trajectory for the host vehicle, use the trained trajectory prediction module to generate predicted trajectories for each of one or more proximate vehicles near the host vehicle, the predicted trajectories for each of one or more proximate vehicles being in reaction to the proposed host vehicle trajectory, determine if the proposed trajectory for the host vehicle will conflict with any of the predicted trajectories of the proximate vehicles, and modify the proposed trajectory for the host vehicle based on the determined conflicts until the conflicts are eliminated.

2. The system of claim 1 being further configured to filter and smooth the perception data.

3. The system of claim 1 being further configured to generate coordinate transformations of the perception data relative to the one or more proximate vehicles.

4. The system of claim 1 wherein the training data includes labeling data that includes context information defining directionality and rate behaviors of vehicles represented in the training data.

5. The system of claim 1 wherein the training data includes labeling data that includes context information defining directionality and rate behaviors of vehicles represented in the training data, the context data further defining a left turn, no turn, or a right turn.

6. The system of claim 1 wherein the perception data including data received from a sensor from the group consisting of:

a camera or image capture device, an inertial measurement unit (IMU), a Global Positioning System (GPS) transceiver, a RADAR unit, and a laser range finder/LIDAR unit.

7. The system of claim 1 being further configured to use regression to predict acceleration of a vehicle.

8. The system of claim 1 being further configured to determine if any of the predicted trajectories for the proximate vehicles may cause the host vehicle to violate a pre-defined goal based on a related score being below a minimum acceptable threshold.

9. The system of claim 1 wherein the proposed trajectory for the host vehicle is output to a vehicle control subsystem causing the host vehicle to follow the output proposed trajectory.

10. A method comprising:

receiving training data and ground truth data from a training data collection system, the training data including perception data and context data corresponding to human driving behaviors;

performing a training phase for training a trajectory prediction module using the training data;

receiving perception data associated with a host vehicle; and

performing an operational phase for extracting host vehicle feature data and proximate vehicle context data from the perception data, generating a proposed trajectory for the host vehicle, using the trained trajectory prediction module to generate predicted trajectories for each of one or more proximate vehicles near the host vehicle, the predicted trajectories for each of one or more proximate vehicles being in reaction to the proposed host vehicle trajectory, determining if the proposed trajectory for the host vehicle will conflict with any of the predicted trajectories of the proximate vehicles, and modifying the proposed trajectory for the host vehicle based on the determined conflicts until the conflicts are eliminated.

11. The method of claim 10 wherein the proximate vehicle context data includes proximate vehicle position and proximate vehicle velocity.

12. The method of claim 10 including determining a position of each proximate vehicle relative to the host vehicle.

13. The method of claim 10 including obtaining training data or training images from an array of perception information gathering devices or sensors.

14. The method of claim 10 wherein the training data includes labeling data obtained from human labelers or automated labeling processes.

15. The method of claim 10 wherein the perception data including data received from a sensor from the group consisting of:

a camera or image capture device, a Global Positioning System (GPS) transceiver, and a laser range finder/LIDAR unit.

16. The method of claim 10 including predicting acceleration of a vehicle.

17. The method of claim 10 including determining if any of the predicted trajectories for the proximate vehicles may cause the host vehicle to violate a pre-defined goal.

18. The method of claim 10 including causing the host vehicle to follow the output proposed trajectory.

19. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:

receive training data and ground truth data from a training data collection system, the training data including perception data and context data corresponding to human driving behaviors;

perform a training phase for training a trajectory prediction module using the training data;

receive perception data associated with a host vehicle; and

perform an operational phase for extracting host vehicle feature data and proximate vehicle context data from the perception data, generating a proposed trajectory for the host vehicle, using the trained trajectory prediction module to generate predicted trajectories for each of one or more proximate vehicles near the host vehicle, the predicted trajectories for each of one or more proximate vehicles being in reaction to the proposed host vehicle trajectory, determining if the proposed trajectory for the host vehicle will conflict with any of the predicted trajectories of the proximate vehicles, and modifying the proposed trajectory for the host vehicle based on the determined conflicts until the conflicts are eliminated.

20. The non-transitory machine-useable storage medium of claim 19 being configured to generate predicted accelerations for each of the one or more proximate vehicles near the host vehicle.