Prediction-based system and method for trajectory planning of autonomous vehicles

• US 10,782,693 B2
• Filed: 11/07/2017
• Issued: 09/22/2020
• Est. Priority Date: 09/07/2017
• Status: Active Grant

• Current Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Original Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Inventors: Zhang, Xiaomin, Chen, Yilun, Li, Guangyu, Sun, Xing, Lin, Wutu, Liu, Liu, Ma, Kai-Chieh, Xuan, Zijie, Zhao, Yufei
• Primary Examiner(s): Kiswanto, Nicholas
• Application Number:
  US15/805,983
• Publication Number:
  US 20190072965A1
• Time in Patent Office: 1,050 Days
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G08G 1/161 Decentralised systems, e.g. inter-vehicle communication
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A prediction-based system and method for trajectory planning of autonomous vehicles are disclosed. A particular embodiment is configured to: receive training data and ground truth data from a training data collection system, the training data including perception data and context data corresponding to human driving behaviors; perform a training phase for training a trajectory prediction module using the training data; receive perception data associated with a host vehicle; and perform an operational phase for extracting host vehicle feature data and proximate vehicle context data from the perception data, using the trained trajectory prediction module to generate predicted trajectories for each of one or more proximate vehicles near the host vehicle, generating a proposed trajectory for the host vehicle, determining if the proposed trajectory for the host vehicle will conflict with any of the predicted trajectories of the proximate vehicles, and modifying the proposed trajectory for the host vehicle until conflicts are eliminated.

Claims

1. A system comprising:

a data processor; and

a prediction-based trajectory planning module, executable by the data processor, the prediction-based trajectory planning module being configured to perform a prediction-based trajectory planning operation for autonomous vehicles, the prediction-based trajectory planning operation being configured to;

receive training data and ground truth data from a training data collection system, the training data including perception data and context data corresponding to human driving behaviors;

perform a training phase to train a trajectory prediction module using the training data;

receive perception data associated with a host vehicle; and

perform an operational phase configured to extract host vehicle feature data and proximate vehicle context data from the perception data, use the trained trajectory prediction module to generate predicted trajectories for each of one or more proximate vehicles near the host vehicle, generate a first proposed trajectory for the host vehicle, determine if the predicted trajectories cause the first proposed trajectory to violate a pre-defined goal, upon determination that the first proposed trajectory violates the pre-defined goal, reject the first proposed trajectory and generate a second proposed trajectory for the host vehicle, use the trained trajectory prediction module to generate new predicted trajectories for each of one or more proximate vehicles based on a current context of the host vehicle, and determine if the new predicted trajectories cause the second proposed trajectory to violate the pre-defined goal.

2. The system of claim 1 wherein a process of generating a proposed trajectory for the host vehicle and a process of determining if predicted trajectories for each of one or more proximate vehicles cause the proposed trajectory to violate the pre-defined goal are repeated until the predicted trajectories cause the proposed trajectory to satisfy the pre-defined goals.

3. The system of claim 1 being further configured to generate coordinate transformations of the perception data relative to the one or more proximate vehicles.

4. The system of claim 1 wherein the training data includes labeling data that includes context information defining directionality and rate behaviors of vehicles represented in the training data.

5. The system of claim 1 wherein the training data includes labeling data that includes context information defining directionality and rate behaviors of vehicles represented in the training data, the context data further defining a left turn, no turn, or a right turn.

6. The system of claim 1 wherein the perception data including data received from a sensor from the group consisting of:

a camera or image capture device, an inertial measurement unit (IMU), a Global Positioning System (GPS) transceiver, a RADAR unit, and a laser range finder/LIDAR unit.

7. The system of claim 1 being further configured to use regression to predict acceleration of a vehicle.

8. The system of claim 1 being further configured to determine if any of the predicted trajectories for the proximate vehicles may cause the host vehicle to violate a pre-defined goal based on a related score being below a minimum acceptable threshold.

9. The system of claim 1 wherein a proposed trajectory for the host vehicle which does not violate the pre-defined goal is output to a vehicle control subsystem causing the host vehicle to follow the output proposed trajectory.

10. A method comprising:

receiving training data and ground truth data from a training data collection system, the training data including perception data and context data corresponding to human driving behaviors;

performing a training phase for training a trajectory prediction module using the training data;

receiving perception data associate with a host vehicle; and

performing an operational phase for extracting host vehicle feature data and proximate vehicle context data from the perception data, using the trained trajectory prediction module to generate predicted trajectories for each of one or more o proximate vehicles near the host vehicle, generating a first proposed trajectory for the host vehicle, determining if the predicted trajectories cause the first proposed trajectory to violate a pre-defined goal, upon determination that the first proposed trajectory violates the pre-defined goal, rejecting the first proposed trajectory and generating a second proposed trajectory for the host vehicle, using the trained trajectory prediction module to generate new predicted trajectories for each of one or more proximate vehicles based on a current context of the host vehicle, and determining if the new predicted trajectories cause the second proposed trajectory to violate the pre-defined goal.

11. The method of claim 10 wherein the proximate vehicle context data includes proximate vehicle position and proximate vehicle velocity.

12. The method of claim 10 including determining a position of each proximate vehicle relative to the host vehicle.

13. The Method of claim 10 including obtaining training data or training images from an array of perception information gathering devices or sensors.

14. The method of claim 10 wherein the training data includes labeling data obtained from human labelers or automated labeling processes.

15. The method of claim 10 wherein the perception data including data received from a sensor from the group consisting of:

a camera or image capture device, an inertial measurement unit (IMU), a Global Positioning System (GPS) transceiver, a RADAR unit, and a laser range finder/LIDAR unit.

16. The method of claim 10 including predicting acceleration of a vehicle.

17. The method of claim 10 including determining if any of the predicted trajectories for the proximate vehicles may cause the host vehicle to violate a pre-defined goal.

18. The method of claim 10 including causing the host vehicle to follow a proposed trajectory for the host vehicle which does not violate the pre-defined goal.

19. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:

receive training data and ground truth data from a training data collection system, the training data including perception data and context data corresponding to human driving behaviors;

perform a training phase for training a trajectory prediction module using the training data;

receive perception data associated with a host vehicle; and

perform an operational phase for extracting host vehicle feature data and proximate vehicle context data from the perception data, using the trained trajectory prediction module to generate predicted trajectories for each of one or more proximate vehicles near the host vehicle, generating a first proposed trajectory for the host vehicle, determining if the predicted trajectories cause the first proposed trajectory to violate a pre-defined goal, upon determination that the first proposed trajectory violates the pre-defined goal, rejecting the first proposed trajectory and generating a second proposed trajectory for the host vehicle, using the trained trajectory prediction module to generate new predicted trajectories for each of one or more proximate vehicles based on a current context of the host vehicle, and determining if the new predicted trajectories cause the second proposed trajectory to violate the pre-defined goal.

20. The non-transitory machine-useable storage medium of claim 19 being configured to generate predicted accelerations for each of the one or more proximate vehicles near the host vehicle.