System and method for using human driving patterns to manage speed control for autonomous vehicles

• US 10,656,644 B2
• Filed: 09/07/2017
• Issued: 05/19/2020
• Est. Priority Date: 09/07/2017
• Status: Active Grant

• Current Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Original Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Inventors: Lin, Wutu, Liu, Liu, Sun, Xing, Ma, Kai-Chieh, Xuan, Zijie, Zhao, Yufei
• Primary Examiner(s): Jha, Abdhesh K
• Application Number:
  US15/698,375
• Publication Number:
  US 20190072960A1
• Time in Patent Office: 985 Days
• Field of Search: None
• CPC Class Codes: B60W 2720/103 Speed profile
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00

Abstract

A system and method for using human driving patterns to manage speed control for autonomous vehicles are disclosed. A particular embodiment includes: generating data corresponding to desired human driving behaviors; training a human driving model module using a reinforcement learning process and the desired human driving behaviors; receiving a proposed vehicle speed control command; determining if the proposed vehicle speed control command conforms to the desired human driving behaviors by use of the human driving model module; and validating or modifying the proposed vehicle speed control command based on the determination.

Claims

1. A system comprising:

a data processor; and

a vehicle speed control module, executable by the data processor, the vehicle speed control module being configured to perform a vehicle speed control command validation operation for an autonomous vehicle, the vehicle speed control command validation operation being configured to;

generate data corresponding to desired human driving behaviors;

train a human driving model module using a reinforcement learning process and the desired human driving behaviors;

receive a proposed vehicle speed control command prior to commanding a vehicle control subsystem to perform a maneuver corresponding to the proposed vehicle speed control command;

determine if the proposed vehicle speed control command conforms to the desired human driving behaviors by use of the human driving model module;

validate or modify the proposed vehicle speed control command based on the determination; and

output the validated or modified vehicle speed control command to the vehicle control subsystem causing the autonomous vehicle to follow a trajectory corresponding to the validated or modified vehicle speed control command.

2. The system of claim 1 being further configured to train the human driving model module using the reinforcement learning process having a simulation training phase and an actual on-the-road training phase.

3. The system of claim 1 being further configured to train the human driving model module by modifying parameters in the human driving model module based on the reinforcement learning process.

4. The system of claim 1 being further configured to train the human driving model module by determining a current state of the autonomous vehicle and determining a deviation between the current state of the autonomous vehicle and a state corresponding to the desired human driving behaviors.

5. The system of claim 1 being further configured to capture data through vehicle sensor subsystems and driving simulation data to model the desired human driving behaviors.

6. The system of claim 1 wherein the data corresponding to desired human driving behaviors are represented as mathematical or data representations.

7. A method comprising:

generating data corresponding to desired human driving behaviors;

training a human driving model module using a reinforcement learning process and the desired human driving behaviors;

receiving a proposed vehicle speed control command prior to commanding a vehicle control subsystem to perform a maneuver corresponding to the proposed vehicle speed control command;

determining if the proposed vehicle speed control command conforms to the desired human driving behaviors by use of the human driving model module;

validating or modifying the proposed vehicle speed control command based on the determination; and

outputting the validated or modified vehicle speed control command to the vehicle control subsystem causing the autonomous vehicle to follow a trajectory corresponding to the validated or modified vehicle speed control command.

8. The method of claim 7 including training the human driving model module using the reinforcement learning process having a simulation training phase and an actual on-the-road training phase.

9. The method of claim 7 including training the human driving model module by modifying parameters in the human driving model module based on the reinforcement learning process.

10. The method of claim 7 including training the human driving model module by determining a current state of the autonomous vehicle and determining a deviation between the current state of the autonomous vehicle and a state corresponding to the desired human driving behaviors.

11. The method of claim 7 including capturing data through vehicle sensor subsystems and driving simulation data to model the desired human driving behaviors.

12. The method of claim 7 wherein the data corresponding to desired human driving behaviors are represented as mathematical or data representations.

13. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:

generate data corresponding to desired human driving behaviors;

train a human driving model module using a reinforcement learning process and the desired human driving behaviors;

receive a proposed vehicle speed control command prior to commanding a vehicle control subsystem to perform a maneuver corresponding to the proposed vehicle speed control command;

determine if the proposed vehicle speed control command conforms to the desired human driving behaviors by use of the human driving model module;

validate or modify the proposed vehicle speed control command based on the determination; and

output the validated or modified vehicle speed control command to the vehicle control subsystem causing the autonomous vehicle to follow a trajectory corresponding to the validated or modified vehicle speed control command.

14. The non-transitory machine-useable storage medium of claim 13 being further configured to train the human driving model module using the reinforcement learning process having a simulation training phase and an actual on-the-road training phase.

15. The non-transitory machine-useable storage medium of claim 13 being further configured to train the human driving model module by modifying parameters in the human driving model module based on the reinforcement learning process.

16. The non-transitory machine-useable storage medium of claim 13 being further configured to train the human driving model module by determining a current state of the autonomous vehicle and determining a deviation between the current state of the autonomous vehicle and a state corresponding to the desired human driving behaviors.