System and method for providing multiple agents for decision making, trajectory planning, and control for autonomous vehicles

• US 10,768,626 B2
• Filed: 09/30/2017
• Issued: 09/08/2020
• Est. Priority Date: 09/30/2017
• Status: Active Grant

• Current Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Original Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Inventors: Sun, Xing, Zhao, Yufei, Lin, Wutu, Xuan, Zijie, Liu, Liu, Ma, Kai-Chieh
• Primary Examiner(s): Butler, Rodney A
• Application Number:
  US15/721,781
• Publication Number:
  US 20190101923A1
• Time in Patent Office: 1,074 Days
• CPC Class Codes: B25J 9/1664 characterised by motion, path, trajectory planning
• CPC Class Codes: B60W 2050/0006 Digital architecture hierarchy
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G06F 9/00 Arrangements for program control, e.g. control units program control for peripheral devices G06F13/10
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues

Abstract

A system and method for providing multiple agents for decision making, trajectory planning, and control for autonomous vehicles are disclosed. A particular embodiment includes: partitioning a multiple agent autonomous vehicle control module for an autonomous vehicle into a plurality of subsystem agents, the plurality of subsystem agents including a deep computing vehicle control subsystem and a fast response vehicle control subsystem; receiving a task request from a vehicle subsystem; dispatching the task request to the deep computing vehicle control subsystem or the fast response vehicle control subsystem based on content of the task request or a context of the autonomous vehicle; causing execution of the deep computing vehicle control subsystem or the fast response vehicle control subsystem by use of a data processor to produce a vehicle control output; and providing the vehicle control output to a vehicle control subsystem of the autonomous vehicle.

Claims

1. A system comprising:

a data processor; and

a memory for storing a multiple agent autonomous vehicle control module, executable by the data processor, the multiple agent autonomous vehicle control module being a data processing module configured to perform an autonomous vehicle control operation for an autonomous vehicle, the multiple agent autonomous vehicle control module being partitioned into a plurality of subsystem agents, the plurality of subsystem agents including a deep computing vehicle control subsystem and a fast response vehicle control subsystem, the fast response vehicle control subsystem configured to preempt the deep computing vehicle control subsystem if the fast response vehicle control subsystem and the deep computing vehicle control subsystem are executed by the same data processor, the autonomous vehicle control operation being configured to;

receive a task request from a vehicle subsystem;

determine, by use of the data processor, if the task request is appropriate for the deep computing vehicle control subsystem or the fast response vehicle control subsystem based on content of the task request or a context of the autonomous vehicle;

dispatch, by use of the data processor, the task request to the deep computing vehicle control subsystem or the fast response vehicle control subsystem based on the determination;

cause execution of the deep computing vehicle control subsystem or the fast response vehicle control subsystem by use of the data processor to produce a vehicle control output; and

provide the vehicle control output to a vehicle control subsystem of the autonomous vehicle.

2. The system of claim 1 being further configured to receive sensor data from a plurality of sensors on the autonomous vehicle.

3. The system of claim 1 wherein the execution of the deep computing vehicle control subsystem and the fast response vehicle control subsystem by use of the data processor is performed in parallel.

4. The system of claim 1 wherein the execution of the deep computing vehicle control subsystem and the fast response vehicle control subsystem is performed by different data processors.

5. The system of claim 1 wherein the multiple agent autonomous vehicle control module being further partitioned into a plurality of subsystems, the plurality of subsystems including a decision making subsystem, a trajectory generation subsystem, and a vehicle control subsystem.

6. The system of claim 1 wherein the multiple agent autonomous vehicle control module being further partitioned into a plurality of subsystems, the plurality of subsystems including a decision making subsystem, a trajectory generation subsystem, and a vehicle control subsystem, the execution of the decision making subsystem, the trajectory generation subsystem, and the vehicle control subsystem by use of the data processor is performed in parallel.

7. The system of claim 1 wherein the vehicle control output is provided to a vehicle control subsystem causing the autonomous vehicle to follow a routing and perform vehicle motion control operations corresponding to the vehicle control output.

8. A method comprising:

partitioning a multiple agent autonomous vehicle control module for an autonomous vehicle into a plurality of subsystem agents, the plurality of subsystem agents including a deep computing vehicle control subsystem and a fast response vehicle control subsystem, the multiple agent autonomous vehicle control module being a data processing module storable in a memory and executable by a data processor, the fast response vehicle control subsystem configured to preempt the deep computing vehicle control subsystem if the fast response vehicle control subsystem and the deep computing vehicle control subsystem are executed by the same data processor;

receiving a task request from a vehicle subsystem;

determining, by use of the data processor, if the task request is appropriate for the deep computing vehicle control subsystem or the fast response vehicle control subsystem based on content of the task request or a context of the autonomous vehicle;

dispatching, by use of the data processor, the task request to the deep computing vehicle control subsystem or the fast response vehicle control subsystem based on the determination;

causing execution of the deep computing vehicle control subsystem or the fast response vehicle control subsystem by use of the data processor to produce a vehicle control output; and

providing the vehicle control output to a vehicle control subsystem of the autonomous vehicle.

9. The method of claim 8 including receiving sensor data from a plurality of sensors on the autonomous vehicle, the sensors being from the group consisting of:

an inertial measurement unit (IMU), a Global Positioning System (GPS) transceiver, a RADAR unit, a laser range finder/LIDAR unit, and one or more cameras or image capture devices.

10. The method of claim 8 wherein the execution of the deep computing vehicle control subsystem and the fast response vehicle control subsystem is performed in parallel by different data processors.

11. The method of claim 8 wherein the completion of a task by execution of the deep computing vehicle control subsystem occurs more slowly than the completion of a task by execution of the fast response vehicle control subsystem.

12. The method of claim 8 wherein the execution of the deep computing vehicle control subsystem and the fast response vehicle control subsystem is performed by the same data processor.

13. The method of claim 8 including partitioning the multiple agent autonomous vehicle control module into a plurality of subsystems, the plurality of subsystems including a decision making subsystem, and a trajectory generation subsystem, the execution of the decision making subsystem, and the trajectory generation subsystem by use of the data processor is performed in parallel.

14. The method of claim 8 wherein the vehicle control output is provided to a vehicle control subsystem causing the autonomous vehicle to follow a routing and perform vehicle motion control operations corresponding to the vehicle control output, the vehicle control subsystem including a steering and throttle control subsystem.

15. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:

partition a multiple agent autonomous vehicle control module for an autonomous vehicle into a plurality of subsystem agents, the plurality of subsystem agents including a deep computing vehicle control subsystem and a fast response vehicle control subsystem, the multiple agent autonomous vehicle control module being a data processing module storable in a memory and executable by a data processor, the fast response vehicle control subsystem configured to preempt the deep computing vehicle control subsystem if the fast response vehicle control subsystem and the deep computing vehicle control subsystem are executed by the same data processor;

receive a task request from a vehicle subsystem;

determine if the task request is appropriate for the deep computing vehicle control subsystem or the fast response vehicle control subsystem based on content of the task request or a context of the autonomous vehicle;

dispatch the task request to the deep computing vehicle control subsystem or the fast response vehicle control subsystem based on the determination;

cause execution of the deep computing vehicle control subsystem or the fast response vehicle control subsystem by use of the data processor to produce a vehicle control output; and

provide the vehicle control output to a vehicle control subsystem of the autonomous vehicle.

16. The non-transitory machine-useable storage medium of claim 15 including receiving sensor data from a plurality of sensors on the autonomous vehicle, the sensors including one or more cameras or image capture devices.

17. The non-transitory machine-useable storage medium of claim 15 wherein the execution of the deep computing vehicle control subsystem and the fast response vehicle control subsystem is performed by the same data processor.

18. The non-transitory machine-useable storage medium of claim 15 wherein the plurality of subsystem agents including a deep computing decision making subsystem and a fast response decision making subsystem.

19. The non-transitory machine-useable storage medium of claim 15 including partitioning the multiple agent autonomous vehicle control module into a plurality of subsystems, the plurality of subsystems including a decision making subsystem, a trajectory generation subsystem, and a vehicle control subsystem.

20. The non-transitory machine-useable storage medium of claim 15 wherein the vehicle control output is provided to a vehicle control subsystem causing the autonomous vehicle to follow a routing and perform vehicle motion control operations corresponding to the vehicle control output, the vehicle control subsystem including a braking and navigation control subsystem.