Trajectory generation and execution architecture

• US 10,353,390 B2
• Filed: 06/23/2017
• Issued: 07/16/2019
• Est. Priority Date: 03/01/2017
• Status: Active Grant

• Current Assignee: Zoox Inc. (Amazon.com, Inc.)
• Original Assignee: Zoox Inc. (Amazon.com, Inc.)
• Inventors: Linscott, Gary, Somers, Robert Edward, Kiiski, Joona Markus Petteri, Kobilarov, Marin, Caldwell, Timothy, Askeland, Jacob Lee, Rege, Ashutosh Gajanan, Funke, Joseph
• Primary Examiner(s): Nguyen, Nga X
• Application Number:
  US15/632,208
• Publication Number:
  US 20180251126A1
• Time in Patent Office: 753 Days
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/18154 Approaching an intersection
• CPC Class Codes: B60W 60/0011 involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles
• CPC Class Codes: B60W 60/0016 of the vehicle or its occupants
• CPC Class Codes: B60W 60/00276 for two or more other traffic participants
• CPC Class Codes: G01C 21/3407 specially adapted for specific applications
• CPC Class Codes: G05D 1/0055 with safety arrangements
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G05D 1/246 using environment maps, e.g. simultaneous localisation and mapping [SLAM]
• CPC Class Codes: G05D 1/617 Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D1/693; arrangements for reacting to or preventing system or operator failure G05D1/80
• CPC Class Codes: G05D 1/644 Optimisation of travel parameters, e.g. of energy consumption, journey time or distance
• CPC Class Codes: G05D 1/646 Following a predefined trajectory, e.g. a line marked on the floor or a flight path
• CPC Class Codes: G06N 3/04 : Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/08 : Learning methods

Abstract

Techniques for generating and executing trajectories to guide autonomous vehicles are described. In an example, a first computer system associated with an autonomous vehicle can generate, at a first operational frequency, a route to guide the autonomous vehicle from a current location to a target location. The first computer system can further determine, at a second operational frequency, an instruction for guiding the autonomous vehicle along the route and can generate, at a third operational frequency, a trajectory based at least partly on the instruction and real-time processed sensor data. A second computer system that is associated with the autonomous vehicle and is in communication with the first computer system can execute, at a fourth operational frequency, the trajectory to cause the autonomous vehicle to travel along the route. The separation of the first computer system and the second computer system can provide enhanced safety, redundancy, and optimization.

Claims

1. A system embodied in an autonomous vehicle, the system comprising:

a first computer system comprising;

one or more first processors; and

one or more first computer readable storage media communicatively coupled to the one or more first processors and storing one or more first modules including instructions that are executable by the one or more first processors, the one or more first modules comprising;

a route planning module configured to generate a route between a current location of the autonomous vehicle and a target location of the autonomous vehicle, the route planning module operating at a first operational frequency;

a decision module configured to generate an instruction based on the route, the instruction including a sequence of actions to guide the autonomous vehicle along at least a portion of the route, the decision module operating at a second operational frequency; and

a trajectory module configured to;

access real-time processed sensor data; and

generate an output trajectory based at least in part on the instruction and the real-time processed sensor data, the trajectory module operating at a third operational frequency; and

a second computer system comprising;

one or more second processors; and

one or more second computer readable storage media communicatively coupled to the one or more second processors and storing one or more second modules including instructions that are executable by the one or more second processors, the one or more second modules comprising an execution module configured to generate one or more signals for causing the autonomous vehicle to drive along the output trajectory, the execution module operating at a fourth operational frequency.

2. The system of claim 1, wherein the one or more second modules include a fallback determination module, the fallback determination module configured to:

receive, from the first computer system, a fallback trajectory for causing the autonomous vehicle perform a safety maneuver; and

store the fallback trajectory for a predetermined period of time,

wherein the fallback determination module operates at the fourth operational frequency.

3. The system of claim 2, wherein the fallback determination module is further configured to:

determine an occurrence of an event warranting a fallback action; and

access the fallback trajectory.

4. The system of claim 1, wherein the one or more second modules include a fallback determination module, the fallback determination module configured to:

determine an occurrence of an event warranting a fallback action; and

generate a fallback trajectory for causing the autonomous vehicle to perform a safety maneuver,

wherein the fallback determination module operates at the fourth operational frequency.

5. The system of claim 1, wherein the execution module is further configured to:

receive a fallback trajectory for causing the autonomous vehicle to perform a safety maneuver; and

execute the fallback trajectory to cause the autonomous vehicle to perform the safety maneuver.

6. The system of claim 1, wherein the fourth operational frequency is higher than the first operational frequency, the second operational frequency, and the third operational frequency.

7. The system of claim 1, wherein:

the third operational frequency is higher than the first operational frequency and the second operational frequency; and

the third operational frequency is lower than the fourth operational frequency.

8. The system of claim 1, wherein the first operational frequency is lower than the second operational frequency, the third operational frequency, and the fourth operational frequency.

9. A method implemented by a transportable computer system, the method comprising:

determining a current location of the transportable computer system;

determining a target location for the transportable computer system;

receiving, from one or more sensors associated with the transportable computer system, real-time processed sensor data;

determining, at a first operational frequency, a route to guide the transportable computer system from the current location to the target location;

determining, at a second operational frequency, an instruction for guiding the transportable computer system along at least a portion of the route based at least in part on the real-time processed sensor data;

generating, at a third operational frequency, a trajectory based at least in part on the real-time processed sensor data and the instruction; and

executing, at a fourth operational frequency, the trajectory to cause the transportable computer system to travel along the route.

10. The method of claim 9, wherein:

the determining the route is performed by a first computer system associated with the transportable computer system;

the determining the instruction for guiding the transportable computer system along at least the portion of the route is performed by the first computer system; and

the generating the trajectory is performed by the first computer system.

11. The method of claim 10, wherein the executing the trajectory to cause the transportable computer system to travel along the route is performed by a second computer system associated with the transportable computer system, the second computer system being different than the first computer system.

12. The method of claim 9, wherein the instruction comprises a policy and generating the trajectory based at least in part on the real-time processed sensor data comprises generating the trajectory based on the policy.

13. The method of claim 9, further comprising:

generating, at the second operational frequency, a fallback instruction for causing the transportable computer system to perform a fallback action;

generating, at the third operational frequency, a fallback trajectory based on the fallback instruction; and

outputting the fallback trajectory with the trajectory at a substantially same time.

14. The method of claim 9, further comprising:

determining an occurrence of an event warranting a fallback action;

generating, at the fourth operational frequency, a fallback trajectory for causing the transportable computer system to perform the fallback action; and

executing the fallback trajectory to cause the transportable computer system to perform the fallback action.

15. A system comprising:

one or more processors; and

one or more computer readable storage media communicatively coupled to the one or more processors and storing one or more modules including instructions that are executable by the one or more processors, the one or more first modules comprising;

a route planning module configured to generate a route between a current location of the system and a target location of the system, the route planning module operating at a first operational frequency;

a decision module configured to generate an instruction based on the route, the instruction including a sequence of actions to guide the system along the route, the decision module operating at a second operational frequency;

a trajectory module configured to;

access real-time processed sensor data; and

generate an output trajectory based at least in part on the instruction and the real-time processed sensor data, the trajectory module operating at a third operational frequency; and

an execution module configured to generate one or more signals for causing the system to move along the output trajectory, the execution module operating at a fourth operational frequency.

16. The system of claim 15, wherein the route planning module, the decision module, and the trajectory module are associated with a first computer system of the system and the execution module is associated with a second computer system of the system that is separate from the first computer system.

17. The system of claim 16, the one or more modules further comprising a fallback determination module associated with the second computer system, the fallback determination module configured to:

determine an occurrence of an event warranting a fallback action;

access a fallback trajectory for causing the system to perform the fallback action; and

cause the fallback trajectory to be executed to cause the system to perform the fallback action.

18. The system of claim 17, wherein the event comprises:

an object is within a threshold distance of the system;

an object is predicted to be within a threshold distance of the system;

an absence of communication between the first computer system and the second computer system;

a confusion level associated with the system is above a confusion threshold;

a failure associated with a sensor of the system;

a malfunction of a component associated with the system;

a change in an occupancy status associated with the system;

a change in a status of a drivable surface associated with the trajectory;

or

a probability of a collision exceeds a threshold probability.

19. The system of claim 15, wherein:

the first operational frequency is less than the second operational frequency;

the second operational frequency is less than the third operational frequency; and

the third operational frequency is less than the fourth operational frequency.

20. The system of claim 15, wherein the decision module is further configured to:

access an executed trajectory;

compare the executed trajectory with the output trajectory;

generate feedback data based at least in part on comparing the executed trajectory and the output trajectory; and

updating at one or more of a model or an algorithm based at least in part on the feedback data.