Systems and methods to control autonomous vehicle motion

• US 10,345,808 B2
• Filed: 03/30/2017
• Issued: 07/09/2019
• Est. Priority Date: 03/30/2017
• Status: Active Grant

• Current Assignee: Uatc, LLC (Uber Technologies, Inc.)
• Original Assignee: Uber Technologies, Inc.
• Inventors: Wilkinson, Eric Lloyd, Panikulam, Jacob, Bagnell, J. Andrew
• Primary Examiner(s): Cheung, Calvin
• Application Number:
  US15/473,686
• Publication Number:
  US 20180284768A1
• Time in Patent Office: 831 Days
• CPC Class Codes: B62D 15/00 Steering not otherwise provided for
• CPC Class Codes: G01C 21/38 Electronic maps specially adapted for navigation; Updating thereof
• CPC Class Codes: G05D 1/0066 for limitation of acceleration or stress
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0217 in accordance with energy consumption, time reduction or distance reduction criteria
• CPC Class Codes: G05D 1/0221 involving a learning process
• CPC Class Codes: G05D 1/228 Command input arrangements located on-board unmanned vehicles
• CPC Class Codes: G05D 1/617 Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D1/693; arrangements for reacting to or preventing system or operator failure G05D1/80
• CPC Class Codes: G05D 1/644 Optimisation of travel parameters, e.g. of energy consumption, journey time or distance
• CPC Class Codes: G05D 1/65 Following a desired speed profile

Abstract

The present disclosure provides systems and methods that control the motion of an autonomous vehicle by rewarding or otherwise encouraging progress toward a goal, rather than simply rewarding distance travelled. In particular, the systems and methods of the present disclosure can project a candidate motion plan that describes a proposed motion path for the autonomous vehicle onto a nominal pathway to determine a projected distance associated with the candidate motion plan. The systems and methods of the present disclosure can use the projected distance to evaluate a reward function that provides a reward that is positively correlated to the magnitude of the projected distance. The motion of the vehicle can be controlled based on the reward value provided by the reward function. For example, the candidate motion plan can be selected for implementation or revised based at least in part on the determined reward value.

Claims

1. A computer-implemented method to control motion of an autonomous vehicle, the method comprising:

obtaining, by one or more computing devices, a candidate motion plan that describes a proposed motion path for the autonomous vehicle;

identifying, by the one or more computing devices, a nominal pathway associated with the autonomous vehicle;

projecting, by the one or more computing devices, the candidate motion plan onto the nominal pathway to determine a projected distance for the candidate motion plan; and

controlling, by the one or more computing devices, motion of the autonomous vehicle based at least in part on the projected distance of the candidate motion plan;

wherein controlling, by the one or more computing devices, the motion of the autonomous vehicle based at least in part on the projected distance comprises;

evaluating, by the one or more computing devices, a reward function with the projected distance of the candidate motion plan to determine a reward amount for the candidate motion plan; and

determining, by the one or more computing devices, whether to select the candidate motion plan for implementation based at least in part on the reward amount for the candidate motion plan;

wherein determining, by the one or more computing devices, whether to select the candidate motion plan for implementation based at least in part on the reward amount for the candidate motion plan comprises iteratively searching, by the one or more computing devices, over a motion planning space to optimize a total cost, the total cost comprising a sum of one or more cost functions and the reward function.

2. The computer-implemented method of claim 1, wherein identifying, by the one or more computing devices, the nominal pathway associated with the autonomous vehicle comprises identifying, by the one or more computing devices, the nominal pathway that corresponds to a center line of one or more lanes in which the autonomous vehicle intends to travel.

3. The computer-implemented method of claim 1, wherein controlling, by the one or more computing devices, motion of the autonomous vehicle based at least in part on the projected distance comprises providing, by the one or more computing devices, a reward that is positively correlated to the projected distance of the candidate motion plan.

4. The computer-implemented method of claim 1, wherein projecting, by the one or more computing devices, the candidate motion plan onto the nominal pathway to determine the projected distance for the candidate motion plan comprises:

identifying, by the one or more computing devices, a plurality of nominal pathway locations of the nominal pathway, wherein a plurality of pathway distances are respectively associated with the plurality of nominal pathway locations;

identifying, by the one or more computing devices, a plurality of motion plan locations of the candidate motion plan; and

determining, by the one or more computing devices, the projected distance for the candidate motion plan based at least in part on the plurality of pathway distances and the plurality of motion plan locations.

5. The computer-implemented method of claim 4, wherein determining, by the one or more computing devices, the projected distance for the candidate motion plan comprises:

determining, by the one or more computing devices, a projected sub-distance for each of the plurality of motion plan locations; and

determining, by the one or more computing devices, the projected distance for the candidate motion plan based at least in part on the plurality of projected sub-distances for the plurality of motion plan locations.

6. The computer-implemented method of claim 5, wherein determining, by the one or more computing devices, the projected sub-distance for each motion plan location comprises:

determining, by the one or more computing devices, a plurality of L2 distances from the motion plan location respectively to the plurality of nominal pathway locations;

determining, by the one or more computing devices, a plurality of weights respectively based at least in part on the plurality of L2 distances from the motion plan location respectively to the plurality of nominal pathway locations; and

determining, by the one or more computing devices, the projected sub-distance as a sum of;

each weight multiplied by the corresponding pathway distance divided by a sum of the plurality of weights.

7. A computer system, comprising:

one or more processors; and

one or more tangible, non-transitory, computer readable media that collectively store instructions that, when executed by the one or more processors, cause the computer system to perform operations, the operations comprising;

obtaining a candidate motion plan that describes a proposed motion path for an autonomous vehicle;

identifying a nominal pathway associated with the autonomous vehicle;

projecting the candidate motion plan onto the nominal pathway to determine a projected distance for the candidate motion plan; and

controlling motion of the autonomous vehicle based at least in part on the projected distance of the candidate motion plan;

wherein controlling the motion of the autonomous vehicle based at least in part on the projected distance comprises;

evaluating a reward function with the projected distance of the candidate motion plan to determine a reward amount for the candidate motion plan; and

determining whether to select the candidate motion plan for implementation based at least in part on the reward amount for the candidate motion plan;

wherein determining whether to select the candidate motion plan for implementation based at least in part on the reward amount for the candidate motion plan comprises iteratively searching over a motion planning space to optimize a total cost, the total cost comprising a sum of one or more cost functions and the reward function.

8. The computer system of claim 7, wherein identifying the nominal pathway associated with the autonomous vehicle comprises identifying the nominal pathway that corresponds to a center line of one or more lanes in which the autonomous vehicle intends to travel.

9. The computer system of claim 7, wherein controlling motion of the autonomous vehicle based at least in part on the projected distance comprises providing a reward that is correlated to the projected distance of the candidate motion plan.

10. The computer system of claim 7, wherein projecting the candidate motion plan onto the nominal pathway to determine the projected distance for the candidate motion plan comprises:

identifying a plurality of nominal pathway locations, wherein a plurality of pathway distances are respectively associated with the plurality of nominal pathway locations;

identifying a plurality of motion plan locations; and

determining the projected distance for the candidate motion plan based at least in part on the plurality of pathway distances and the plurality of motion plan locations.

11. The computer system of claim 10, wherein determining the projected distance for the candidate motion plan comprises:

determining a projected sub-distance for each of the plurality of motion plan locations; and

determining the projected distance for the candidate motion plan based at least in part on the plurality of projected sub-distances for the plurality of motion plan locations.

12. The computer system of claim 11, wherein determining the projected sub-distance for each motion plan location comprises:

determining a plurality of L2 distances from the motion plan location respectively to the plurality of nominal pathway locations;

determining a plurality of weights respectively based at least in part on the plurality of L2 distances from the motion plan location respectively to the plurality of nominal pathway locations; and

determining the projected sub-distance as a sum of;

each weight multiplied by the corresponding pathway distance divided by a sum of the plurality of weights.

13. An autonomous vehicle, comprising:

one or more processors;

a motion planning system implemented by one or more processors, the motion planning system comprising an optimization planner that searches a motion planning space to optimize a total cost associated with a motion plan, the total cost based at least in part on one or more cost functions and a reward function that provides a reward that is positively correlated to a projected distance associated with the motion plan; and

one or more tangible, non-transitory, computer-readable media that store instructions that, when executed by the one or more processors, cause the one or more processors perform operations, the operations comprising;

obtaining a candidate motion plan that describes a proposed motion path for an autonomous vehicle;

identifying a nominal pathway associated with the autonomous vehicle; and

projecting the candidate motion plan onto the nominal pathway to determine a projected distance for the candidate motion plan;

wherein projecting the candidate motion plan onto the nominal pathway to determine the projected distance for the candidate motion plan comprises;

identifying one or more plan locations; and

projecting each of the one or more plan locations onto the nominal pathway.

14. The autonomous vehicle of claim 13, wherein projecting each of the one or more plan locations onto the nominal pathway comprises:

identifying one or more nominal pathway locations; and

determining a distance between each of the one or more plan locations and each of the one or more nominal pathway locations.

15. The autonomous vehicle of claim 14, wherein projecting the candidate motion plan onto the nominal pathway to determine the projected distance for the candidate motion plan further comprises:

determining, for each pair of plan location and nominal pathway location, a weight based on the distance between such plan location and such nominal pathway location, the weight negatively correlated to the distance between such plan location and such nominal pathway location; and

multiplying, for each pair of plan location and nominal pathway location, the corresponding weight with a respective nominal pathway distance that describes a distance along the nominal pathway at which such nominal pathway location is located.

16. A computer-implemented method to control motion of an autonomous vehicle, the method comprising:

obtaining, by one or more computing devices, a candidate motion plan that describes a proposed motion path for the autonomous vehicle;

identifying, by the one or more computing devices, a nominal pathway associated with the autonomous vehicle;

projecting, by the one or more computing devices, the candidate motion plan onto the nominal pathway to determine a projected distance for the candidate motion plan, wherein projecting, by the one or more computing devices, the candidate motion plan onto the nominal pathway to determine the projected distance for the candidate motion plan comprises;

identifying, by the one or more computing devices, a plurality of nominal pathway locations of the nominal pathway, wherein a plurality of pathway distances are respectively associated with the plurality of nominal pathway locations;

identifying, by the one or more computing devices, a plurality of motion plan locations of the candidate motion plan; and

determining, by the one or more computing devices, the projected distance for the candidate motion plan based at least in part on the plurality of pathway distances and the plurality of motion plan locations; and

controlling, by the one or more computing devices, motion of the autonomous vehicle based at least in part on the projected distance of the candidate motion plan.

17. The computer-implemented method of claim 16, wherein determining, by the one or more computing devices, the projected distance for the candidate motion plan comprises:

determining, by the one or more computing devices, a projected sub-distance for each of the plurality of motion plan locations; and

determining, by the one or more computing devices, the projected distance for the candidate motion plan based at least in part on the plurality of projected sub-distances for the plurality of motion plan locations.

18. The computer-implemented method of claim 17, wherein determining, by the one or more computing devices, the projected sub-distance for each motion plan location comprises:

determining, by the one or more computing devices, a plurality of L2 distances from the motion plan location respectively to the plurality of nominal pathway locations;

determining by the one or more computing devices a plurality of weights respectively based at least in part on the plurality of L2 distances from the motion plan location respectively to the plurality of nominal pathway locations; and

determining, by the one or more computing devices, the projected sub-distance as a sum of;

each weight multiplied by the corresponding pathway distance divided by a sum of the plurality of weights.