Machine-learning systems and techniques to optimize teleoperation and/or planner decisions

• US 9,632,502 B1
• Filed: 11/05/2015
• Issued: 04/25/2017
• Est. Priority Date: 11/04/2015
• Status: Active Grant

• Current Assignee: Zoox Inc. (Amazon.com, Inc.)
• Original Assignee: Zoox Inc. (Amazon.com, Inc.)
• Inventors: Levinson, Jesse Sol, Sibley, Gabriel Thurston, Rege, Ashutosh Gajanan
• Primary Examiner(s): Lee, Tyler J
• Application Number:
  US14/933,602
• Publication Number:
  US 20170123419A1
• Time in Patent Office: 537 Days
• Field of Search: 701 23- 28
• CPC Class Codes: B60Q 1/28 for indicating front of vehicle
• CPC Class Codes: B60Q 1/30 for indicating rear of vehicle, e.g. by means of reflecting surfaces
• CPC Class Codes: B60Q 1/503 using luminous text or symbol displays in or on the vehicle, e.g. static text
• CPC Class Codes: B60Q 1/507 specific to autonomous vehicles
• CPC Class Codes: B60Q 1/508 specific to vehicles driving in fleets or convoys
• CPC Class Codes: B60Q 1/549 for expressing greetings, gratitude or emotions
• CPC Class Codes: B60Q 2400/40 Welcome lights, i.e. specific or existing exterior lamps to assist leaving or approaching the vehicle
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/87 Combinations of radar systems, e.g. primary radar and secondary radar
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 2013/9316 combined with communication equipment with other vehicles or with base stations
• CPC Class Codes: G01S 2013/9322 using additional data, e.g. driver condition, road state or weather data
• CPC Class Codes: G01S 7/4972 Alignment of sensor
• CPC Class Codes: G05D 1/0022 characterised by the communication link data switching networks in general H04L12/00
• CPC Class Codes: G05D 1/0027 involving a plurality of vehicles, e.g. fleet or convoy travelling fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 : Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 10/00 : Administration; Management
• CPC Class Codes: G06Q 10/06 : Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G08G 1/005 : including pedestrian guidance indicator
• CPC Class Codes: G08G 1/165 : for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/202 : Dispatching vehicles on the basis of a location, e.g. taxi dispatching
• CPC Class Codes: H04L 41/0816 : the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 41/16 : using machine learning or artificial intelligence
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/12 : specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Abstract

A system, an apparatus or a process may be configured to implement an application that applies artificial intelligence and/or machine-learning techniques to predict an optimal course of action (or a subset of courses of action) for an autonomous vehicle system (e.g., one or more of a planner of an autonomous vehicle, a simulator, or a teleoperator) to undertake based on suboptimal autonomous vehicle performance and/or changes in detected sensor data (e.g., new buildings, landmarks, potholes, etc.). The application may determine a subset of trajectories based on a number of decisions and interactions when resolving an anomaly due to an event or condition. The application may use aggregated sensor data from multiple autonomous vehicles to assist in identifying events or conditions that might affect travel (e.g., using semantic scene classification). An optimal subset of trajectories may be formed based on recommendations responsive to semantic changes (e.g., road construction).

Claims

1. A method, comprising:

receiving, from one or more of a sensor or a communication interface, telemetry data associated with an event, the event being a situation or condition associated with operation of an autonomous vehicle;

obtaining policy data, the policy data including instructions for operating an autonomous vehicle according to a trajectory and the policy data including a confidence level associated with the received trajectory, the confidence level indicating a degree of certainty that the autonomous vehicle, in operating according to the trajectory, will operate safely in view of the event;

obtaining candidate trajectories responsive to the event, based on the telemetry data, each candidate trajectory having an associated confidence level;

generating, by machine-learning by a processor and based at least in part on the candidate trajectories and the telemetry data, updated policy data that includes instructions for operating the autonomous vehicle responsive to the event differently than according to the policy data; and

communicating the updated policy data to at least one autonomous vehicle via a communications interface.

2. The method of claim 1 further comprising:

determining that the confidence level associated with the received trajectory is below a confidence level threshold, indicating a non-normative operational state of the autonomous vehicle.

3. The method of claim 2, wherein at least one of the associated confidence levels is greater than the confidence level associated with the received trajectory.

4. The method of claim 2, wherein the candidate trajectories are obtained at least in part responsive to the confidence level associated with the received trajectory indicating the non-normative operational state.

5. The method of claim 1, wherein communicating the updated policy data to at least one autonomous vehicle comprises communicating the updated policy data to an autonomous vehicle other than the autonomous vehicle with which the event is associated.

6. The method of claim 5, wherein the telemetry data and policy data are obtained from the autonomous vehicle with which the event is associated.

7. The method of claim 1, wherein the telemetry data is received and the policy data is obtained at a teleoperator computing device and wherein the teleoperator computing device is configured to communicate the updated policy data to at least one autonomous vehicle.

8. The method of claim 1, wherein the telemetry data is received and the policy data is obtained at a simulator, the method further comprising, by the simulator:

obtaining the candidate trajectories responsive to the event by simulating the candidate trajectories based at least in part on one or more of the telemetry data or the policy data;

determining confidence levels associated with the simulated candidate trajectories;

receiving selection of one of the candidate trajectories for implementation by the autonomous vehicle; and

providing the selected trajectory to the at least one autonomous vehicle as part of the updated policy data.

9. The method of claim 8 the method further comprising, by the simulator:

conducting the generation of the updated policy data based at least in part on one or more of simulating the candidate trajectories or the selected trajectory; and

communicating one or more of the policy data or the updated policy data to one or more of;

a teleoperator computing device, or

the at least one autonomous vehicle.

10. The method of claim 1, wherein the telemetry data is received and the policy data is obtained at a simulator configured to simulate an autonomous vehicle in a synthetic environment.

11. The method of claim 1, wherein the telemetry data is received, the policy data is obtained, and the updated policy data is generated at an autonomous vehicle.

12. The method of claim 1, wherein the policy data is obtained from one or more of an autonomous vehicle that captured the telemetry data or a data repository associated with a service for a fleet of autonomous vehicles.

13. The method of claim 1, further comprising:

determining that the at least one autonomous vehicle has operated according to the teleoperation trajectory;

obtaining further telemetry data from the at least one autonomous vehicle;

determining from at least the further telemetry data that operation of the at least one autonomous vehicle according to the teleoperation trajectory and responsive to the event resulted in normative operation of the at least one autonomous vehicle;

generating, by machine-learning, the updated policy responsive to the determining that operating the at least one autonomous vehicle according to the teleoperation trajectory resulted in normative operation; and

communicating the updated policy data to at least one autonomous vehicle.

14. The method of claim 1, wherein the policy data includes one or more of:

criteria for selecting a candidate trajectory for operating the autonomous vehicle;

rules defining constraints for generating trajectories;

threshold confidence levels for determining whether operations of the autonomous vehicle are normative;

a classifier for detecting, from the telemetry data, objects in an environment in which the autonomous vehicle is operating;

a semantic classification of an object detected from the telemetry data;

or

a semantic classification associated with the event; and

wherein the updated policy data includes an update to one or more of the criteria, the rules, the threshold confidence levels, the semantic classification of the object, or the semantic classification associated with the event.

15. A system comprising:

one or more processors;

a memory having stored thereon first criteria for controlling motion of an autonomous vehicle and a planner module executable by the one or more processors and that, when executed by the one or more processors, configure the system to perform operations including;

obtaining sensor data related to operation of an autonomous vehicle and an event;

determining, based at least on the sensor data and the first criteria, a first trajectory for operation of the autonomous vehicle responsive to the event;

receiving instructions for implementing a second trajectory for operation of the autonomous vehicle responsive to the event; and

learning, from at least the instructions and the sensor data, second criteria to update the first criteria, the second criteria configuring the one or more processors to determine the second trajectory based at least on the sensor data and the instructions.

16. A system as claim 15 recites, the first criteria including one or more of:

criteria for selecting a candidate trajectory for operating the autonomous vehicle;

rules defining constraints for generating trajectories;

threshold confidence levels for determining whether operations of the autonomous vehicle are normative;

a classifier for detecting, from the sensor data, objects in an environment in which the autonomous vehicle is operating;

a semantic classification of an object detected by the perception engine from the sensor data;

a semantic classification associated with the event;

one or more candidate trajectories;

or

confidence levels associated with the candidate trajectories or rules for determining confidence levels associated with the candidate trajectories;

and wherein the second criteria includes a modification of or addition to one or more of;

the criteria for selecting a candidate trajectory,

the rules defining constraints for generating trajectories,

the threshold confidence levels,

the perception engine configuration,

the semantic classifications,

the candidate trajectories,

the confidence levels, or

rules for determining confidence levels.

17. The system as claim 15 recites, the system comprising an autonomous vehicle and further including one or more of:

one or more sensors for obtaining the sensor data or

a communication interface for receiving the instructions.

18. A system as claim 15 recites, the operations further including simulating in a synthetic environment candidate trajectories based at least in part on one or more of the sensor data and the first criteria, and wherein receiving a second trajectory for operation of the autonomous vehicle includes receiving selection of one of the simulated candidate trajectories.

19. The system as claim 15 recites, the system comprising a teleoperator computing device, the teleoperator computing device being configured to control operations of the autonomous vehicle.

20. One or more non-transitory computer-readable media storing processor-executable instructions that, when executed, cause one or more processors to perform operations, the operations comprising:

receiving telemetry data associated with operation of an autonomous vehicle, the autonomous vehicle storing first criteria used by the autonomous vehicle to determine first trajectories to control motion of the autonomous vehicle from the telemetry data;

receiving instructions for the autonomous vehicle to operate according to a defined trajectory;

learning, from the instructions and the telemetry data, second criteria for determining second trajectories from the telemetry data, where the first trajectories and the second trajectories are different and the second criteria includes an update to the first criteria; and

communicating the second criteria to at least one autonomous vehicle to update the first criteria of the at least one autonomous vehicle.

21. One or more non-transitory computer-readable media as claim 20 recites, the operations further comprising:

generating a plurality of candidate trajectories for the autonomous vehicle responsive to receiving the telemetry data;

determining confidence levels corresponding to respective ones of the plurality of candidate trajectories, a confidence level indicating a degree of certainty that the autonomous vehicle, in operating according to one of the plurality of candidate trajectories, will operate safely in view of the event; and

at least one of;

rank the plurality of candidate trajectories according to confidence levels of the plurality of candidate trajectories, and

select, based at least in part on the ranking, the confidence levels, and an acceptable confidence level specified by the first criteria or the second criteria, respectively, a trajectory from the plurality of candidate trajectories for implementation by the autonomous vehicle;

or

transmit a request for teleoperation of the autonomous vehicle if none of the plurality of candidate trajectories is associated with a confidence level that meets the acceptable confidence level, the instructions for the autonomous vehicle to operate according to the defined trajectory being received in response to the request for teleoperation.

22. One or more non-transitory computer-readable media as claim 21 recites, determining the confidence levels being based at least in part on the first criteria or the second criteria, respectively, and determining a confidence level for a respective candidate trajectory in view of the telemetry data and the determining including one or more of:

determining a probability that by operating according to the candidate trajectory, the autonomous vehicle will operate safely in view of the telemetry data;

determining whether the candidate trajectory comports with traffic laws;

determining whether the candidate trajectory violates the traffic laws;

determining consequences of violating the traffic laws;

determining consequences of not violating the traffic laws;

or

determining whether the candidate trajectory violates a passenger comfort policy,

wherein the acceptable confidence level is a minimum required confidence level to permit the autonomous vehicle to implement the candidate trajectory.

23. One or more non-transitory computer-readable media as claim 20 recites, the one or more computer-readable media being a memory of an autonomous vehicle, a teleoperator computing device, a computing device storing a simulator, or a fleet management service device.