Machine-learning systems and techniques to optimize teleoperation and/or planner decisions

• US 10,591,910 B2
• Filed: 05/14/2018
• Issued: 03/17/2020
• Est. Priority Date: 11/04/2015
• Status: Active Grant

• Current Assignee: Zoox Inc. (Amazon.com, Inc.)
• Original Assignee: Zoox Inc. (Amazon.com, Inc.)
• Inventors: Levinson, Jesse Sol, Sibley, Gabriel Thurston, Rege, Ashutosh Gajanan
• Primary Examiner(s): Lee, Tyler J
• Application Number:
  US15/979,413
• Publication Number:
  US 20180329411A1
• Time in Patent Office: 673 Days
• Field of Search: None
• CPC Class Codes: B60Q 1/28 for indicating front of vehicle
• CPC Class Codes: B60Q 1/30 for indicating rear of vehicle, e.g. by means of reflecting surfaces
• CPC Class Codes: B60Q 1/503 using luminous text or symbol displays in or on the vehicle, e.g. static text
• CPC Class Codes: B60Q 1/507 specific to autonomous vehicles
• CPC Class Codes: B60Q 1/508 specific to vehicles driving in fleets or convoys
• CPC Class Codes: B60Q 1/549 for expressing greetings, gratitude or emotions
• CPC Class Codes: B60Q 2400/40 Welcome lights, i.e. specific or existing exterior lamps to assist leaving or approaching the vehicle
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/87 Combinations of radar systems, e.g. primary radar and secondary radar
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 2013/9316 combined with communication equipment with other vehicles or with base stations
• CPC Class Codes: G01S 2013/9322 using additional data, e.g. driver condition, road state or weather data
• CPC Class Codes: G01S 7/4972 Alignment of sensor
• CPC Class Codes: G05D 1/0022 characterised by the communication link data switching networks in general H04L12/00
• CPC Class Codes: G05D 1/0027 involving a plurality of vehicles, e.g. fleet or convoy travelling fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 : Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 10/00 : Administration; Management
• CPC Class Codes: G06Q 10/06 : Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G08G 1/005 : including pedestrian guidance indicator
• CPC Class Codes: G08G 1/165 : for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/202 : Dispatching vehicles on the basis of a location, e.g. taxi dispatching
• CPC Class Codes: H04L 41/0816 : the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 41/16 : using machine learning or artificial intelligence
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/12 : specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Abstract

A system, an apparatus or a process may be configured to implement an application that applies artificial intelligence and/or machine-learning techniques to predict an optimal course of action (or a subset of courses of action) for an autonomous vehicle system (e.g., one or more of a planner of an autonomous vehicle, a simulator, or a teleoperator) to undertake based on suboptimal autonomous vehicle performance and/or changes in detected sensor data (e.g., new buildings, landmarks, potholes, etc.). The application may determine a subset of trajectories based on a number of decisions and interactions when resolving an anomaly due to an event or condition. The application may use aggregated sensor data from multiple autonomous vehicles to assist in identifying events or conditions that might affect travel (e.g., using semantic scene classification). An optimal subset of trajectories may be formed based on recommendations responsive to semantic changes (e.g., road construction).

Claims

1. A method comprising:

receiving telemetry data from an autonomous vehicle, the telemetry data comprising sensor data from a sensor on the autonomous vehicle;

receiving policy data;

determining, based at least in part on at least one of the telemetry data or the policy data, an event in a region of an environment through which the autonomous vehicle has traversed;

determining, based at least in part on the event, that a state of operation of the autonomous vehicle is non-normative, the non-normative state indicating an insufficiency of guaranteeing collision-free travel;

determining, based at least in part on the state of operation, a plurality of trajectories;

determining, based at least in part on a trajectory of the plurality of trajectories, updated policy data; and

transmitting the updated policy data to the autonomous vehicle, the updated policy data configured to cause the autonomous vehicle to operate according to one or more policies of the updated policy data.

2. The method as claim 1 recites,

wherein determining updated policy data comprises determining, via a policy explorer, the updated policy data, and

wherein the policy data comprises one or more of:

requiring trajectories to have stand-off distances to nearby objects;

or

requiring trajectories to stay within a double yellow line.

3. The method as claim 1 recites, wherein determining that the state of operation of the autonomous vehicle is non-normative comprises determining whether a confidence level associated with the event meets or exceeds a threshold confidence level.

4. The method as claim 1 recites,

wherein determining updated policy data comprises simulating one or more simulated autonomous vehicle in a simulated environment.

5. The method as claim 4 recites, wherein simulating one or more simulated autonomous vehicles comprises simulating one or more simulated LIDAR devices, simulated RADAR sensors, or simulated camera sensors.

6. The method as claim 1 recites, wherein determining updated policy data comprises receiving, from a teleoperator via a policy explorer, the updated policy data.

7. A system comprising:

one or more processors;

memory having stored thereon processor-executable instructions that, when executed by the one or more processors, cause the system to;

receive telemetry data from an autonomous vehicle, the telemetry data comprising sensor data from a sensor on the autonomous vehicle;

receive policy data;

determine, based at least in part on at least one of the telemetry data or the policy data, a confidence level associated with an event in a region of an environment surrounding the autonomous vehicle;

determine, based at least in part on the confidence level, a plurality of trajectories;

determine, based at least in part on a trajectory of the plurality of trajectories, updated policy data; and

transmit the updated policy data to the autonomous vehicle, the updated policy data configured to cause the autonomous vehicle to operate according to one or more policies of the updated policy data.

8. The system as claim 7 recites,

wherein determining the confidence level comprises determining at least a first confidence level associated with a first candidate trajectory generated by the autonomous vehicle;

and

wherein the plurality of trajectories comprise alternate trajectories that differ from the first candidate trajectory.

9. The system as claim 8 recites,

wherein the first candidate trajectory is one of a plurality of candidate trajectories, and

wherein at least one of the policy data or the updated policy data comprises a rule associated with the event that causes the autonomous vehicle to select a guiding trajectory for controlling motion of the autonomous vehicle from among the alternate trajectories instead of the plurality of candidate trajectories.

10. The system as claim 7 recites,

wherein determining updated policy data comprises determining, via a policy explorer, the updated policy data, and

wherein at least one of the policy data or the updated policy data comprises one or more of:

a rule associated with the event or a semantic classification of an object in the environment, the rule causing the autonomous vehicle to select a guiding trajectory differently during the event or detection of the object than during normative operation;

requiring trajectories to have stand-off distances to nearby objects;

or

requiring trajectories to stay within a double yellow line.

11. The system as claim 7 recites, wherein the instructions further cause the system to determine, based at least in part on the confidence level, that a state of operation of the autonomous vehicle is non-normative based at least in part on determining that the confidence level does not meet a threshold confidence level, wherein a non-normative state of operation is indicative of an insufficiency of guaranteeing collision-free travel.

12. The system as claim 7 recites, wherein determining updated policy data comprises simulating one or more simulated autonomous vehicles in a simulated environment.

13. The system as claim 12 recites, wherein simulating one or more simulated autonomous vehicles comprises simulating one or more simulated LIDAR devices, simulated RADAR sensors, or simulated camera sensors.

14. The system as claim 7, wherein determining updated policy data comprises receiving, from a teleoperator via a policy explorer, the updated policy data.

15. The system as claim 14 recites, wherein the instructions, when executed by the one or more processors, further cause the system to transmit the updated policy data to a second autonomous vehicle.

16. A system comprising:

one or more processors;

one or more sensors;

a memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to;

receive telemetry data from the one or more sensors;

receive policy data;

determine, based at least in part on at least one of the telemetry data or the policy data, an event in a region of an environment surrounding an autonomous vehicle and a confidence level associated with the event;

transmit, to a second device, at least part of at least one of the telemetry data or the policy data;

receive, from the second device, updated policy data;

alter, based at least in part on the updated policy data, operation of the autonomous vehicle during the event.

17. The system as claim 16 recites, wherein altering operation of the autonomous vehicle comprises at least one of:

generating an alternate trajectory in addition to a plurality of trajectories generated by the autonomous vehicle at substantially the same time that the event was detected;

selecting, based at least in part on a rule associated with the event or a semantic classification, a trajectory for controlling motion of the autonomous vehicle, the updated policy data comprising the rule;

or

modifying a stand-off distance to nearby objects.

18. The system as claim 16 recites, wherein the second device comprises a second autonomous vehicle or a teleoperation device.

19. The system as claim 16 recites, wherein transmitting the at least one of the telemetry data or the policy data is based at least in part on determining that the confidence level does not meet or exceed a threshold confidence level.

20. The system as claim 16 recites,

wherein determining the confidence level comprises determining at least a first confidence level associated with a first candidate trajectory generated by the autonomous vehicle;

and

wherein the updated policy data causes the autonomous vehicle to determine an alternate trajectory that differs from the first candidate trajectory.