Autonomous vehicle fleet model training and testing

• US 10,459,444 B1
• Filed: 11/03/2017
• Issued: 10/29/2019
• Est. Priority Date: 11/03/2017
• Status: Active Grant

• Current Assignee: Zoox Inc. (Amazon.com, Inc.)
• Original Assignee: Zoox Inc. (Amazon.com, Inc.)
• Inventors: Kentley-Klay, Timothy David
• Primary Examiner(s): Zanelli, Michael J
• Application Number:
  US15/803,603
• Time in Patent Office: 725 Days
• CPC Class Codes: G05D 1/0027 involving a plurality of vehicles, e.g. fleet or convoy travelling fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0221 involving a learning process
• CPC Class Codes: G05D 1/0293 Convoy travelling
• CPC Class Codes: G07C 5/008 communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G08G 1/123 indicating the position of vehicles, e.g. scheduled vehicles; Managing passenger vehicles circulating according to a fixed timetable, e.g. buses, trains, tramstransmission of navigation instructions to vehicles G08G1/0968

Abstract

A method and system of using excess computational resources on autonomous vehicles. Such excess computational resources may be available during periods of low demand, or other periods of idleness (e.g., parking). Where portions of computing resources are available amongst a fleet of vehicles, such excess computing resources may be pooled as a single resource. The excess computational resources may be used, for example, to train and/or test machine-learning models. Performance metrics of such models may be determined using hardware and software on the autonomous vehicle, for example sensors. Models having performance metrics outperforming current models may be considered as validated models. Validated models may be transmitted to a remote computing system for dissemination to a fleet of vehicles.

Claims

1. A vehicle comprising:

a vehicle controller operably connected to at least one sensor, a network interface, and a drive system to control operation of the vehicle, the vehicle controller being configured to;

receive an indication that the vehicle controller has unallocated computational resources;

receive a command to utilize the unallocated computational resources, the command causing the vehicle controller to be further configured to conduct one or more of;

testing an experimental machine learning (ML) model based at least in part on causing the drive system to control operation of the vehicle using the experimental ML model, the testing yielding a test result, or

training a target ML model to create a trained ML model, based at least in part on causing the drive system to control operation of the vehicle using the target ML model.

2. The vehicle of claim 1, wherein the indication that the vehicle controller has unallocated computational resources comprises one or more of:

an indication that a first service demand for the vehicle is unlikely to meet a first threshold, based at least in part on present or historical fleet information,

an indication that the vehicle is idle, or

an indication that a second service demand for the vehicle is unlikely to meet a second threshold during a time period.

3. The vehicle of claim 2, wherein the at least one of the first threshold or the second threshold comprises at least one of a number of requests for rides, an average wait time before arrival of a vehicle, or a ratio of vehicles in-service to requests.

4. The vehicle of claim 2, the vehicle controller being further configured to terminate conducting the testing or training based at least in part on one or more of:

determining that the time period has lapsed,

receiving a task for the vehicle to perform, or

determining or receiving an indication that the service demand for the vehicle has increased past the threshold.

5. The vehicle of claim 1, wherein:

the vehicle controller is configured to conduct an evaluation, and

testing the experimental ML model further comprises;

determining instructions for controlling the vehicle from the vehicle controller based at least in part on an output of the experimental ML model; and

determining performance metrics of the experimental ML model based at least in part on control of the vehicle according to the instructions.

6. The vehicle of claim 1, wherein:

the vehicle controller is a first instance of the vehicle controller that comprises a current ML model, and

conducting one or more of the testing or the training comprises;

instantiating a second vehicle controller in a virtual machine, the second vehicle controller comprising one or more of the experimental ML model or the target ML model;

receiving first instructions for controlling the vehicle from the first instance of the vehicle controller at a first time;

receiving second instructions for controlling the vehicle from the second instance of the vehicle controller at a second time, the second time being substantially the same as the first time;

controlling the vehicle according to the first instructions or the second instructions; and

determining performance metrics of one or more of the first instance of the vehicle controller or the second vehicle controller based at least in part on one or more of the first instructions, the second instructions, or sensor data.

7. The vehicle of claim 1, wherein:

the vehicle receives a test mission via the network interface, the test mission including a location and a condition; and

the vehicle controller is configured to conduct one or more of testing or training, the testing and training configured to consume sensor data associated with the location and the condition.

8. The vehicle of claim 7, wherein one or more of the location or the condition correlates with a decrease in vehicle performance, and

further wherein the sensor data is previously recorded sensor data.

9. The vehicle of claim 1, wherein conducting one or more of the testing or the training includes:

instantiating a virtual machine on a processor of the vehicle; and

evaluating one or more of the experimental ML model or the target ML model on the virtual machine.

10. The vehicle of claim 9, the vehicle controller being further configured to:

terminate conducting the testing or the training; and

migrate, via the network interface of the vehicle, one or more of a machine state, sensor data, performance metrics, the experimental ML model, or the trained ML model to one or more of a remote computing device or another vehicle, based at least in part on the termination.

11. The vehicle of claim 9, the vehicle controller being further configured to:

terminate conducting the testing or the training;

store one or more of a machine state, sensor data, performance metrics, the experimental ML model, or the updated ML model;

receive a subsequent indication that the vehicle controller has unallocated computational resources; and

resume conducting one or more of the testing or the training based at least in part on receiving the subsequent indication that the vehicle controller has unallocated computational resources, based at least in part on one or more of the machine state, the sensor data, the performance metrics, the experimental ML model, or the updated ML model.

12. The vehicle of claim 1, the vehicle controller being further configured to:

form an ad hoc connection with one or more other vehicles via the network interface;

receive a test mission from a remote computing device via the network interface; and

transmit instructions to the one or more other vehicles to cause the one or more other vehicles to one or more of conduct a maneuver or change a mode of operation, according to the test mission.

13. The vehicle of claim 1, the vehicle controller being further configured to:

form an ad hoc connection with one or more other vehicles via the network interface;

transmit a portion of data to be processed to at least one of the one or more of the vehicles; and

receive a processing result determined by the at least one of the one or more of the vehicles.

14. The vehicle of claim 13, wherein forming the ad hoc connection includes instantiating a single system image across the one or more vehicles or instantiating a virtual machine cluster among the one or more vehicles.

15. The vehicle of claim 1, wherein the vehicle controller is further configured to transition the vehicle from an in-service state to an out-of-service state, based at least in part on determining that the vehicle has unallocated computational resources.

16. A method of controlling a vehicle comprising:

receiving an indication that a vehicle controller of the vehicle has or will have unallocated computational resources at a time period;

based at least in part on receiving the indication, causing the vehicle controller to one or more of;

train a target machine-learning model, creating a trained model;

or

test an experimental model; and

control the vehicle according to an output of the trained model or the experimental model.

17. The method of claim 16, wherein the method further comprises receiving a test mission, the test mission comprising one or more of a location or condition;

and

the vehicle controller is further configured to evaluate the experimental model or the trained model based at least in part on the test mission.

18. The method of claim 17 further comprising:

instantiating a virtual machine to determine an output of the trained model or the experimental model;

determining that the vehicle is not able to complete the test mission;

terminating training the trained model or testing the experimental model according to the test mission;

recording one or more of a machine state, sensor data received from one or more sensors, performance metrics, the experimental model, or an updated model derived from training the target model; and

at least one of;

transmitting one or more of the virtual machine state, sensor data, performance metrics, the experimental model, or the updated model to one or more of a remote computing device or another vehicle;

or

resuming the test mission using one or more of the machine state, the sensor data, the performance metrics, the experimental model, or the updated model, based at least in part on receiving an indication that the unallocated computing resources are subsequently available.

19. A non-transitory computer-readable medium having a set of instructions that, when executed, cause one or more processors to perform operations comprising:

determining that a vehicle has unallocated computational resources during a time period;

transmitting to the vehicle instructions to conduct a test mission during the time period and one or more of an experimental model or instructions to train a target model stored at the vehicle;

receiving one or more of sensor data, an updated target model, or performance metrics from the vehicle that correspond to completion of at least a portion of the test mission; and

determining that one or more of the updated target model or the experimental model changed performance of the vehicle.

20. The non-transitory computer-readable medium of claim 19, the set of instructions further causing the one or more processors to perform operations comprising transmitting one or more of the trained model or the experimental model to one or more additional vehicles of a fleet, based at least in part on determining that one or more of the trained model or the experimental model improves performance of the vehicle.