Systems and methods for determining path confidence for unmanned vehicles

• US 10,621,448 B2
• Filed: 08/02/2017
• Issued: 04/14/2020
• Est. Priority Date: 08/02/2017
• Status: Active Grant

• Current Assignee: Wing Aviation LLC (Alphabet Inc.)
• Original Assignee: Wing Aviation LLC (Alphabet Inc.)
• Inventors: Schubert, Martin, Watson, Philip, Grundmann, Michael, Levine, Gabriella
• Primary Examiner(s): Marc-Coleman, Marthe Y
• Application Number:
  US15/667,391
• Publication Number:
  US 20190042859A1
• Time in Patent Office: 986 Days
• Field of Search: 701 25
• CPC Class Codes: B60K 31/0008 including means for detecting potential obstacles in vehicle path
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2556/45 External transmission of data to or from the vehicle
• CPC Class Codes: B60W 2556/65 Data transmitted between vehicles
• CPC Class Codes: B60W 50/082 Selecting or switching between different modes of propelling
• CPC Class Codes: B60W 60/001 Planning or execution of driving tasks
• CPC Class Codes: B60W 60/0011 involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles
• CPC Class Codes: B62D 1/28 non-mechanical , e.g. following a line or other known markers
• CPC Class Codes: B62D 1/283 for unmanned vehicles
• CPC Class Codes: B62D 15/025 Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation B62D1/28 takes precedence
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/3446 Details of route searching algorithms, e.g. Dijkstra, A*, arc-flags, using precalculated routes
• CPC Class Codes: G01C 21/3484 Personalized, e.g. from learned user behaviour or user-defined profiles
• CPC Class Codes: G05D 1/00 Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: G05D 1/0027 involving a plurality of vehicles, e.g. fleet or convoy travelling fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0221 involving a learning process
• CPC Class Codes: G05D 1/0242 using non-visible light signals, e.g. IR or UV signals
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06V 10/82 : using neural networks
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/165 : for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Examples implementations relate to determining path confidence for a vehicle. An example method includes receiving a request for a vehicle to navigate a target location. The method further includes determining a navigation path for the vehicle to traverse a first segment of the target location based on a plurality of prior navigation paths previously determined for traversal of segments similar to the first segment of the target location. The method also includes determining a confidence level associated with the navigation path. Based on the determined confidence level, the method additionally includes selecting a navigation mode for the vehicle from a plurality of navigation modes corresponding to a plurality of levels of remote assistance. The method further includes causing the vehicle to traverse the first segment of the target location using a level of remote assistance corresponding to the selected navigation mode for the vehicle.

Claims

1. A method comprising:

receiving a request for a vehicle to navigate a target location;

determining a navigation path for the vehicle to traverse a first segment of the target location using an artificial neural network (ANN) trained based on a plurality of prior navigation paths previously determined for traversal of segments similar to the first segment of the target location, wherein the ANN comprises at least one of an input node configured to receive as input sensor data representing the first segment of the target location from a sensor on the vehicle, and an input node configured to receive as input one or more locomotive capabilities of the vehicle;

determining a confidence level associated with the navigation path;

based on the determined confidence level, selecting a navigation mode for the vehicle from a plurality of navigation modes corresponding to a plurality of levels of remote assistance; and

causing the vehicle to traverse the first segment of the target location using a level of remote assistance corresponding to the selected navigation mode for the vehicle.

2. The method of claim 1, wherein determining the navigation path further comprises determining a distribution of navigation paths and associated probabilities using a probabilistic path-finding model, wherein the confidence level is further based on the distribution of navigation paths and associated probabilities.

3. The method of claim 1, wherein determining the navigation path comprises using a generative adversarial network (GAN), wherein the GAN comprises a path-planning model trained to generate navigation paths and a discriminator model trained to distinguish between operator-provided navigation paths and navigation paths generated by the path-planning model.

4. The method of claim 1, wherein the prior navigation paths comprise paths navigated by a remote operator of the vehicle.

5. The method of claim 1, wherein the prior navigation paths comprise paths navigated by one or more vehicles other than the vehicle.

6. The method of claim 1, wherein the prior navigation paths comprise paths input by an operator at a user interface for simulating vehicle delivery.

7. The method of claim 1, further comprising:

determining that the confidence level is greater than a predefined confidence level threshold; and

based on determining that the confidence level is greater than the predefined confidence level threshold, selecting, as the navigation mode, autonomous control by a control system on the vehicle to traverse the first segment of the target location.

8. The method of claim 1, further comprising:

determining that the confidence level is less than a predefined confidence level threshold; and

based on determining that the confidence level is less than the predefined confidence level threshold, selecting, as the navigation mode, remote control of the vehicle by a remote operator to traverse the first segment of the target location.

9. The method of claim 1, wherein selecting the navigation mode for the vehicle includes:

determining that the confidence level is greater than a first predefined confidence level threshold and less than a second predefined confidence level; and

based on determining that the confidence level is greater than the first predefined confidence level threshold and less than the second predefined confidence level, selecting, as the navigation mode, autonomous control by a control system on the vehicle that follows a plurality of waypoints.

10. The method of claim 1, further comprising:

determining a second navigation path for the vehicle to traverse a second segment of the target location;

determining a second confidence level associated with the determined second navigation path for the vehicle to traverse the second segment of the target location;

based on the determined second confidence level, selecting a different navigation mode for the vehicle from the plurality of navigation modes; and

causing the vehicle to traverse the second segment of the target location using a different level of remote assistance corresponding to the selected different navigation mode for the vehicle.

11. The method of claim 10, wherein each of the first segment and the second segment comprise a discrete surface for the vehicle to traverse at the target location.

12. The method of claim 1, wherein the segments similar to the first segment of the target location have a similar topology as the first segment of the target location.

13. The method of claim 1, wherein the segments similar to the first segment of the target location comprise a same surface material as the first segment of the target location.

14. The method of claim 1, wherein the segments similar to the first segment of the target location comprise the first segment of the target location.

15. A system comprising:

a vehicle that includes a sensor configured to collect sensor data representing the target location; and

a control system configured to;

receive a request for the vehicle to navigate a target location;

determine a navigation path for the vehicle to traverse a first segment of the target location using an artificial neural network (ANN) trained based on a plurality of prior navigation paths previously determined for traversal of segments similar to the first segment of the target location, wherein the ANN includes an input node configured to receive as input the sensor data representing the target location from the sensor on the vehicle;

determine a confidence level associated with the navigation path;

based on the determined confidence level, select a navigation mode for the vehicle from a plurality of navigation modes corresponding to a plurality of levels of remote assistance; and

cause the vehicle to traverse the first segment of the target location using a level of remote assistance corresponding to the selected navigation mode for the vehicle.

16. A non-transitory computer readable medium having stored therein program instructions executable by a computing system to cause the computing system to perform operations, the operations comprising:

receiving a request for a vehicle to navigate to a target location;

determining a navigation path for the vehicle to traverse a first segment of the target location using an artificial neural network (ANN) trained based on a plurality of prior navigation paths previously determined for traversal of segments similar to the first segment of the target location;

determining a confidence level associated with the distribution of navigation paths;

based on the determined confidence level, selecting a navigation mode for the vehicle from a plurality of navigation modes corresponding to a plurality of levels of remote assistance; and

causing the vehicle to traverse the first segment of the target location using a level of remote assistance corresponding to the selected navigation mode for the vehicle;

wherein selecting the navigation mode for the vehicle includes;

determining that the confidence level is greater than a first predefined confidence level and less than a second predefined confidence level;

based on determining that the confidence level is greater than the first predefined confidence level threshold and less than the second predefined confidence level, selecting, as the navigation mode, autonomous control by a control system on the vehicle that follows a plurality of waypoints, wherein selecting, as the navigation mode, autonomous control by a control system on the vehicle that follows a plurality of waypoints includes;

based on determining that the confidence level is less than a third predefined confidence level between the first predefined confidence level threshold and the second predefined confidence level, selecting, as the navigation mode, autonomous control by a control system on the vehicle to traverse a series of waypoints generated by a remote operator; and

based on determining that the confidence level is not less than a third predefined confidence level between the first predefined confidence level threshold and the second predefined confidence level, selecting, as the navigation mode, autonomous control by a control system on the vehicle to traverse a series of waypoints generated by the control system on the vehicle and confirmed by a remote operator.

17. A non-transitory computer readable medium having stored therein program instructions executable by a computing system to cause the computing system to perform operations, the operations comprising:

receiving a request for a vehicle to navigate a target location;

determining a navigation path for the vehicle to traverse a first segment of the target location using an artificial neural network (ANN) trained based on a plurality of prior navigation paths previously determined for traversal of segments similar to the first segment of the target location, wherein determining the navigation path using an ANN comprises using a generative adversarial network (GAN), wherein the GAN comprises a path-planning model trained to generate navigation paths and a discriminator model trained to distinguish between operator-provided navigation paths and navigation paths generated by the path-planning model;

determining a confidence level associated with the navigation path;

based on the determined confidence level, selecting a navigation mode for the vehicle from a plurality of navigation modes corresponding to a plurality of levels of remote assistance; and

causing the vehicle to traverse the first segment of the target location using a level of remote assistance corresponding to the selected navigation mode for the vehicle.