Model for determining drop-off spot at delivery location

• US 10,545,500 B2
• Filed: 08/02/2017
• Issued: 01/28/2020
• Est. Priority Date: 08/02/2017
• Status: Active Grant

• Current Assignee: Wing Aviation LLC (Alphabet Inc.)
• Original Assignee: Wing Aviation LLC (Alphabet Inc.)
• Inventors: Schubert, Martin, Watson, Philip, Grundmann, Michael, Levine, Gabriella
• Primary Examiner(s): Lonsberry, Hunter B
• Assistant Examiner(s): Martinez Borrero, Luis A
• Application Number:
  US15/667,180
• Publication Number:
  US 20190041852A1
• Time in Patent Office: 909 Days
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 2101/60 for transporting passengers; for transporting goods other than weapons
• CPC Class Codes: B64U 2101/64 for parcel delivery or retrieval
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/104 using satellite radio beacon positioning systems, e.g. GPS
• CPC Class Codes: B64U 70/00 Launching, take-off or landing arrangements
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/102 specially adapted for vertical take-off of aircraft
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06Q 10/0836 Recipient pick-ups

Abstract

An example system includes a delivery vehicle, a sensor connected to the delivery vehicle, and a control system that determines a delivery destination for an object. The control system receives sensor data representing a physical environment of at least a portion of the delivery destination and determines a drop-off spot for the object within the delivery destination by way of an artificial neural network (ANN). The ANN is trained to determine the drop-off spot based on previously-designated drop-off spots within corresponding delivery destinations and includes an input node that receives the sensor data, hidden nodes connected to the input node, and an output node connected to the hidden nodes that provides data indicative of a location of the drop-off spot. The control system additionally causes the delivery vehicle to move to and place the object at the drop-off spot.

Claims

1. A system comprising:

an unmanned aerial delivery vehicle;

a sensor connected to the unmanned aerial delivery vehicle; and

a control system comprising one or more processors that are configured to;

determine a delivery destination for an object to be delivered by the unmanned aerial delivery vehicle;

receive, from the sensor, three-dimensional sensor data representing three-dimensional physical features of a physical environment of at least a portion of the delivery destination;

determine a virtual model of the delivery destination based on the three-dimensional sensor data, wherein the virtual model is a three-dimensional model or a two-dimensional model that includes depth information;

determine a drop-off spot for the object within the delivery destination by way of an artificial neural network (ANN) trained to determine the drop-off spot based on (a) a plurality of previously-designated drop-off spots designated for previously-delivered objects by respective object recipients, by respective human pilots, or by the ANN, or (b) a plurality of previously-designated pick-up spots designated by the respective object recipients for previously placed objects, wherein the ANN comprises (i) an input node configured to receive as input the virtual model, (ii) a plurality of hidden nodes connected to the input node, and (iii) an output node connected to one or more of the plurality of hidden nodes and configured to provide data indicative of a location of the drop-off spot within the delivery destination; and

provide instructions to cause the unmanned aerial delivery vehicle to move to the drop-off spot and place the object at the drop-off spot.

2. The system of claim 1, wherein the drop-off spot is an alternative drop-off spot, and wherein the control system is further configured to:

receive data indicative of a user-selected drop-off spot within the delivery destination;

determine, based on the sensor data, that the object cannot be placed at the user-selected drop-off spot; and

in response to determining that the object cannot be placed at the user-selected drop-off spot, determine the alternative drop-off spot within the delivery destination by way of the ANN.

3. The system of claim 2, wherein the control system is configured to determine that the object cannot be placed at the user-selected drop-off spot by:

determining, based on the sensor data, that the user-selected drop-off spot is occupied by another object.

4. The system of claim 2, wherein the control system is configured to determine that the object cannot be placed at the user-selected drop-off spot by:

determining, based on the sensor data, that a path to the user-selected drop-off spot is obstructed by another object within the physical environment.

5. The system of claim 1, wherein the control system is further configured to:

determine a trajectory for the unmanned aerial delivery vehicle to follow to move from a current location of the unmanned aerial delivery vehicle to the location of the drop-off spot within the delivery destination.

6. The system of claim 1, wherein the control system is configured to determine the drop-off spot for the object by:

determining a plurality of candidate drop-off spots by way of the ANN;

determining a confidence level associated with each of the plurality of candidate drop-off spots; and

selecting, from the plurality of candidate drop-off spots, a drop-off spot associated with a highest confidence level to be the drop-off spot for the object.

7. The system of claim 1, wherein the control system is further configured to:

determine an amount of time allotted to delivery of the object to the delivery destination;

determine a plurality of candidate drop-off spots for the object within the delivery destination by way of the ANN;

for each respective candidate drop-off spot of the plurality of candidate drop-off spots, determine a predicted amount of time to move the object to the respective candidate drop-off spot and place the object at the respective candidate drop-off spot; and

based on (i) the amount of time allotted to delivery of the object and (ii) the predicted amount of time determined for each respective candidate drop-off spot, select, from the plurality of candidate drop-off spots, the drop-off spot for the object.

8. The system of claim 1, wherein the control system is configured to determine the drop-off spot by:

classifying, by way of the ANN, physical features within the physical environment into a plurality of predefined categories;

determining one or more structural arrangements formed by the classified physical features, wherein each of the structural arrangements is formed by two or more of the classified physical features, wherein the one or more structural arrangements are selected from a plurality of predefined structural arrangements, and wherein each respective predefined structural arrangement is associated with a respective drop-off spot score indicating suitability of the respective predefined arrangement to receive delivery of the object; and

selecting, from the one or more structural arrangements, a structural arrangement associated with a highest respective drop-off score to serve as the drop-off spot for the object within the delivery destination.

9. The system of claim 1, wherein the control system is further configured to:

determine a dimension of the object; and

determine the drop-off spot within the delivery destination by providing the dimension of the object as input to the ANN, wherein the ANN further comprises an additional input node configured to receive as input the dimension of the object, wherein the ANN is trained to determine the drop-off spot based on respective dimensions of respective objects associated with the plurality of previously-designated drop-off spots.

10. The system of claim 9, wherein the control system is further configured to:

determine an area at the drop-off location to be occupied by the object by providing the dimension of the object as an input to the ANN, wherein the ANN is trained to determine the area at the drop-off location to be occupied by the object based on respective areas occupied by the respective objects associated with the plurality of previously designated drop-off spots, wherein the output node of the ANN is further configured to provide data indicative of the area at the drop-off location to be occupied by the object; and

provide instructions to cause the unmanned aerial delivery vehicle to place the object within the determined area at the drop-off location.

11. The system of claim 1, wherein the control system is further configured to:

determine physical capabilities of the unmanned aerial delivery vehicle, wherein the physical capabilities indicate operations that the unmanned aerial delivery vehicle can perform based on physical construction of the unmanned aerial delivery vehicle; and

determine the drop-off spot within the delivery destination by providing an indication of the physical capabilities of the unmanned aerial delivery vehicle as input to the ANN, wherein the ANN further comprises an additional input node configured to receive as input the indication of the physical capabilities of the unmanned aerial delivery vehicle, and wherein the ANN is further trained to determine the drop-off spot based on physical capabilities of respective delivery vehicles tasked with object delivery to the plurality of previously-designated drop-off spots.

12. The system of claim 1, wherein the ANN is trained to determine the coordinates of the drop-off spot based on respective virtual models corresponding to the plurality of previously-designated drop-off spots.

13. The system of claim 1, wherein connections between nodes of the ANN are associated with respective weighting values, and wherein the control system is further configured to:

receive user feedback from a recipient of the object indicating satisfaction of the recipient with the determined drop-off spot; and

train the ANN based on the received user feedback by adjusting the respective weighting values associated with connections between at least a portion of the nodes of the ANN.

14. The system of claim 13, wherein the user feedback is received after the object is placed at the drop-off spot, and wherein the control system is further configured to:

when the user feedback indicates that the recipient is dissatisfied with the determined drop-off spot, provide instructions to prompt the recipient to indicate an alternative drop-off spot within the delivery destination at which the recipient would have preferred the object to be placed.

15. A method comprising:

determining a delivery destination for an object to be delivered by an unmanned aerial delivery vehicle;

receiving, from a sensor connected to the unmanned aerial delivery vehicle, three-dimensional sensor data representing three-dimensional physical features of a physical environment of at least a portion of the delivery destination;

determining a drop-off spot for the object within the delivery destination by way of an artificial neural network (ANN) trained to determine the drop-off spot based on (a) a plurality of previously-designated drop-off spots designated for previously-delivered objects by respective object recipients, by respective human pilots, or by the ANN, or (b) a plurality of previously-designated pick-up spots designated by the respective object recipients for previously placed objects, wherein the ANN comprises (i) an input node configured to receive as input the three-dimensional sensor data, (ii) a plurality of hidden nodes connected to the input node, and (iii) an output node connected to one or more of the plurality of hidden nodes and configured to provide data indicative of a location of the drop-off spot within the delivery destination; and

providing instructions to cause the unmanned aerial delivery vehicle to move to the drop-off spot and place the object at the drop-off spot.

16. The method of claim 15, wherein the drop-off spot is an alternative drop-off spot, the method further comprising:

receiving data indicative of a user-selected drop-off spot within the delivery destination;

determining, based on the sensor data, that the object cannot be placed at the user-selected drop-off spot; and

in response to determining that the object cannot be placed at the user-selected drop-off spot, determining the alternative drop-off spot within the delivery destination by way of the ANN.

17. The method of claim 15, further comprising:

providing instructions to display, by a user interface of a computing device, a virtual model of an additional delivery destination, wherein the virtual model represents physical features of the additional delivery destination;

providing instructions to prompt, by the user interface, for designation of a target drop-off spot within the virtual model of the additional delivery destination to accumulate the plurality of previously-designated drop-off spots;

receiving, from the computing device, the designation of the drop-off spot within the virtual model; and

storing in a database the designation of the drop-off spot within the virtual model.

18. The method of claim 17, further comprising:

providing instructions to display, by way of the user interface of the computing device, an indication of a reward for providing the designation of the drop-off spot.

19. The method of claim 15, further comprising:

training the ANN to determine the drop-off spot by determining respective weighting values associated with connections between nodes of the ANN based on the plurality of previously-designated drop-off spots.

20. A non-transitory computer readable storage medium having stored thereon instructions that, when executed by a computing device, cause the computing device to perform operations comprising:

determining a delivery destination for an object to be delivered by an unmanned aerial delivery vehicle;

receiving, from a sensor connected to the unmanned aerial delivery vehicle, three-dimensional sensor data representing three-dimensional physical features of a physical environment of at least a portion of the delivery destination;

determining a virtual model of the delivery destination based on the three-dimensional sensor data, wherein the virtual model is a three-dimensional model or a two-dimensional model that includes depth information;

determining a drop-off spot for the object within the delivery destination by way of an artificial neural network (ANN) trained to determine the drop-off spot based on (a) a plurality of previously-designated drop-off spots designated for previously-delivered objects by respective object recipients, by respective human pilots, or by the ANN, or (b) a plurality of previously-designated pick-up spots designated by the respective object recipients for previously placed objects, wherein the ANN comprises (i) an input node configured to receive as input the virtual model, (ii) a plurality of hidden nodes connected to the input node, and (iii) an output node connected to one or more of the plurality of hidden nodes and configured to provide data indicative of a location of the drop-off spot within the delivery destination; and

providing instructions to cause the unmanned aerial delivery vehicle to move to the drop-off spot and place the object at the drop-off spot.