Human interaction with unmanned aerial vehicles

• US 9,921,579 B1
• Filed: 07/18/2016
• Issued: 03/20/2018
• Est. Priority Date: 09/29/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Schaffalitzky, Frederik
• Primary Examiner(s): Patel, Shardul D
• Application Number:
  US15/213,307
• Time in Patent Office: 610 Days
• Field of Search: 701 2- 18, 701 23, 701 25, 701120, 700 59, 700245, 382103, 382115-128, 702152, 702153, 244189, 244190, 244195
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 1/10 Stowage arrangements for the devices in aircraft
• CPC Class Codes: B64D 1/12 Releasing
• CPC Class Codes: B64D 31/04 actuated personally
• CPC Class Codes: B64D 31/06 actuated automatically
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/60 for transporting passengers; for transporting goods other than weapons
• CPC Class Codes: B64U 2101/69 the UAVs provided with means for airdropping goods, e.g. deploying a parachute during descent
• CPC Class Codes: B64U 2201/00 UAVs characterised by their flight controls
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: B64U 30/21 Rotary wings
• CPC Class Codes: B64U 60/50 with landing legs
• CPC Class Codes: B64U 70/30 for capturing UAVs in flight by ground or sea-based arresting gear, e.g. by a cable or a net
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0022 characterised by the communication link data switching networks in general H04L12/00
• CPC Class Codes: G05D 1/0033 by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06Q 10/083 Shipping

Abstract

In some examples, an unmanned aerial vehicle is provided. The unmanned aerial vehicle may include a propulsion device, a sensor device, and a management system. In some examples, the management system may be configured to receive human gestures via the sensor device and, in response, instruct the propulsion device to affect an adjustment to the behavior of the unmanned aerial vehicle. Human gestures may include visible gestures, audible gestures, and other gestures capable of recognition by the unmanned vehicle.

Claims

1. An unmanned aerial vehicle (UAV), comprising:

a frame;

a retaining system connected to the frame and configured to retain a package;

a communication system connected to the frame and configured to receive gesture input; and

a management module associated with at least the communication system and configured to at least;

navigate to a location associated with a profile of a human user;

receive, via the communication system, gesture input comprising a gesture by the human user;

in response to receiving the gesture input, access gesture information describing one or more gestures capable of being recognized by the communication system; and

determine, based on the gesture input and the gesture information, a behavior adjustment for the unmanned aerial vehicle, the behavior adjustment comprising instructing the retaining system to release the package at the location.

2. The UAV of claim 1, wherein the gesture by the human user comprises an audible gesture or a visual gesture.

3. The UAV of claim 1, wherein the communication system comprises a light sensor including at least one of a visible light camera, an infrared camera, an RGB camera, a depth sensor, or an ultraviolet sensitive camera, and the gesture input is received via the light sensor.

4. The UAV of claim 1, wherein:

the gesture information is accessed in a gesture database local to the management module; and

accessing the gesture information from the gesture database includes accessing locally the portion of the gesture information from the gesture database.

5. An apparatus, comprising:

a frame;

a propulsion device connected to the frame;

a retaining system connected to the frame;

a sensor device configured to detect human gestures; and

a management system in communication at least with the propulsion system, the retaining system, and the sensor device, the management system configured to at least;

instruct the propulsion device to move the apparatus according to a delivery plan in order to deliver a payload at a location associated with a profile of a human user;

receive, via the sensor device, a human gesture performed by the human user while the apparatus is moving according to the delivery plan;

determine, based at least in part on the human gesture, a behavior adjustment that causes the retaining system to release the payload; and

instruct the retaining system to implement the behavior adjustment at the location.

6. The apparatus of claim 5, wherein the management system is further configured to at least access gesture information from a gesture repository prior to determining the behavior adjustment, the gesture repository storing information about one or more gestures capable of being recognized by the management system.

7. The apparatus of claim 6, wherein:

the human gesture is a first human gesture and the behavior adjustment is a first behavior adjustment; and

the management system is further configured to at least;

receive, via the sensor device, a second human gesture performed by the human user or a different human user before implementation of the behavior adjustment or after implementation of the behavior adjustment;

determine, based at least in part on the second human gesture, a second behavior adjustment that causes the propulsion device to change at least one of a course of the apparatus or a speed of the apparatus; and

instruct the propulsion device to implement the second behavior adjustment.

8. The apparatus of claim 7, wherein the second behavior adjustment comprises an adjustment resulting in the apparatus moving closer to the human user or the different human user or an adjustment resulting in the apparatus moving further from the human user or the different human user.

9. The apparatus of claim 5, wherein the management system is further configured to at least verify an identity of the human user prior to instructing the retaining system to implement the behavior adjustment at the location.

10. The apparatus of claim 9, wherein verifying the identity of the human user comprises at least one of:

verification by comparison of customer information identifying the human user with information provided by the human user during delivery, verification by connection of the apparatus with a user device of the human user, verification by connection of the apparatus with a specialized device of the human user, verification by the apparatus scanning a code provided by the human user, verification by a remote user receiving verification information from the human user via the apparatus, verification by the human gesture, or verification by a second human gesture distinct from or in combination with the human gesture.

11. The apparatus of claim 5, wherein the propulsion device comprises a plurality of propulsion devices, individual propulsion devices configured to contribute to at least one of vertical movement, lateral movement, or hovering of the apparatus.

12. The apparatus of claim 5, wherein the sensor device comprises at least one of a visual sensing device configured to at least receive visible human gestures or an audible sensing device configured to at least receive audible human gestures.

13. The apparatus of claim 12, wherein the human gesture comprises at least one of:

a voice gesture, an arm gesture, a multiple-arm gesture, a hand gesture, a multiple-hand gesture, a body gesture, a natural reaction gesture, a flag gesture, a sign gesture, or a static gesture.

14. The apparatus of claim 5, wherein the management system is further configured to at least receive the delivery plan from a management module associated with an electronic marketplace.

15. The apparatus of claim 5, wherein:

the human gesture is one of a plurality of human gestures performed by a plurality of human users; and

the management system is further configured to at least identify, from the plurality of human gestures, the human gesture.

16. The apparatus of claim 5, wherein receiving the human gesture comprises receiving the human gesture while the human user is located at the location associated with the profile of the human user.

17. A computer-implemented method comprising:

accessing a delivery plan identifying instructions for delivery of an item by an unmanned device at a location associated with a profile of a human user;

receiving information about a gesture performed by the human user while the unmanned device is executing a portion of the delivery plan;

accessing a data repository, the data repository storing a plurality of gestures capable of being recognized by the unmanned device; and

determining, based at least in part on a comparison of the information about the gesture with information about one or more other gestures of the plurality of gestures in the data repository, whether to deliver the item at the location.

18. The computer-implemented method of claim 17, wherein the delivery plan comprises a coarse positioning portion, a fine positioning portion, and a delivery decision portion.

19. The computer-implemented method of claim 18, wherein the information about the gesture is received while the unmanned device is executing the delivery decision portion of the delivery plan.

20. The computer-implemented method of claim 17, wherein the delivery plan is adjusted when the comparison of the information about the gesture with the information about the one or more other gestures of the plurality of gestures results in a match.