Speech interaction for unmanned aerial vehicles

• US 9,747,901 B1
• Filed: 02/27/2015
• Issued: 08/29/2017
• Est. Priority Date: 02/27/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Gentry, Nicholas Kristofer
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Lewandroski, Sara
• Application Number:
  US14/634,578
• Time in Patent Office: 914 Days
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/06 for indicating aircraft presence
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 2101/60 for transporting passengers; for transporting goods other than weapons
• CPC Class Codes: B64U 2101/64 for parcel delivery or retrieval
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

An unmanned aerial vehicle (UAV) may be used for delivering products or other articles. The UAV is configured to detect the presence of nearby people, animals, or other interactive objects. Upon detecting a nearby object, the UAV may produce speech in order to warn or instruct the object. The UAV may also have speech input capabilities in order to capture and respond to speech from the object. The UAV may conduct a speech dialog with a nearby person in order to request information and/or answer questions from the object. In certain situations, the UAV may detect whether an object is in the way of a desired landing area and may communicate with the object through speech to ask the object to move or to ask the object to specify an alternative landing area.

Claims

1. An unmanned aerial vehicle (UAV) comprising:

an article support configured to carry an article to a delivery destination;

one or more sensors configured to detect an interactive object;

a loudspeaker configured to produce audio output;

one or more processors;

one or more non-transitory computer-readable media storing computer-executable instructions that, when executed on the one or more processors, cause the UAV to perform actions comprising;

detecting the interactive object at the delivery destination using the one or more sensors;

determining whether the interactive object is within an unsafe distance of an article drop site at the delivery destination based on the detection;

when it is determined that the interactive object is within an unsafe distance from the article drop site, producing a first output speech using the loudspeaker, wherein the first output speech requests that the interactive object move away from the article drop site;

when it is determined that the interactive object is outside the unsafe distance from the article drop site, descending to the article drop site to deliver the article.

2. The UAV of claim 1, the actions further comprising:

detecting that the UAV is disabled at a ground location; and

producing second output speech using the loudspeaker, wherein the second output speech requests an interactive object to report that the UAV is disabled at a ground location.

3. The UAV of claim 1, further comprising:

a microphone configured to capture speech of the interactive object, wherein the captured speech comprises a question; and

the actions further comprising producing second output speech using the loudspeaker, wherein the second output speech comprises an answer to the question.

4. The UAV of claim 1, further comprising:

a microphone configured to capture speech of the interactive object; and

the actions further comprising conducting a speech dialog with the interactive object, wherein the speech dialog comprises one or more dialog turns, each dialog turn comprising speech of the interactive object and second output speech produced using the loudspeaker.

5. A method, comprising:

carrying an article to a delivery destination with an unmanned aerial vehicle (UAV);

detecting an interactive object in an article drop site at the delivery destination using one or more sensors on the UAV;

determining, by the UAV, whether the interactive object is within a predetermined distance from the article drop site based on the detection;

when it is determined that the interactive object is within the predetermined distance from the article drop site, producing, by the UAV, a first output speech using a loudspeaker, wherein the first output speech requests that the interactive object move away from the article drop site;

capturing input speech of the interactive object at the delivery destination using a microphone of the UAV;

analyzing the input speech; and

producing second output speech using a loudspeaker of the UAV, wherein the output speech is responsive to the analyzed input speech;

and

when it is determined that the interactive object is outside of the predetermined distance from the article drop site, descending, by the UAV, to the article drop site to deliver the article.

6. The method of claim 5, wherein analyzing the input speech comprises transmitting the captured input speech to a support service and receiving the responsive output speech from the support service.

7. The method of claim 5, wherein analyzing the input speech comprises performing speech recognition on an audio signal produced using the microphone of the UAV.

8. The method of claim 5, wherein analyzing the input speech comprises:

performing speech recognition on an audio signal produced using the microphone of the UAV to determine words of the input speech; and

performing natural language understanding on the words of the input speech.

9. The method of claim 5, further comprising transmitting the input speech to a customer service representative, wherein the second output speech comprises speech of the customer service representative.

10. The method of claim 5, further comprising:

initiating a speech dialog with the interactive object in response to detecting the interactive object at the delivery destination.

11. The method of claim 5, further comprising determining the article drop site based at least in part on the analyzed input speech.

12. The method of claim 5, further comprising performing voice recognition to determine whether the interactive object is an authorized recipient of the article.

13. The method of claim 5, further comprising performing face recognition to determine whether the interactive object is an authorized recipient of the article.

14. The method of claim 5, wherein the input speech comprises an inquiry regarding the article and the second output speech comprises an answer to the inquiry.

15. An unmanned aerial vehicle (UAV) comprising:

an article support configured to carry an article for delivery to a delivery destination;

a loudspeaker configured to produce audio output;

one or more processors;

one or more non-transitory computer-readable media storing computer-executable instructions that, when executed on the one or more processors, cause the UAV to perform actions comprising;

detecting an interactive object proximate to an article drop site at the delivery destination;

determining whether the interactive object is within a predetermined distance of the article drop site at the delivery destination based on the detection;

when it is determined that the interactive object is within the predetermined distance from the article drop site, producing output speech using the loudspeaker; and

when it is determined that the interactive object is outside of the predetermined distance from the article drop site, descending to the article drop site to deliver the article.

16. The UAV of claim 15, further comprising:

detecting that the UAV is ready to land or takeoff.

17. The UAV of claim 15, further comprising:

one or more sensors configured to detect the interactive object.

18. The UAV of claim 15,

wherein the output speech requests that the interactive object move away from the article drop site.

19. The UAV of claim 15, wherein the output speech warns the interactive object to stay at a safe distance from the UAV.

20. The UAV of claim 15, wherein the output speech asks the interactive object to report a location of the UAV.