Outcome-oriented dialogs on a speech recognition platform

• US 10,482,884 B1
• Filed: 07/28/2017
• Issued: 11/19/2019
• Est. Priority Date: 12/17/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Beal, Jeff Bradley, Charter, Kevin Robert, Gopalakrishnan, Ajay, Kshirsagar, Sumedha Arvind, Kumar, Nishant
• Primary Examiner(s): Patel, Shreyans A
• Application Number:
  US15/663,514
• Time in Patent Office: 844 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A speech recognition platform configured to receive an audio signal that includes speech from a user and perform automatic speech recognition (ASR) on the audio signal to identify ASR results. The platform may identify: (i) a domain of a voice command within the speech based on the ASR results and based on context information associated with the speech or the user, and (ii) an intent of the voice command. In response to identifying the intent, the platform may perform multiple actions corresponding to this intent. The platform may select a target action to perform, and may engage in a back-and-forth dialog to obtain information for completing the target action. The action may include streaming audio to the device, setting a reminder for the user, purchasing an item on behalf of the user, making a reservation for the user or launching an application for the user.

Claims

1. A system comprising:

one or more processors; and

one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

receiving first audio data representing a request;

determining, based at least in part on the first audio data, first intent data representing a first intent associated with the request, the first intent associated with one or more slots;

determining, based at least in part on the first audio data, second intent data representing a second intent associated with the request;

determining a first number of values associated with the first intent;

determining a second number of values associated with the second intent;

selecting the first intent based at least in part on the first number of values being more favorable than the second number of values;

determining that a slot of the one or more slots is unfilled;

generating, based at least in part on the slot being unfilled, second audio data representing a query for additional information;

sending the second audio data to a device to output audio corresponding to the second audio data;

receiving third audio data representing a response to the query; and

associating a value with the slot based at least in part on the third audio data.

2. The system of claim 1, wherein generating the second audio data is based at least in part on contextual data associated with the request.

3. The system of claim 1, wherein the first intent is associated with a domain and generating the second audio data is based at least in part on an activity associated with the domain.

4. The system of claim 1, wherein generating the second audio data is based at least in part on an estimated latency associated with generating the second audio data.

5. The system of claim 1, wherein generating the second audio data is based at least in part on historical data indicating previously-generated audio data associated with the first intent.

6. A method comprising:

receiving first audio data representing a request;

determining, based at least in part on the first audio data, first intent data representing a first intent associated with the request, the first intent associated with one or more slots;

determining, based at least in part on the first audio data, second intent data representing a second intent associated with the request;

determining a first number of values associated with the first intent;

determining a second number of values associated with the second intent;

selecting the first intent based at least in part on the first number of values being more favorable than the second number of values;

determining that a slot of the one or more slots is unfilled;

generating, based at least in part on the slot being unfilled, second audio data representing a query for additional information;

sending the second audio data to a device to output audio corresponding to the second audio data;

receiving third audio data representing a response to the query; and

associating a value with the slot based at least in part on the third audio data.

7. The method of claim 6, wherein generating the second audio data is based at least in part on contextual data associated with the request.

8. The method of claim 7, wherein the contextual information includes previous audio data representing a previous request received from the device.

9. The method of claim 6, further comprising sending, to the device, fourth audio data representing an indication of an action to be performed by the device.

10. The method of claim 6, wherein generating the second audio data is based at least in part on an estimated latency associated with generating the second audio data.

11. The method of claim 6, wherein the first intent corresponds to a domain and generating the second audio data is based at least in part on an activity associated with the domain.

12. The method of claim 6, wherein the request comprises a request to at least one of set a reminder, play a media file, add an item to a list, provide a recommendation, purchase an item, or place a telephone call.

13. The method of claim 6, wherein generating the second audio data is based at least in part on a type of value associated with the slot.

14. The method of claim 6, wherein generating the second audio data is based at least in part on an indication that causing output of the audio corresponding to the second audio data is more likely to fill the slot with the value than taking one or more other actions.

15. A system comprising:

one or more processors; and

one or more computer-readable media including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

receiving first audio data representing a request;

determining, based at least in part on the first audio data and a context associated with the request, first intent data representing a first intent associated with the request, the first intent associated with one or more slots;

determining, based at least in part on the first audio data and the context associated with the request, second intent data representing a second intent associated with the request;

determining a first number of values associated with the first intent;

determining a second number of values associated with the second intent;

selecting the first intent based at least in part on the first number of values being more favorable than the second number of values;

determining that a slot of the one or more slots is unfilled;

generating, based at least in part on the slot being unfilled, second audio data representing a query for additional information;

sending the second audio data to a device to output audio corresponding to the second audio data;

receiving third audio data representing a response to the query; and

associating a value with the slot based at least in part on the third audio data.

16. The system of claim 15, wherein generating the second audio data is based at least in part on contextual data associated with the request.

17. The system of claim 15, wherein the context includes a location of the device.

18. The system of claim 15, wherein generating the second audio data is based at least in part on a confidence value indicating that the response to the query will identify the value.

19. The system of claim 18, wherein generating the second audio data is based at least in part on the confidence value being at least a threshold confidence value.

20. The system of claim 15, wherein the first intent is associated with a domain and generating the second audio data is based at least in part on an activity associated with the domain.