User feedback for speech interactions

• US 10,460,719 B1
• Filed: 03/19/2018
• Issued: 10/29/2019
• Est. Priority Date: 01/11/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Belin, Gilles Jean Roger, Rogers, III, Charles S., Owen, Robert David, Adams, Jeffrey Penrod, Ramachandran, Rajiv, Hart, Gregory Michael
• Primary Examiner(s): Tzeng, Feng-Tzer
• Application Number:
  US15/925,397
• Time in Patent Office: 589 Days
• Field of Search: 704251, 704257, 704270, 704275, 704E15001
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/221 Announcement of recognition results

Abstract

An interactive system may be implemented in part by an audio device located within a user environment, which may accept speech commands from a user and may also interact with the user by means of generated speech. In order to improve performance of the interactive system, a user may use a separate device, such as a personal computer or mobile device, to access a graphical user interface that lists details of historical speech interactions. The graphical user interface may be configured to allow the user to provide feedback and/or corrections regarding the details of specific interactions.

Claims

1. A method comprising:

receiving, via one or more processors of a device, text corresponding to a prior utterance of a user;

presenting the text on a display associated with the device;

receiving, via an input component associated with the display, a request to output sound associated with the prior utterance;

sending a signal corresponding to the sound for output, the sound corresponding to the prior utterance;

receiving, via the input component associated with the display, feedback associated with the text; and

sending an indication of the feedback to one or more servers.

2. The method as recited in claim 1, wherein the feedback is received as a user comment via an input field presented on the display.

3. The method as recited in claim 1, further comprising:

receiving, via the input component associated with the display, a request to delete at least one prior utterance stored in a repository associated with the one or more servers; and

deleting the at least one prior utterance based at least in part on the request.

4. The method as recited in claim 1, further comprising:

receiving, via the input component associated with the display, a selection of the prior utterance; and

providing additional data for presentation on the display, the additional data associated with the prior utterance.

5. The method as recited in claim 1, further comprising:

updating, based at least in part on the indication of the feedback, at least one of a model for recognizing speech or a model for determining user intent.

18. The method as recited in claim 1, further comprising outputting the sound via one or more speakers that are associated with the device.

19. The method as recited in claim 1, wherein the device is a first device, the method further comprising:

outputting the sound via one or more speakers that are associated with a second device.

6. A non-transitory computer-readable medium storing instructions that, when executed, cause one or more processors to perform acts comprising:

receiving text corresponding to a prior utterance of a user;

providing the text for presentation on a display associated with a device;

receiving, via an input component associated with the display, a request to output sound associated with the prior utterance;

causing the sound corresponding to the prior utterance to be output;

receiving, via the input component associated with the display, feedback associated with the text; and

sending an indication of the feedback to one or more servers.

7. The non-transitory computer-readable medium as recited in claim 6, the acts further comprising:

providing additional data for presentation on the display, the additional data representing at least an identity of the user associated with the prior utterance, the identity of the user based at least in part on voice identification of the prior utterance.

8. The non-transitory computer-readable medium as recited in claim 6, the acts further comprising:

receiving the feedback as a user comment via an input field presented on the display.

9. The non-transitory computer-readable medium as recited in claim 6, the acts further comprising:

receiving, via the input component associated with the display, a request to delete at least one prior utterance stored in a repository associated with the one or more servers; and

deleting the at least one prior utterance based at least in part on the request.

10. The non-transitory computer-readable medium as recited in claim 6, wherein the text is displayed by an application that executes on a user device which is a mobile device of the user who uttered the prior utterance.

11. The non-transitory computer-readable medium as recited in claim 6, the acts further comprising:

providing for presentation on the display a plurality of prior utterances, the plurality of prior utterances presented in chronological order.

12. A system comprising:

one or more processors;

a memory; and

computer-executable instructions stored in the memory that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;

receiving text corresponding to a prior utterance of a user;

presenting the text on a display;

receiving, via an input component associated with the display, a request to output sound associated with the prior utterance;

outputting the sound associated with the prior utterance;

receiving, via the input component associated with the display, feedback associated with the text; and

sending an indication of the feedback to one or more servers.

13. The system as recited in claim 12, the acts further comprising:

receiving the indication of the feedback as a user comment via an input field presented on the display.

14. The system as recited in claim 12, wherein the text corresponding to the prior utterance is determined via automatic speech recognition.

15. The system as recited in claim 12, the acts further comprising:

receiving, via the input component associated with the display, a request to delete at least one prior utterance stored in a repository associated with the one or more servers; and

deleting the at least one prior utterance based at least in part on the request.

16. The system as recited in claim 12, the acts further comprising:

receiving, via the input component associated with the display, a selection of the prior utterance; and

providing additional data for presentation on the display, the additional data associated with the prior utterance.

17. The system as recited in claim 12, wherein the prior utterance is captured by a speech interface device in an environment of a user.

20. The system as recited in claim 12, wherein the sound corresponds to at least one of:

the prior utterance;

or

speech generated in response to the prior utterance.