Dynamic speech processing

• US 10,515,637 B1
• Filed: 09/19/2017
• Issued: 12/24/2019
• Est. Priority Date: 09/19/2017
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Devries, David William, Mittal, Rajesh
• Primary Examiner(s): Roberts, Shaun
• Application Number:
  US15/709,119
• Time in Patent Office: 826 Days
• Field of Search: 704235, 704270
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/63 for estimating an emotional state

Abstract

Techniques for dynamically maintaining speech processing data on a local device for frequently input commands are described. A system determines a usage history associated with a user profile. The usage history represents at least a first command. The system determines the first command is associated with an input frequency that satisfies an input frequently threshold. The system also determines the first command is missing from first speech processing data stored by a device associated with the user profile. The system then generates second speech processing data specific to the first command and sends the second speech processing data to the device.

Claims

1. A method implemented by a computing system, comprising:

receiving, from a first device associated with at least one user profile, first data;

determining that the first data corresponds to a first command;

determining a unique identifier (ID) associated with the at least one user profile;

determining an updated spoken command history by updating, based at least in part on the first data corresponding to the first command, a spoken command history stored by the computing system and associated with the unique ID;

determining, using the updated spoken command history, that the first command is associated with an input frequency satisfying an input frequency threshold;

determining that the first command is not included in a first speech-recognition language model stored by the first device;

generating a second speech-recognition language model including second data sufficient to determine text corresponding to the first command;

sending, to the first device, the second speech-recognition language model; and

sending, to the first device, a first instruction to delete the first speech-recognition language model.

2. The method of claim 1, further comprising:

determining a portion of the second speech-recognition language model associated with a second command;

determining, using the updated spoken command history, that the second command is associated with a second input frequency falling below the input frequency threshold; and

sending, to the first device, a second instruction to delete the portion of the second speech-recognition language model associated with the second command.

3. The method of claim 1, further comprising:

determining that the first command corresponds to a first intent;

generating natural language processing data usable to determine the first intent from text corresponding to the first command; and

sending the natural language processing data to the first device.

4. The method of claim 1, further comprising:

receiving, from the first device, audio data corresponding to an utterance;

determining, using the updated spoken command history, that the utterance is associated with a second input frequency falling below the input frequency threshold;

performing speech processing on the input audio data to determine output data; and

sending the output data to the first device.

5. A computing system comprising:

at least one processor; and

at least one memory including instructions that, when executed by the at least one processor, cause the computing system to;

receive, from a first device associated with a user profile, first data;

determine that the first data corresponds to a first command;

determine an updated spoken command history by updating, based at least in part on the first data corresponding to the first command, a spoken command history stored by the computing system and associated with the user profile;

determine, using the updated spoken command history, that the first command is associated with an input frequency satisfying an input frequency threshold;

determine that the first command is at least in part not processable using first speech processing data stored by the first device;

generate second speech processing data including second data specific to the first command; and

send the second speech processing data to the first device.

6. The computing system of claim 5, wherein the at least one memory further includes additional instructions that, when executed by the at least one processor, further cause the system to:

send, to the first device, an instruction to delete the first speech processing data.

7. The computing system of claim 5, wherein the at least one memory further includes additional instructions that, when executed by the at least one processor, further cause the system to:

determine a second command represented in the second speech processing data;

determine, using the updated spoken command history, that the second command is associated with a second input frequency falling below the input frequency threshold; and

send, to the first device, an instruction to delete only a portion of the second speech processing data associated with the second command.

8. The computing system of claim 5, wherein the second speech processing data includes data for performing speech recognition.

9. The computing system of claim 5, wherein the at least one memory further includes additional instructions that, when executed by the at least one processor, further cause the system to:

determine that the first command corresponds to a first intent,

wherein the second speech processing data corresponds to natural language processing data specific to the first intent.

10. The computing system of claim 5, wherein the at least one memory further includes additional instructions that, when executed by the at least one processor, further cause the system to:

receive, from the first device, audio data corresponding to a second command;

determine, using the updated spoken command history, that the second command is associated with a second input frequency falling below the input frequency threshold;

perform speech processing on the audio data to determine output data; and

send the output data to the first device.

11. The computing system of claim 5, wherein the at least one memory further includes additional instructions that, when executed by the at least one processor, further cause the system to:

receive, from the first device, natural language processing results;

perform text-to-speech processing on the natural language processing results to generate output audio data; and

send the output audio data to the first device.

12. The computing system of claim 5, wherein the second speech processing data includes data for performing text-to-speech processing.

13. A method implemented by a computing system, comprising:

receiving, from a first device associated with a user profile, first data;

determining that the first data corresponds to a first command;

determining an updated spoken command history by updating, based at least in part on the first data corresponding to the first command, a spoken command history stored by the computing system and associated with the user profile;

determining, using the updated spoken command history, that the first command is associated with an input frequency satisfying an input frequency threshold;

determining that the first command is at least in part not processable using first speech processing data stored by the first device;

generating second speech processing data including second data specific to the first command; and

sending the second speech processing data to the first device.

14. The method of claim 13, further comprising

sending, to the first device, an instruction to delete the first speech processing data.

15. The method of claim 13, further comprising:

determining a second command represented in the second speech processing data;

determining, using the updated spoken command history, that the second command is associated with a second input frequency falling below the input frequency threshold; and

sending, to the first device, an instruction to delete only a portion of the second speech processing data associated with the second command.

16. The method of claim 13, wherein the second speech processing data includes data for performing speech recognition.

17. The method of claim 13, further comprising:

determining that the first command corresponds to a first intent,

wherein the second speech processing data corresponds to natural language processing data specific to the first intent.

18. The method of claim 13, further comprising:

receiving, from the first device, audio data corresponding to a second command;

determining, using the updated spoken command history, that the second command is associated with a second input frequency falling below the input frequency threshold;

performing speech processing on the input audio data to determine output data; and

sending the output data to the first device.

19. The method of claim 13, further comprising:

receiving, from the first device, natural language processing results;

performing text-to-speech processing on the natural language processing results to generate output audio data; and

sending the output audio data to the first device.

20. The method of claim 13, wherein the second speech processing data includes data for performing text-to-speech processing.