Predicting and learning carrier phrases for speech input

• US 8,738,377 B2
• Filed: 06/07/2010
• Issued: 05/27/2014
• Est. Priority Date: 06/07/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Byrne, William J., Beeferman, Douglas, Gruenstein, Alexander H.
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Sirjani, Fariba
• Application Number:
  US12/795,257
• Publication Number:
  US 20110301955A1
• Time in Patent Office: 1,450 Days
• Field of Search: 704/9, 704/10, 704/231, 704/246, 704/251, 704/257
• US Class Current: 704/251
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 2015/0638 Interactive procedures
• CPC Class Codes: G10L 2015/221 Announcement of recognition results

Abstract

Predicting and learning users'"'"' intended actions on an electronic device based on free-form speech input. Users'"'"' actions can be monitored to develop of a list of carrier phrases having one or more actions that correspond to the carrier phrases. A user can speak a command into a device to initiate an action. The spoken command can be parsed and compared to a list of carrier phrases. If the spoken command matches one of the known carrier phrases, the corresponding action(s) can be presented to the user for selection. If the spoken command does not match one of the known carrier phrases, search results (e.g., Internet search results) corresponding to the spoken command can be presented to the user. The actions of the user in response to the presented action(s) and/or the search results can be monitored to update the list of carrier phrases.

Claims

1. A computer-implemented method comprising:

identifying a particular term from a transcription of speech data;

determining that (i) the particular term is not included as a key term in a grammar that references key terms that correspond to an action, and (ii) the particular term is semantically similar to a particular key term that is referenced in the grammar as corresponding to the action;

receiving data validating that the particular term corresponds to the action; and

in response to determining that (i) the particular term is not included as a key term in the grammar, and (ii) the particular term is semantically similar to the particular key term that is referenced in the grammar as corresponding to the action, and further in response to receiving the data validating that the particular term corresponds to the action, adjusting, by one or more computers, (a) a relevance score for the action and the particular key term to indicate that the particular term that is not included as a key term in the grammar has increased relevance to the action, and (b) a relevance score for a different action and a key term associated with the different action to indicate that the particular term has decreased relevance to the different action.

2. The computer-implemented method of claim 1, comprising:

in response to receiving the data validating that the particular term corresponds to the action, performing the action associated with the grammar.

3. The computer-implemented method of claim 1, comprising:

in response to determining that (i) the particular term is not included as a key term in the grammar, and (ii) the particular term is semantically similar to the particular key term that is referenced in the grammar as corresponding to the action, and before receiving data validating that the particular term corresponds to the action, providing a graphical output to a user that requests input from the user validating that the particular term corresponds to the action.

4. The computer-implemented method of claim 3, wherein the graphical output includes a hyperlink, and wherein the input from the user comprises a user selection of the hyperlink.

5. The computer-implemented method of claim 1, wherein the data validating that the particular term corresponds to the action indicates that the particular term corresponds to the action and comprising:

adding the particular term as a key term in the grammar as a term that corresponds to the action; and

after adjusting the relevance score for the action and the particular key term to indicate that the particular term that is not included as a key term in the grammar has increased relevance to the action;

identifying the particular term from a transcription of second speech data;

determining that the particular term is included as a key term in the grammar that references key terms that correspond to the action; and

performing the action associated with the grammar.

6. The computer-implemented method of claim 1, wherein the data validating that the particular term corresponds to the action comprises information about the relevance of the particular term to the action associated with the grammar.

7. The computer-implemented method of claim 6, further comprising:

identifying the particular term from a transcription of second speech data;

determining that the relevance score for the particular term exceeds a threshold value; and

in response to determining that the relevance score for the particular term exceeds the threshold value, performing the action associated with the grammar.

8. The method of claim 1, further comprising:

before receiving data validating that the particular term corresponds to the action, determining that the relevance score for the action and the particular key term satisfies a relevance score threshold; and

based on determining that the relevance score for the action and the particular key term satisfies the relevance score threshold, performing the action associated with the grammar.

23. The computer-implemented method of claim 1, wherein adjusting, by one or more computers, the relevance score for the action and the particular key term to indicate that the particular term that is not included as a key term in the grammar has increased relevance to the action comprises:

increasing the relevance score for the action and the particular key term to indicate that the particular term that is not included as a key term in the grammar has increased relevance to the action, after the data validating that the particular term corresponds to the action indicates that the particular term corresponds to the action.

9. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

identifying a particular term from a transcription of speech data;

determining that (i) the particular term is not included as a key term in a grammar that references key terms that correspond to an action, and (ii) the particular term is semantically similar to a particular key term that is referenced in the grammar as corresponding to the action;

receiving data validating that the particular term corresponds to the action; and

in response to determining that (i) the particular term is not included as a key term in the grammar, and (ii) the particular term is semantically similar to the particular key term that is referenced in the grammar as corresponding to the action, and further in response to receiving the data validating that the particular term corresponds to the action, adjusting (a) a relevance score for the particular action and the particular key term to indicate that the particular term that is not included as a key term in the grammar has increased relevance to the action and (b) a relevance score for a different action and a key term associated with the different action to indicate that the particular term has decreased relevance to the different action.

10. The system of claim 9, wherein the operations comprise:

in response to receiving the data validating that the particular term corresponds to the action, performing the action associated with the grammar.

11. The system of claim 9, wherein the operations comprise:

in response to determining that (i) the particular term is not included as a key term in the grammar, and (ii) the particular term is semantically similar to the particular key term that is referenced in the grammar as corresponding to the action, and before receiving data validating that the particular term corresponds to the action, providing a graphical output to a user that requests input from the user validating that the particular term corresponds to the action.

12. The system of claim 11, wherein the graphical output includes a hyperlink, and wherein the input from the user comprises a user selection of the hyperlink.

13. The system of claim 9, wherein the data validating that the particular term corresponds to the action indicates that the particular term corresponds to the action and wherein the operations further comprise:

adding the particular term as a key term in the grammar as a term that corresponds to the action; and

after adjusting the relevance score for the action and the particular key term to indicate that the particular term that is not included as a key term in the grammar has increased relevance to the action;

identifying the particular term from a transcription of second speech data;

determining that the particular term is included as a key term in the grammar that references key terms that correspond to the action; and

performing the action associated with the grammar.

14. The system of claim 9, wherein the data validating that the particular term corresponds to the action comprises information about the relevance of the particular term to the action associated with the grammar.

15. The system of claim 14, wherein the operations further comprise:

identifying the particular term from a transcription of second speech data;

determining that the relevance score for the particular term exceeds a threshold value; and

in response to determining that the relevance score for the particular term exceeds the threshold value, performing the action associated with the grammar.

16. A non-transitory computer-readable medium storing software comprising instructions executable on one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

identifying a particular term from a transcription of speech data;

determining that (i) the particular term is not included as a key term in a grammar that references key terms that correspond to an action, and (ii) the particular term is semantically similar to a particular key term that is referenced in the grammar as corresponding to the action;

receiving data validating whether that the particular term corresponds to the action; and

in response to determining that (i) the particular term is not included as a key term in the grammar, and (ii) the particular term is semantically similar to the particular key term that is referenced in the grammar as corresponding to the action, and further in response to receiving the data validating that the particular term corresponds to the action, adjusting (a) a relevance score for the particular action and the particular key term to indicate that the particular term that is not included as a key term in the grammar has increased relevance to the action, and (b) a relevance score for a different action and a key term associated with the different action to indicate that the particular term has decreased relevance to the different action.

17. The computer-readable medium of claim 16, wherein the operations comprise:

in response to receiving the data validating that the particular term corresponds to the action, performing the action associated with the grammar.

18. The computer-readable medium of claim 16, wherein the operations comprise:

in response to determining that (i) the particular term is not included as a key term in the grammar, and (ii) the particular term is semantically similar to the particular key term that is referenced in the grammar as corresponding to the action, and before receiving data validating that the particular term corresponds to the action, providing a graphical output to a user that requests input from the user validating that the particular term corresponds to the action.

19. The computer-readable medium of claim 18, wherein the graphical output includes a hyperlink, and wherein the input from the user comprises a user selection of the hyperlink.

20. The computer-readable medium of claim 16, wherein the data validating that the particular term corresponds to the action indicates that the particular term corresponds to the action and wherein the operations further comprise:

adding the particular term as a key term in the grammar as a term that corresponds to the action; and

after adjusting the relevance score for the action and the particular key term to indicate that the particular term that is not included as a key term in the grammar has increased relevance to the action;

identifying the particular term from a transcription of second speech data;

determining that the particular term is included as a key term in the grammar that references key terms that correspond to the action; and

performing the action associated with the grammar.

21. The computer-readable medium of claim 16, wherein the data validating that the particular term corresponds to the action comprises information about the relevance of the particular term to the action associated with the grammar.

22. The computer-readable medium of claim 21, wherein the operations further comprise:

identifying the particular term from a transcription of second speech data;

determining that the relevance score for the particular term exceeds a threshold value; and

in response to determining that the relevance score for the particular term exceeds the threshold value, performing the action associated with the grammar.

24. A computer-implemented method comprising:

identifying a particular term from a transcription of speech data;

determining that the particular term is not included as a key term in a grammar that references key terms that correspond to an action;

providing at least a portion of the transcription of the speech data as a search query to a search engine;

receiving one or more search results responsive to providing the at least the portion of the transcription of the speech data as a search query to the search engine;

receiving data indicating a user selection of a particular search result from among the one or more search results;

determining that an action associated with the particular search result selected by the user corresponds to the action associated with the grammar; and

in response to determining that the action corresponding to the particular search result selected by the user corresponds to the action associated with the grammar, adjusting, by one or more computers, (a) a relevance score for the action and the particular key term to indicate that the particular term that is not included as a key term in the grammar has increased relevance to the action, and (b) a relevance score for a different action and a key term associated with the different action to indicate that the particular term has decreased relevance to the different action.