Predicting and learning carrier phrases for speech input

• US 10,297,252 B2
• Filed: 07/05/2016
• Issued: 05/21/2019
• Est. Priority Date: 06/07/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Byrne, William J., Gruenstein, Alexander H., Beeferman, Douglas H.
• Primary Examiner(s): Sirjani, Fariba
• Application Number:
  US15/201,955
• Publication Number:
  US 20160314786A1
• Time in Patent Office: 1,050 Days
• Field of Search: None
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 2015/0638 Interactive procedures
• CPC Class Codes: G10L 2015/221 Announcement of recognition results

Abstract

Predicting and learning users'"'"' intended actions on an electronic device based on free-form speech input. Users'"'"' actions can be monitored to develop a list of carrier phrases having one or more actions that correspond to the carrier phrases. A user can speak a command into a device to initiate an action. The spoken command can be parsed and compared to a list of carrier phrases. If the spoken command matches one of the known carrier phrases, the corresponding action(s) can be presented to the user for selection. If the spoken command does not match one of the known carrier phrases, search results (e.g., Internet search results) corresponding to the spoken command can be presented to the user. The actions of the user in response to the presented action(s) and/or the search results can be monitored to update the list of carrier phrases.

Claims

1. A computer-implemented method comprising:

obtaining, by an automated speech recognizer implemented on one or more computing devices, a transcription of an utterance that includes a candidate carrier phrase;

determining that the candidate carrier phrase is not included among a set of known carrier phrases that are associated with one or more actions;

in response to determining that the candidate carrier phrase is not included among a set of known carrier phrases that are associated with one or more actions, performing, by a search engine, a search query in which the candidate carrier phrase is used as a query term;

providing, by the one or more computing devices, a user interface that includes one or more internet search results of the search query;

receiving, by the one or more computing devices, data indicating a selection, by a user, of a particular search result from among the one or more internet search results of the search query in which the candidate carrier phrase is used as a query term;

based on the data indicating the selection of the particular search result;

updating by the one or more computing devices, the set of known carrier phrases to include the candidate carrier phrase; and

associating, by the one or more computing devices, the candidate carrier phrase with a particular action corresponding to the particular, selected search result of the search query in which the candidate carrier phrase is used as a query term;

after associating the candidate carrier phrase with the particular action corresponding to the particular, selected search result of the search query in which the candidate carrier phrase is used as a query term, obtaining, by the automated speech recognizer, a transcription of, a subsequent utterance that includes the candidate carrier phrase; and

in response to obtaining the transcription of the subsequent utterance that includes the candidate carrier phrase, performing, by the one or more computing devices, the particular action.

2. The method of claim 1, wherein the particular action is performed in response to obtaining the transcription of the subsequent utterance, without requiring a speaker of the subsequent utterance to explicitly confirm that the particular action is to be performed.

3. The method of claim 1, wherein determining that the candidate carrier phrase is not included among a set of known carrier phrases that are associated with one or more actions comprises determining that the candidate carrier phrase is not identical to or semantically similar to any of the known carrier phrases of the set.

4. The method of claim 1, wherein providing the user interface that includes the one or more results of the search query comprises:

providing a user interface that includes (i) the one or more results of the search query and (ii) a selectable control that references a modified transcription of the voice command in which a synonym of the candidate carrier phrase is substituted for the candidate carrier phrase.

5. The method of claim 1, wherein associating the candidate carrier phrase with the particular action corresponding to the particular, selected search result comprises increasing a relevance score for the candidate carrier phrase with respect to the particular action.

6. The method of claim 1, wherein associating the candidate carrier phrase with the particular action corresponding to the particular, selected search result comprises decreasing a relevance score for the candidate carrier phrase with respect to one or more actions other than the particular action.

7. The method of claim 1, wherein the utterance includes a command, and candidate carrier phrase is a partial command that includes fewer than all of the words of the command.

8. The method of claim 1, wherein the utterance includes a command, and wherein the candidate carrier phrase includes an initial portion of the command and excludes one or more words at the end of the command.

9. The method of claim 1, wherein associating the candidate carrier phrase with the particular action comprises associating the candidate carrier phrase with the particular action for one or more users that are different from the user that spoke the utterance that includes the candidate carrier phrase.

10. A system comprising:

a processor configured to execute computer program instructions; and

one or more computer storage media encoded with one or more computer programs that, when executed by the processor, cause the system to perform operations comprising;

obtaining, by an automated speech recognizer implemented on one or more computing devices, a transcription of an utterance that includes a candidate carrier phrase;

determining that the candidate carrier phrase is not included among a set of known carrier phrases that are associated with one or more actions;

in response to determining that the candidate carrier phrase is not included among a set of known carrier phrases that are associated with one or more actions, performing, by a search engine, a search query in which the candidate carrier phrase is used as a query term;

providing, by the one or more computing devices, a user interface that includes one or more internet search results of the search query;

receiving, by the one or more computing devices, data indicating a selection, by a user, of a particular search result from among the one or more internet search results of the search query in which the candidate carrier phrase is used as a query term;

based on the data indicating the selection of the particular search result;

updating, by the one or more computing devices, the set of known carrier phrases to include the candidate carrier phrase; and

associating, by the one or more computing devices, the candidate carrier phrase with a particular action corresponding to the particular, selected search result of the search query in which the candidate carrier phrase is used as a query term;

after associating the candidate carrier phrase with the particular action corresponding to the particular, selected search result of the search query in which the candidate carrier phrase is used as a query term, obtaining, by the automated speech recognizer, a transcription of a subsequent utterance that includes the candidate carrier phrase; and

in response to obtaining the transcription of the subsequent utterance that includes the candidate carrier phrase, performing, by the one or more computing devices, the particular action.

11. The system of claim 10, wherein the particular action is performed in response to obtaining the transcription of the subsequent utterance, without requiring a speaker of the subsequent utterance to explicitly confirm that the particular action is to be performed.

12. The system of claim 10, wherein determining that the candidate carrier phrase is not included among a set of known carrier phrases that are associated with one or more actions comprises determining that the candidate carrier phrase is not identical to or semantically similar to any of the known carrier phrases of the set.

13. The system of claim 10, wherein providing the user interface that includes the one or more results of the search query comprises:

providing a user interface that includes (i) the one or more results of the search query and (ii) a selectable control that references a modified transcription of the voice command in which a synonym of the candidate carrier phrase is substituted for the candidate carrier phrase.

14. The system of claim 10, wherein associating the candidate carrier phrase with the particular action corresponding to the particular, selected search result comprises increasing a relevance score for the candidate carrier phrase with respect to the particular action.

15. One or more computer-readable devices storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

obtaining, by an automated speech recognizer implemented on one or more computing devices, a transcription of an utterance that includes a candidate carrier phrase;

determining that the candidate carrier phrase is not included among a set of known carrier phrases that are associated with one or more actions;

in response to determining that the candidate carrier phrase is not included among a set of known carrier phrases that are associated with one or more actions,

performing, by a search engine, a search query in which the candidate carrier phrase is used as a query term;

providing, by the one or more computing devices, a user interface that includes one or more internet search results of the search query;

receiving, by the one or more computing devices, data indicating a selection, by a user, of a particular search result from among the one or more internet search results of the search query in which the candidate carrier phrase is used as a query term;

based on the data indicating the selection of the particular search result;

updating, by the one or more computing devices, the set of known carrier phrases to include the candidate carrier phrase; and

associating, by the one or more computing devices, the candidate carrier phrase with a particular action corresponding to the particular, selected search result of the search query in which the candidate carrier phrase is used as a query term;

after associating the candidate carrier phrase with the particular action corresponding to the particular, selected search result of the search query in which the candidate carrier phrase is used as a query term, obtaining, by the automated speech recognizer, a transcription of a subsequent utterance that includes the candidate carrier phrase; and

in response to obtaining the transcription of the subsequent utterance that includes the candidate carrier phrase, performing, by the one or more computing devices, the particular action.

16. The one or more computer-readable devices of claim 15, wherein the particular action is performed in response to obtaining the transcription of the subsequent utterance, without requiring a speaker of the subsequent utterance to explicitly confirm that the particular action is to be performed.

17. The one or more computer-readable devices of claim 15, wherein determining that the candidate carrier phrase is not included among a set of known carrier phrases that are associated with one or more actions comprises determining that the candidate carrier phrase is not identical to or semantically similar to any of the known carrier phrases of the set.

18. The one or more computer-readable devices of claim 15, wherein providing the user interface that includes the one or more results of the search query comprises:

providing a user interface that includes (i) the one or more results of the search query and (ii) a selectable control that references a modified transcription of the voice command in which a synonym of the candidate carrier phrase is substituted for the candidate carrier phrase.