Predicting and learning carrier phrases for speech input

• US 9,412,360 B2
• Filed: 04/15/2014
• Issued: 08/09/2016
• Est. Priority Date: 06/07/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Byrne, William J., Gruenstein, Alexander H., Beeferman, Douglas H.
• Primary Examiner(s): Sirjani, Fariba
• Application Number:
  US14/252,913
• Publication Number:
  US 20140229185A1
• Time in Patent Office: 847 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 2015/0638 Interactive procedures
• CPC Class Codes: G10L 2015/221 Announcement of recognition results

Abstract

Predicting and learning users'"'"' intended actions on an electronic device based on free-form speech input. Users'"'"' actions can be monitored to develop a list of carrier phrases having one or more actions that correspond to the carrier phrases. A user can speak a command into a device to initiate an action. The spoken command can be parsed and compared to a list of carrier phrases. If the spoken command matches one of the known carrier phrases, the corresponding action(s) can be presented to the user for selection. If the spoken command does not match one of the known carrier phrases, search results (e.g., Internet search results) corresponding to the spoken command can be presented to the user. The actions of the user in response to the presented action(s) and/or the search results can be monitored to update the list of carrier phrases.

Claims

1. A computer-implemented method comprising:

receiving first speech data that corresponds to a first voice command input by a user;

generating, by an automated speech recognizer, a transcription of the first speech data that corresponds to the first voice command input by the user;

identifying, from the transcription of the first speech data that corresponds to the first voice command input by the user, (i) a candidate carrier phrase, and (ii) one or more terms that reference a particular subject;

determining that a transcription of second speech data that corresponds to a second voice command subsequently input by the user includes the same one or more terms that reference the particular subject and does not include the candidate carrier phrase;

in response to determining that the transcription of the second speech data that corresponds to the second voice command includes the same one or more terms that reference the particular subject and does not include the candidate carrier phrase, providing, for output, a search results page that includes (i) one or more results of an Internet search query in which the one or more terms that reference the particular subject are used as query terms, and (ii) an actuatable link for performing an action with respect to the particular subject;

receiving a selection of the actuatable link for performing the action with respect to the particular subject;

in response to receiving the selection of the actuatable link for performing the action with respect to the particular subject, adjusting a relevance score for the candidate carrier phrase and the action to indicate that the candidate carrier phrase has increased relevance to the action; and

using, by an automated voice command processing engine, the relevance score to determine whether the action is to be performed upon subsequently identifying the candidate carrier phrase from a transcription of third speech data.

2. The computer-implemented method of claim 1, comprising:

identifying, from the transcription of the second speech data, a second, different candidate carrier phrase; and

in response to receiving the selection of the actuatable link for performing the action with respect to the particular subject, adjusting a relevance score for the second, different candidate carrier phrase and the action to indicate that the second, different candidate carrier phrase has increased relevance to the action.

3. The computer-implemented method of claim 1, wherein the second voice command that corresponds to the second voice command subsequently input by the user is input after the first voice command input by the user without the user inputting an intervening voice command.

4. The computer-implemented method of claim 3, comprising:

after determining that the transcription of the second speech data that corresponds to the second voice command subsequently input by the user includes the same one or more terms that reference the particular subject and does not include the candidate carrier phrase, logging one or more operations performed at a client device by the user;

identifying the action based on the log of the one or more operations performed at the client device by the user; and

in response to identifying the action based on the log of the one or more operations performed at the client device by the user, adjusting the relevance score for the candidate carrier phrase and the action to indicate that the candidate carrier phrase has increased relevance to the action.

5. The computer-implemented method of claim 1, comprising:

in response to providing, for output, the search results page that includes (i) the one or more results of the Internet search query in which the one or more terms that reference the particular subject are used as query terms, and (ii) the actuatable link for performing the action with respect to the particular subject, receiving data indicating that the action is not the intended action to be performed with respect to the particular subject.

6. The computer-implemented method of claim 1, comprising:

in response to receiving the selection of the actuatable link for performing the action with respect to the particular subject, adjusting a relevance score for a second, different action and the candidate carrier phrase to indicate that the candidate carrier phrase has decreased relevance to the second, different action.

7. The computer-implemented method of claim 1, comprising:

in response to receiving the selection of the actuatable link for performing the action with respect to the particular subject, performing the action with respect to the particular subject.

8. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving first speech data that corresponds to a first voice command input by a user;

generating, by an automated speech recognizer, a transcription of the first speech data that corresponds to the first voice command input by the user;

identifying, from the transcription of the first speech data that corresponds to the first voice command input by the user, (i) a candidate carrier phrase, and (ii) one or more terms that reference a particular subject;

determining that a transcription of second speech data that corresponds to a second voice command subsequently input by the user includes the same one or more terms that reference the particular subject and does not include the candidate carrier phrase;

in response to determining that the transcription of the second speech data that corresponds to the second voice command includes the same one or more terms that reference the particular subject and does not include the candidate carrier phrase, providing, for output, a search results page that includes (i) one or more results of an Internet search query in which the one or more terms that reference the particular subject are used as query terms, and (ii) an actuatable link for performing an action with respect to the particular subject;

receiving a selection of the actuatable link for performing the action with respect to the particular subject;

in response to receiving the selection of the actuatable link for performing the action with respect to the particular subject, adjusting a relevance score for the candidate carrier phrase and the action to indicate that the candidate carrier phrase has increased relevance to the action; and

using, by an automated voice command processing engine, the relevance score to determine whether the action is to be performed upon subsequently identifying the candidate carrier phrase from a transcription of third speech data.

9. The system of claim 8, wherein the operations comprise:

identifying, from the transcription of the second speech data, a second, different candidate carrier phrase; and

in response to receiving the selection of the actuatable link for performing the action with respect to the particular subject, adjusting a relevance score for the second, different candidate carrier phrase and the action to indicate that the second, different candidate carrier phrase has increased relevance to the action.

10. The system of claim 8, wherein the second voice command that corresponds to the second voice command subsequently input by the user is input after the first voice command input by the user without the user inputting an intervening voice command.

11. The system of claim 10, wherein the operations comprise:

after determining that the transcription of the second speech data that corresponds to the second voice command subsequently input by the user includes the same one or more terms that reference the particular subject and does not include the candidate carrier phrase, logging one or more operations performed at a client device by the user;

identifying the action based on the log of the one or more operations performed at the client device by the user; and

in response to identifying the action based on the log of the one or more operations performed at the client device by the user, adjusting the relevance score for the candidate carrier phrase and the action to indicate that the candidate carrier phrase has increased relevance to the action.

12. The system of claim 8, wherein the operations comprise:

in response to providing, for output, the search results page that includes (i) the one or more results of the Internet search query in which the one or more terms that reference the particular subject are used as query terms, and (ii) the actuatable link for performing the action with respect to the particular subject, receiving data indicating that the action is not the intended action to be performed with respect to the particular subject.

13. The system of claim 8, wherein the operations comprise:

in response to receiving the selection of the actuatable link for performing the action with respect to the particular subject, adjusting a relevance score for a second, different action and the candidate carrier phrase to indicate that the candidate carrier phrase has decreased relevance to the second, different action.

14. The system of claim 8, wherein the operations comprise:

in response to receiving the selection of the actuatable link for performing the action with respect to the particular subject, performing the action with respect to the particular subject.

15. A computer-readable device storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving first speech data that corresponds to a first voice command input by a user;

generating, by an automated speech recognizer, a transcription of the first speech data that corresponds to the first voice command input by the user;

identifying, from the transcription of the first speech data that corresponds to the first voice command input by the user, (i) a candidate carrier phrase, and (ii) one or more terms that reference a particular subject;

determining that a transcription of second speech data that corresponds to a second voice command subsequently input by the user includes the same one or more terms that reference the particular subject and does not include the candidate carrier phrase;

in response to determining that the transcription of the second speech data that corresponds to the second voice command includes the same one or more terms that reference the particular subject and does not include the candidate carrier phrase, providing, for output, a search results page that includes (i) one or more results of an Internet search query in which the one or more terms that reference the particular subject are used as query terms, and (ii) an actuatable link for performing an action with respect to the particular subject;

receiving a selection of the actuatable link for performing the action with respect to the particular subject;

in response to receiving the selection of the actuatable link for performing the action with respect to the particular subject, adjusting a relevance score for the candidate carrier phrase and the action to indicate that the candidate carrier phrase has increased relevance to the action; and

using, by an automated voice command processing engine, the relevance score to determine whether the action is to be performed upon subsequently identifying the candidate carrier phrase from a transcription of third speech data.

16. The computer-readable device of claim 15, wherein the operations comprise:

identifying, from the transcription of the second speech data, a second, different candidate carrier phrase; and

in response to receiving the selection of the actuatable link for performing the action with respect to the particular subject, adjusting a relevance score for the second, different candidate carrier phrase and the action to indicate that the second, different candidate carrier phrase trigger has increased relevance to the action.

17. The computer-readable device of claim 15, wherein the second voice command that corresponds to the second voice command subsequently input by the user is input after the first voice command input by the user without the user inputting an intervening voice command.

18. The computer-readable device of claim 17, wherein the operations comprise:

after determining that the transcription of the second speech data that corresponds to the second voice command subsequently input by the user includes the same one or more terms that reference the particular subject and does not include the candidate carrier phrase, logging one or more operations performed at a client device by the user;

identifying the action based on the log of the one or more operations performed at the client device by the user; and

in response to identifying the action based on the log of the one or more operations performed at the client device by the user, adjusting the relevance score for the candidate carrier phrase and the action to indicate that the candidate carrier phrase has increased relevance to the action.

19. The computer-readable device of claim 15, wherein the operations comprise:

in response to providing, for output, the search results page that includes (i) the one or more results of the Internet search query in which the one or more terms that reference the particular subject are used as query terms, and (ii) the actuatable link for performing the action with respect to the particular subject, receiving data indicating that the action is not the intended action to be performed with respect to the particular subject.

20. The computer-readable device of claim 15, wherein the operations comprise:

in response to receiving the selection of the actuatable link for performing the action with respect to the particular subject, adjusting a relevance score for a second, different action and the candidate carrier phrase to indicate that the candidate carrier phrase has decreased relevance to the second, different action.