Developer voice actions system

• US 9,472,196 B1
• Filed: 04/22/2015
• Issued: 10/18/2016
• Est. Priority Date: 04/22/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Wang, Bo, Vemuri, Sunil, Shetti, Nitin Mangesh, Huffman, Scott B., Rey, Javier Alejandro, Boortz, Jeffrey A., Gupta, Pravir Kumar
• Primary Examiner(s): Jackson, Jakieda
• Application Number:
  US14/693,330
• Publication Number:
  US 20160314791A1
• Time in Patent Office: 545 Days
• Field of Search: 704/275, 704/235
• US Class Current: 1/1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/0638 Interactive procedures
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Methods, systems, and apparatus for receiving data identifying an application and a voice command trigger term, validating the received data, inducting the received data to generate an intent that specifies the application, the voice command trigger term, and one or more other voice command trigger terms that are determined based at least on the voice command trigger term, and storing the intent at a contextual intent database, wherein the contextual intent database comprises one or more other intents.

Claims

1. A computer-implemented method comprising:

receiving data specifying a new voice action submitted by an application developer, the data identifying (i) an application, and (ii) a voice command trigger term;

validating the received data;

generating a data structure instance that specifies (i) the application, (ii) the voice command trigger term, and (iii) one or more alternate voice command trigger terms that are each determined based at least on the voice command trigger term;

after generating the data structure instance, enabling triggering of the new voice action by a spoken utterance based at least on storing the data structure instance at a database that comprises one or more other data structure instances, wherein one or more of the other data structure instances specify (i) an application, and (ii) one or more voice command trigger terms that includes at least one voice command trigger term that is determined based at least on another voice command trigger term in the same data structure instance;

after enabling triggering of the new voice action by a spoken utterance and based at least on determining that a transcription of a spoken utterance includes a particular one of the alternate voice command trigger terms that are each determined based at least on the voice command trigger term, selecting a particular data structure instance from the database that specifies the particular one of the alternate voice command trigger terms that are each determined based at least on the voice command trigger term; and

identifying a particular application that is specified by the particular data structure instance.

2. The computer-implemented method of claim 1, wherein validating the received data comprises determining that the voice command trigger term is not associated with a second application that is different from the application.

3. The computer-implemented method of claim 1, comprising receiving data submitted by the application developer, the data identifying an action, wherein validating the received data comprises determining that the action is compatible with the application.

4. The computer-implemented method of claim 1, wherein generating the data structure instance that specifies (i) the application, (ii) the voice command trigger term, and (iii) the one or more alternate voice command trigger terms that are each determined based at least on the voice command trigger term comprises:

translating the received data to a different data format.

5. The computer-implemented method of claim 1, wherein generating the data structure instance that specifies (i) the application, (ii) the voice command trigger term, and (iii) the one or more alternate voice command trigger terms that are each determined based at least on the voice command trigger term comprises:

generating one or more alternate voice command trigger terms by applying one or more synonym rules, optionalization rules, or expansion rules to the voice command trigger term.

6. The computer-implemented method of claim 1, comprising:

based on identifying the particular application that is specified by the particular data structure instance, causing a task to be performed.

7. The computer-implemented method of claim 6, wherein causing the task to be performed comprises causing a particular application that is specified by the particular data structure instance to perform the task.

8. The computer-implemented method of claim 1, comprising:

identifying one or more data structure instances in the database that each specify (i) an application, and the particular one of the alternate voice command trigger terms that are each determined based at least on the voice command trigger term;

requesting a selection of a particular application from among a set of applications that includes the applications specified by the one or more identified data structure instances;

receiving data indicating a selection of a particular application from among the set of applications that includes the applications specified by the one or more identified data structure instances; and

in response to receiving the data indicating the selection of the particular application, adjusting a strength of a relationship between the identified data structure instance that specifies the particular application and the particular one of the alternate voice command trigger terms that are each determined based at least on the voice command trigger term.

9. The computer-implemented method of claim 1, comprising:

identifying one or more data structure instances in the contextual intent database that each specify (i) an application, and the particular one of the alternate voice command trigger terms that are each determined based at least on the voice command trigger term;

determining, for each of the one or more identified data structure instances, an affinity measure for the data structure instance that indicates an affinity of the particular one of the alternate voice command trigger terms that are each determined based at least on the voice command trigger term with the application specified by the data structure instance;

selecting, based at least on the affinity measures for each of the one or more identified data structure instances, a particular data structure instance; and

causing a task specified by the particular data structure instance to be performed.

10. The computer-implemented method of claim 1, comprising:

identifying one or more data structure instances in the contextual intent database that each specify (i) an application, and the particular one of the alternate voice command trigger terms that are each determined based at least on the voice command trigger term;

ranking the one or more identified data structure instances;

selecting a particular data structure instance from among the one or more identified data structure instances based at least on the ranking of the one or more identified data structure instances; and

causing a task specified by the particular data structure instance to be performed.

11. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving data specifying a new voice action submitted by an application developer, the data identifying (i) an application, and (ii) a voice command trigger term;

validating the received data;

generating a data structure instance that specifies (i) the application, (ii) the voice command trigger term, and (iii) one or more alternate voice command trigger terms that are each determined based at least on the voice command trigger term;

after generating the data structure instance enabling triggering of the new voice action by a spoken utterance based at least on storing the data structure instance at a database that comprises one or more other data structure instances, wherein one or more of the other data structure instances specify (i) an application, and (ii) one or more voice command trigger terms that includes at least one voice command trigger term that is determined based at least on another voice command trigger term in the same data structure instance;

after enabling triggering of the new voice action by a spoken utterance and based at least on determining that a transcription of a spoken utterance includes a particular one of the alternate voice command trigger terms that are each determined based at least on the voice command trigger term, selecting a particular data structure instance from the database that specifies the particular one of the alternate voice command trigger terms that are each determined based at least on the voice command trigger term; and

identifying a particular application that is specified by the particular data structure instance.

12. The system of claim 11, wherein the operations comprise receiving data submitted by the application developer, the data identifying an action, wherein validating the received data comprises determining that the action is compatible with the application.

13. The system of claim 11, wherein generating the data structure instance that specifies (i) the application, (ii) the voice command trigger term, and (iii) the one or more alternate voice command trigger terms that are each determined based at least on the voice command trigger term comprises:

translating the received data to a different data format.

14. The system of claim 11, wherein generating the data structure instance that specifies (i) the application, (ii) the voice command trigger term, and (iii) the one or more alternate voice command trigger terms that are each determined based at least on the voice command trigger term comprises:

generating one or more alternate voice command trigger terms by applying one or more synonym rules, optionalization rules, or expansion rules to the voice command trigger term.

15. The system of claim 11, wherein the operations comprise:

based on identifying the particular application that is specified by the particular data structure instance, causing a task to be performed.

16. The system of claim 11, wherein the operations comprise:

identifying one or more data structure instances in the database that each specify (i) an application, and the particular one of the alternate voice command trigger terms that are each determined based at least on the voice command trigger term;

requesting a selection of a particular application from among a set of applications that includes the applications specified by the one or more identified data structure instances;

receiving data indicating a selection of a particular application from among the set of applications that includes the applications specified by the one or more identified data structure instances; and

in response to receiving the data indicating the selection of the particular application, adjusting a strength of a relationship between the identified data structure instance that specifies the particular application and the particular one of the alternate voice command trigger terms that are each determined based at least on the voice command trigger term.

17. A non-transitory computer-readable storage device storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving data specifying a new voice action submitted by an application developer, the data identifying (i) an application, and (ii) a voice command trigger term;

validating the received data;

generating a data structure instance that specifies (i) the application, (ii) the voice command trigger term, and (iii) one or more alternate voice command trigger terms that are each determined based at least on the voice command trigger term;

after generating the data structure instance, enabling triggering of the new voice action by a spoken utterance based at least on storing the data structure instance at a database that comprises one or more other data structure instances, wherein one or more of the other data structure instances specify (i) an application, and (ii) one or more voice command trigger terms that includes at least one voice command trigger term that is determined based at least on another voice command trigger term in the same data structure instance;

after enabling triggering of the new voice action by a spoken utterance and based at least on determining that a transcription of a spoken utterance includes a particular one of the alternate voice command trigger terms that are each determined based at least on the voice command trigger term, selecting a particular data structure instance from the database that specifies the particular one of the alternate voice command trigger terms that are each determined based at least on the voice command trigger term; and

identifying a particular application that is specified by the particular data structure instance.

18. The computer-readable device of claim 17, wherein generating the data structure instance that specifies (i) the application, (ii) the voice command trigger term, and (iii) the one or more alternate voice command trigger terms that are each determined based at least on the voice command trigger term comprises:

generating one or more alternate voice command trigger terms by applying one or more synonym rules, optionalization rules, or expansion rules to the voice command trigger term.

19. The computer-readable device of claim 17, wherein the operations comprise:

based on identifying the particular application that is specified by the particular data structure instance, causing a task to be performed.

20. The computer-readable device of claim 17, wherein the operations comprise:

identifying one or more data structure instances in the database that each specify (i) an application, and the particular one of the alternate voice command trigger terms that are each determined based at least on the voice command trigger term;

requesting a selection of a particular application from among a set of applications that includes the applications specified by the one or more identified data structure instances;

receiving data indicating a selection of a particular application from among the set of applications that includes the applications specified by the one or more identified data structure instances; and

in response to receiving the data indicating the selection of the particular application, adjusting a strength of a relationship between the identified data structure instance that specifies the particular application and the particular one of the alternate voice command trigger terms that are each determined based at least on the voice command trigger term.