Developer voice actions system

• US 10,008,203 B2
• Filed: 09/07/2016
• Issued: 06/26/2018
• Est. Priority Date: 04/22/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Wang, Bo, Vemuri, Sunil, Shetti, Nitin Mangesh, Gupta, Pravir Kumar, Huffman, Scott B., Rey, Javier Alejandro, Boortz, Jeffrey A.
• Primary Examiner(s): JACKSON, JAKIEDA R
• Application Number:
  US15/258,084
• Publication Number:
  US 20170186427A1
• Time in Patent Office: 657 Days
• Field of Search: 704275, 704235
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/0638 Interactive procedures
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Methods, systems, and apparatus for receiving data identifying an application and a voice command trigger term, validating the received data, inducting the received data to generate an intent that specifies the application, the voice command trigger term, and one or more other voice command trigger terms that are determined based at least on the voice command trigger term, and storing the intent at a contextual intent database, wherein the contextual intent database comprises one or more other intents.

Claims

1. A computer-implemented method comprising:

receiving data specifying a new voice action, the data identifying (i) an application, (ii) a voice command trigger phrase for triggering the application, and (iii) a context that must be satisfied for the application to be triggered;

generating a data structure instance that specifies (i) the application, (ii) the voice command trigger phrase, and (iii) an alternate voice command trigger phrase, the alternate voice command trigger phrase being based on the received voice command trigger phrase; and

after generating the data structure instance, enabling triggering of the new voice action by a spoken utterance based at least on storing the data structure instance at a database that comprises a plurality of other data structure instances, wherein each of the other data structure instances specifies (i) an application, and (ii) one or more voice command trigger phrases;

determining that the context is satisfied;

after enabling the triggering of the new voice action and based at least on determining that a transcription of a spoken utterance includes the alternate voice command trigger phrase specified by the generated data structure instance, selecting the generated data structure instance from the database; and

based on the selection of the generated data structure instance and based on the determination that the context is satisfied, causing an activity associated with the application specified by the generated data structure to be performed on or by the application.

2. The computer implemented method of claim 1, wherein the context defines a predefined version of the application that must exist on computing device that provides the spoken utterance that is transcribed.

3. The computer implemented method of claim 1, wherein the context defines particular software that must exist on a computing device that provides the spoken utterance that is transcribed.

4. The computer implemented method of claim 1, wherein the context defines particular software that must be absent from a computing device that provides the spoken utterance that is transcribed.

5. The computer implemented method of claim 1, wherein the context defines at least one of a time of day, a location of a computing device that provides the spoken utterance that is transcribed, proximity of a computing device that provides the spoken utterance that is transcribed to an item or to a person carrying an item.

6. The computer-implemented method of claim 1, wherein the received voice command trigger phrase includes a placeholder term that may be replaced by one or more other terms, and wherein generating the alternate voice command trigger phrase includes determining an alternate voice command trigger phrase that includes a variant of the placeholder term.

7. The computer-implemented method of claim 1, further comprising validating the received data by determining that the voice command trigger phrase is not associated with a second application that is different from the application identified in the received data.

8. The computer-implemented method of claim 1, further comprising:

validating the received data by determining whether the context identified in the received data can be satisfied.

9. The computer-implemented method of claim 1, further comprising:

providing the alternate voice command trigger phrase to a developer of the application; and

receiving a confirmation from the application developer that the alternate voice command trigger phrase should be specified by the generated data structure instance.

10. The computer-implemented method of claim 1, wherein generating the data structure instance includes generating the alternate voice command trigger phrase by applying to the voice command trigger phrase identified by the received data one or more of:

a synonym rule, an optionalization rule, or an expansion rule.

11. The computer-implemented method of claim 1, comprising:

a. identifying a plurality of data structure instances in the database that each specify an application, and the alternate voice command trigger phrase;

b. requesting a selection of a particular application from among a set of applications that includes the applications specified by the plurality of identified data structure instances;

c. receiving an indication of a selection of a particular application from among the set of applications; and

d. in response to receiving the indication, adjusting a strength of a relationship between the identified data structure instance that specifies the particular application and the alternate voice command trigger phrase.

12. A computer-implemented method comprising:

receiving data specifying a new voice action to add to a database, the data identifying (i) an application and (ii) a voice command trigger phrase for triggering the application;

generating, prior to enabling triggering of the new voice action, an alternate voice command trigger phrase by applying to the voice command trigger phrase identified by the received data one or more of;

a synonym rule, an optimization rule, or an expansion rule;

generating a data structure instance that specifies (i) the application, (ii) the voice command trigger phrase, and (iii) the alternate voice command trigger phrase; and

after generating the data structure instance, enabling triggering of the new voice action by a spoken utterance based at least on storing the data structure instance in the database on a computer-readable storage device, the database including a plurality of other data structure instances, wherein each of the other data structure instances specifies (i) an application, and (ii) one or more voice command trigger phrases;

after enabling the triggering of the new voice action, using the alternate voice command trigger phrase specified by the generated data structure instance to determine whether to perform the new voice action based on a spoken utterance; and

based on determining to perform the voice action based on the spoken utterance, causing an activity associated with the application specified by the generated data structure to be performed on or by the application.

13. A computer system comprising:

at least one processor; and

memory storing instructions that, when executed by the at least one processor, causes the system to;

receive data specifying a new voice action, the data identifying (i) an application, (ii) a voice command trigger phrase for triggering the application, and (iii) a context that must be satisfied for the application to be triggered;

generate a data structure instance that specifies (i) the application, (ii) the voice command trigger phrase, and (iii) an alternate voice command trigger phrase, the alternate voice command trigger phrase being based on the received voice command trigger phrase; and

after generating the data structure instance, enable triggering of the new voice action by a spoken utterance based at least on storing the data structure instance at a database that comprises a plurality of other data structure instances, wherein each of the other data structure instances specifies (i) an application, and (ii) one or more voice command trigger phrases;

determine that the context is satisfied;

after enabling the triggering of the new voice action and based at least on determining that a transcription of a spoken utterance includes the alternate voice command trigger phrase specified by the generated data structure instance, select the generated data structure instance from the database; and

based on the selection of the generated data structure instance and based on the determination that the context is satisfied, cause an activity associated with the application specified by the generated data structure to be performed on or by the application.

14. The computer system of claim 13, wherein the context defines a predefined version of the application that must exist on a computing device that provides the spoken utterance that is transcribed.

15. The computer system of claim 13, wherein the context defines particular software that must exist on a computing device that provides the spoken utterance that is transcribed.

16. The computer system of claim 13, wherein the context defines particular software that must be absent from a computing device that provides the spoken utterance that is transcribed.

17. The computer system of claim 13, wherein the context defines at least one of a time of day, a location of a computing device that provides the spoken utterance that is transcribed, proximity of a computing device that provides the spoken utterance that is transcribed to an item or to a person carrying an item.

18. The computer system of claim 13, wherein the received voice command trigger phrase includes a placeholder term that may be replaced by one or more other terms, and wherein generating the alternate voice command trigger phrase includes determining an alternate voice command trigger phrase that includes a variant of the placeholder term.

19. The computer system of claim 13, further comprising validating the received data by determining that the voice command trigger phrase is not associated with a second application that is different from the application identified in the received data.

20. The computer system of claim 13, the memory further storing instructions that, when executed by the at least one processor, causes the system to:

identify a plurality of data structure instances in the database that each specify (i) an application, and the alternate voice command trigger phrase;

request a selection of a particular application from among a set of applications that includes the applications specified by the plurality of identified data structure instances;

receive an indication of a selection of a particular application from among the set of applications; and

in response to receiving the indication, adjust a strength of a relationship between the identified data structure instance that specifies the particular application and the alternate voice command trigger phrase.