Evaluating pronouns in context

• US 8,606,568 B1
• Filed: 10/23/2012
• Issued: 12/10/2013
• Est. Priority Date: 10/10/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Cohen, Richard Z., Tickner, Simon
• Primary Examiner(s): Saint Cyr, Leonard
• Application Number:
  US13/658,110
• Time in Patent Office: 413 Days
• Field of Search: None
• US Class Current: 704/231
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Methods, computer program products, and systems are described for receiving, by a speech recognition engine, audio data that encodes an utterance and determining, by the speech recognition engine, that a transcription of the utterance includes one or more keywords associated with a command, and a pronoun. In addition, the methods, computer program products, and systems described herein pertain to transmitting a disambiguation request to an application, wherein the disambiguation request identifies the pronoun, receiving, by the speech recognition engine, a response to the disambiguation request, wherein the response references an item of content identified by the application, and generating, by the speech recognition engine, the command using the keywords and the response.

Claims

1. A computer-implemented method comprising:

receiving, by a speech recognition engine, data indicating a selection of a control for initiating speech recognition that is presented by a referring application;

receiving, by a speech recognition engine, audio data that encodes an utterance;

determining, by the speech recognition engine, that a transcription of the utterance includes one or more keywords associated with a command, and a pronoun;

transmitting, from the speech recognition engine to the referring application, a disambiguation request, wherein the disambiguation request identifies the pronoun;

receiving, by the speech recognition engine, a response to the disambiguation request from the referring application, wherein the response references an item of content identified by the application; and

generating, by the speech recognition engine, the command using the keywords and the response.

2. The method of claim 1, further comprising:

transmitting one or more additional disambiguation requests to a disambiguation engine;

receiving, by the speech recognition engine, a response to the one or more additional disambiguation requests;

merging results from the one or more additional disambiguation requests and the disambiguation request;

evaluating the merged results to determine a semantic connection between the referring application and the command; and

generating a mapping of the referring application to the command.

3. The method of claim 2, further comprising weighting the merged results and generating a set of weighted results.

4. The method of claim 1, wherein generating the command comprises accessing predetermined rules pertaining to one or more keywords associated with the pronoun.

5. The method of claim 1, wherein the response to the disambiguation request includes a GPS location and a user identifier.

6. The method of claim 5, wherein the speech recognition engine employs one or more predetermined rules based on the GPS location and the user identifier.

7. The method of claim 1, further comprising:

in response to determining that a transcription of the utterance includes one or more keywords associated with the command and the pronoun, selecting an application, other than the referring application, to receive the disambiguation request; and

transmitting the disambiguation request to the selected application.

9. The method of claim 7, wherein:

the disambiguation request further identifies a command associated with the pronoun, wherein the item of content is identified further based on the command.

10. The method of claim 9, wherein:

the item of content is identified based at least in part on the command and a predetermined rule set.

11. The method of claim 10, wherein the predetermined rule set pertains to the application.

8. A computer-implemented method comprising:

determining, by a referring application, that a control for initiating speech recognition that is presented by the referring application has been selected;

receiving, by the referring application, a disambiguation request from a speech recognition engine, wherein the disambiguation request identifies a pronoun;

identifying, by the referring application, an item of content based on the pronoun; and

transmitting, by the referring application, a response to the disambiguation request, wherein the response references the item of content.

12. A non-transitory computer-readable medium having instructions encoded thereon, which, when executed by a processor, cause the processor to perform operations comprising:

receiving, by a speech recognition engine, data indicating a selection of a control for initiating speech recognition that is presented by a referring application;

receiving, by a speech recognition engine, audio data that encodes an utterance;

determining, by the speech recognition engine, that a transcription of the utterance includes one or more keywords associated with a command, and a pronoun;

transmitting, from the speech recognition engine to the referring application, a disambiguation request, wherein the disambiguation request identifies the pronoun;

receiving, by the speech recognition engine, a response to the disambiguation request from the referring application, wherein the response references an item of content identified by the application; and

generating, by the speech recognition engine, the command using the keywords and the response.

13. The non-transitory medium of claim 12, further comprising:

transmitting one or more additional disambiguation requests to a disambiguation engine;

receiving, by the speech recognition engine, a response to the one or more additional disambiguation requests;

merging results from the one or more additional disambiguation requests and the disambiguation request;

evaluating the merged results to determine a semantic connection between the referring application and the command; and

generating a mapping of the referring application to the command.

14. The non-transitory medium of claim 13, further comprising weighting the merged results and generating a set of weighted results.

15. The non-transitory medium of claim 12, wherein generating the command comprises accessing predetermined rules pertaining to one or more keywords associated with the pronoun.

16. The non-transitory medium of claim 12, wherein the response to the disambiguation request includes a GPS location and a user identifier.

17. The non-transitory medium of claim 16, wherein the speech recognition engine employs one or more predetermined rules based on the GPS location and the user identifier.

18. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, by a speech recognition engine, data indicating a selection of a control for initiating speech recognition that is presented by a referring application;

receiving, by a speech recognition engine, audio data that encodes an utterance;

determining, by the speech recognition engine, that a transcription of the utterance includes one or more keywords associated with a command, and a pronoun;

transmitting, from the speech recognition engine to the referring application, a disambiguation request, wherein the disambiguation request identifies the pronoun;

receiving, by the speech recognition engine, a response to the disambiguation request from the referring application, wherein the response references an item of content identified by the application; and

generating, by the speech recognition engine, the command using the keywords and the response.

19. The system of claim 18, further comprising:

transmitting one or more additional disambiguation requests to a disambiguation engine;

receiving, by the speech recognition engine, a response to the one or more additional disambiguation requests;

merging results from the one or more additional disambiguation requests and the disambiguation request;

evaluating the merged results to determine a semantic connection between the referring application and the command; and

generating a mapping of the referring application to the command.

20. The system of claim 19, further comprising weighting the merged results and generating a set of weighted results.

21. The system of claim 18, wherein generating the command comprises accessing predetermined rules pertaining to one or more keywords associated with the pronoun.

22. The system of claim 18, wherein the response to the disambiguation request includes a GPS location and a user identifier.

23. The system of claim 22, wherein the speech recognition engine employs one or more predetermined rules based on the GPS location and the user identifier.