ANSWERING QUESTIONS USING ENVIRONMENTAL CONTEXT

• US 20140074466A1
• Filed: 09/25/2012
• Published: 03/13/2014
• Est. Priority Date: 09/10/2012
• Status: Abandoned Application

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Sharifi, Matthew, Postelnicu, Gheorghe
• Application Number:
  US13/626,439
• Publication Number:
  US 20140074466A1
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/686 using information manually generated, e.g. tags, keywords, comments, title or artist information, time, location or usage information, user ratings
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/225 Feedback of the input speech

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving audio data encoding an utterance and environmental data, obtaining a transcription of the utterance, identifying an entity using the environmental data, submitting a query to a natural language query processing engine, wherein the query includes at least a portion of the transcription and data that identifies the entity, and obtaining one or more results of the query.

Claims

1. A computer-implemented method comprising:

receiving audio data encoding (i) an utterance and (ii) background audio data;

obtaining a transcription of the utterance;

identifying an entity using the background audio data;

submitting a query to a natural language query processing engine, wherein the query includes at least a portion of the transcription and data that identifies the entity that is identified using the background audio data; and

obtaining one or more results of the query.

2. The computer-implemented method of claim 1, further comprising outputting a representation of at least one of the results.

3. The computer-implemented method of claim 1, wherein the entity is identified further using the utterance.

4. The computer-implemented method of claim 1, further comprising generating the query.

5. The computer-implemented method of claim 4, wherein generating the query comprises associating the transcription with the data that identifies the entity.

6. The computer-implemented method of claim 5, wherein associating further includes tagging the transcription with the data that identifies the entity.

7. The computer-implemented method of claim 5, wherein associating further includes substituting a portion of the transcription with the data that identifies the entity.

8. The computer-implemented method of claim 7, wherein substituting further includes substituting one or more words of the transcription with the data that identifies the entity.

23. The computer-implemented method of claim 1, further comprising:

providing, to a query generation engine, the transcription of the utterance and the data that identifies the entity; and

receiving, from the query generation engine, the query based on providing the transcription of the utterance and the data that identifies the entity to the query generation engine.

24. The computer-implemented method of claim 1, further comprising:

determining, for each result of the one or more results, a score based on a matching between the query and the result;

comparing, for each result of the one or more results, the score to a threshold score; and

based on comparing the score to the threshold score, identifying one or more particular results of the one or more results associated with a score that satisfied the threshold score.

9-10. -10. (canceled)

11. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, from a computing device, audio data encoding an utterance recorded by the computing device, and image data encoding an image captured by the computing device;

obtaining a transcription of the utterance;

identifying an entity using the image data;

submitting a query to a natural language query processing engine, wherein the query includes at least a portion of the transcription and data that identifies the entity that is identified using the image data; and

obtaining one or more results of the query.

12. The system of claim 11, the operations further include generating the query, wherein generating the query comprises associating the transcription with the data that identifies the entity.

13. The system of claim 12, wherein associating further includes tagging the transcription with the data that identifies the entity.

14. The system of claim 12, wherein associating further includes substituting a portion of the transcription with the data that identifies the entity.

15. The system of claim 14, wherein substituting further includes substituting one or more words of the transcription with the data that identifies the entity.

16-17. -17. (canceled)

18. A computer-readable storage device storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving (i) audio data encoding an utterance, and (ii) environmental data;

obtaining a transcription of the utterance;

identifying a title of an item of media content using the environmental data;

submitting a query to a natural language query processing engine, wherein the query includes at least a portion of the transcription and data that identifies the title of the item of media content that is identified using the environmental data; and

obtaining one or more results of the query.

19. The computer-readable storage device of claim 18, the operations further comprise generating the query, wherein generating the query comprises associating the transcription with the data that identifies the entity.

20. The computer-readable storage device of claim 19, wherein associating further includes tagging the transcription with the data that identifies the entity.

21. The computer-readable storage device of claim 19, wherein associating further includes substituting a portion of the transcription with the data that identifies the entity.

22. The computer-readable storage device of claim 21, wherein substituting further includes substituting one or more words of the transcription with the data that identifies the entity.