Choosing recognized text from a background environment

• US 9,015,043 B2
• Filed: 10/01/2010
• Issued: 04/21/2015
• Est. Priority Date: 10/01/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Jenson, Scott
• Primary Examiner(s): Saint Cyr, Leonard
• Application Number:
  US12/896,507
• Publication Number:
  US 20120084312A1
• Time in Patent Office: 1,663 Days
• Field of Search: 704/235, 704/246, 704/247, 704/251, 704/252
• US Class Current: 704/235
• CPC Class Codes: G06F 16/90324 using system suggestions

Abstract

A computer-implemented method includes receiving an electronic representation of one or more human voices, recognizing words in a first portion of the electronic representation of the one or more human voices, and sending suggested search terms to a display device for display to a user in a text format. The suggested search terms are based on the recognized words in the first portion of the electronic representation of the one or more human voices. A search query is received from the user, which includes one or more of the suggested search terms that were displayed to the user.

Claims

1. A computer-implemented method comprising:

receiving an electronic representation of a conversation between a plurality of human voices;

recognizing one or more words in a first portion of the electronic representation of the conversation;

recognizing one or more words in a second portion of the electronic representation of the conversation, wherein the second portion is different from the first portion, wherein the first portion has a duration from a first time to a second time, and wherein the second portion has a duration from a third time to a fourth time, the third time being after the first time and before the second time;

selecting search terms for a search query, wherein the search terms include;

at least one word from the one or more recognized words in the first portion that is selected as a search term based on whether the word is a proper noun, based on an inverse of a frequency of the word in a corpus of documents, and based on a number of times that the word is recognized in the first portion of the electronic representation of the conversation,

at least one word that was not spoken in the conversation and is related to a word that was spoken in the conversation, and

one or more words recognized in the second portion of the electronic representation of the conversation;

causing the search terms to be displayed on a display device in a text format;

receiving a search query that includes at least one of the search terms.

2. The method of claim 1, wherein the electronic representation of the conversation includes an audio file.

3. The method of claim 2, further comprising receiving an indication that the first portion of the electronic representation of the conversation begins at a first time of the audio file and ends a second time of the audio file.

4. The method of claim 1, wherein the first portion of the electronic representation of the conversation includes a portion of a continuous stream of audio data.

5. The method of claim 1, wherein the search terms are displayed in a hypertext format to facilitate selection of the at least one of the search terms used to formulate the search query.

6. The method of claim 5, wherein the display device comprises a touchscreen, wherein the at least one of the search terms is selected to formulate the search query by tapping on the at least one of the search terms.

7. The method of claim 1,

wherein the electronic representation of the conversation is generated by a mobile computing device,

wherein receiving the electronic representation of the conversation includes receiving the electronic representation at a server computer, and

wherein recognizing the one or more words in the electronic representation of the conversation includes recognizing the words through operation of a voice recognition operation performed at the server computer.

8. The method of claim 1, wherein the search terms are displayed in a format that includes indicia used to convey a significance of each search term.

9. The method of claim 8, wherein the indicia used to convey the significance comprises a size of the search term relative to other search terms.

10. The method of claim 1, further comprising sending to a server computing device the search query, the search query including at least one of the search terms.

11. The method of claim 10, further comprising receiving results based on the search query executed by the server computing device.

12. The method of claim 1, wherein the first portion and the second portion have a duration of a predetermined time period.

13. The method of claim 12, further comprising determining the duration of the predetermined time period based on input from an input interface.

14. The method of claim 1, wherein the first portion and the second portion include a predetermined number of recognized words.

15. The method of claim 14, further comprising determining the predetermined number of recognized words based on input from an input interface.

16. The method of claim 1, further comprising:

recognizing words in a series of different portions of the electronic representation of the conversation, and

causing the recognized words in the series of different portions to be displayed on the display device in near real time during the conversation.

17. The method of claim 1, wherein selecting the search terms for the search query is further based on whether the word was spoken more recently in the conversation.

18. The method of claim 1, wherein selecting the search terms for the search query is further based on a time at which the word was spoken in the conversation, wherein words spoken more recently are displayed more prominently on the display device.

19. A computing system, comprising:

a microphone configured to facilitate generating an electronic representation of a conversation between a plurality of human voices;

a transmitter configured to transmit, to a server computing system, a first portion of the electronic representation of the conversation between the plurality of human voices and a second portion of the electronic representation of the conversation, wherein the second portion is different from the first portion, wherein the first portion has a duration from a first time to a second time, and wherein the second portion has a duration from a third time to a fourth time, the third time being after the first time and before the second time;

a receiver configured to receive search terms from the server computing device, wherein the search terms include;

one or more words recognized in the first portion of the electronic representation of the conversation based on whether the word is a proper noun, based on an inverse of a frequency of the word in a corpus of documents, and based on a number of times that the word is recognized in the first portion of electronic representation of the conversation,

one or more words that was not spoken in the conversation and is related to a word that was spoken in the conversation, and

one or more words recognized in the second portion of the electronic representation of the conversation;

a display device configured to display the search terms in a text format; and

an input interface configured for receiving a selection of at least one of the search terms displayed on the display device.

20. The system of claim 19, wherein the electronic representation of the conversation includes an audio file.

21. The system of claim 20, wherein the input interface is further configured for receiving an indication that the first portion of the electronic representation begins at a first time of the audio file and ends a second time of the audio file.

22. The system of claim 19, wherein the first portion of the electronic representation of the conversation includes a portion of a continuous stream of audio data.

23. The system of claim 19, wherein the text format includes a hypertext format to facilitate selection of the at least one of the search terms to formulate a search query.

24. The system of claim 23, wherein the display device includes a touchscreen allowing selection of the at least one of the search terms displayed in a hypertext format to formulate the search query by tapping on the search term.

25. The system of claim 19, wherein the display device is configured to display the search terms in a first text format that includes indicia used to convey a significance of a search term relative to the other search terms.

26. The system of claim 19, wherein the transmitter is further configured to transmit to the server computing system a search query that includes the selection of the at least one of the search terms.

27. A server computing system, comprising:

a receiver configured to receive an electronic representation of a conversation between a plurality of human voices;

a voice recognition engine configured to recognize one or more words in a first portion of the electronic representation of the conversation and one or more words in a second portion of the electronic representation of the conversation, wherein the second portion is different from the first portion, wherein the first portion has a duration from a first time to a second time, and wherein the second portion has a duration from a third time to a fourth time, the third time being after the first time and before the second time;

a query term generation engine configured to select search terms for a search query, wherein the search terms include;

at least one of the one or more recognized words in the first portion that is selected as a search term based on whether the word is a proper noun, based on an inverse of a frequency of the word in a corpus of documents, and based on a number of times that the word is recognized in the first portion of the electronic representation of the conversation,

at least one word that was not spoken in the conversation and is related to a word that was spoken in the conversation, and

one or more words recognized in the second portion of the electronic representation of the conversation;

a transmitter configured to transmit the search terms to a client computing device for display in a text format on a display device; and

a query execution engine configured to receive a search query from the client computing device that includes at least one of the search terms that were displayed on the display device and to query a database for information matching the search query.

28. The server computing system of claim 27, wherein the electronic representation of the conversation includes an audio file.

29. The server computing system of claim 27, wherein the first portion of the electronic representation of the conversation includes a portion of a continuous stream of audio data.

30. The server computing system of claim 27, further comprising:

a storage medium configured to store an index of the corpus of documents; and

wherein the query execution engine is configured to query the index for information matching the search query received from the client computing device.