AUTOMATICALLY PROVIDING CONTENT ASSOCIATED WITH CAPTURED INFORMATION, SUCH AS INFORMATION CAPTURED IN REAL-TIME

• US 20110043652A1
• Filed: 03/12/2010
• Published: 02/24/2011
• Est. Priority Date: 03/12/2009
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: King, Martin T., Smith, Michael J., Stephens, Redwood, Sanvitale, Mark, Daley-Watson, Christopher J., Peterson, Jesse, Mannby, Claes-Fredrik
• Granted Patent: US 8,990,235 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: G06F 16/332 Query formulation
• CPC Class Codes: G06F 40/194 Calculation of difference between files
• CPC Class Codes: G06F 40/197 Version control for software G06F8/71

Abstract

A system and method for automatically providing content associated with captured information is described. In some examples, the system receives input by a user, and automatically provides content or links to content associated with the input. In some examples, the system receives input via text entry or by capturing text from a rendered document, such as a printed document, an object, an audio stream, and so on.

Claims

1. A method of providing relevant information to a user that is performed by one or more first computing devices, each first computing device including a processor and a memory, the method comprising:

while receiving text provided by the user, repeatedly automatically performing the following by at least one of the one or more first computing devices;

monitoring the received text;

selecting a portion of the text;

forming a query based at least in part upon the selected portion of the text;

transmitting, without a specific request by the user for the transmitting, the query to at least one of one or more second computing devices;

receiving information relevant to the query from at least one of the one or more second computing devices; and

displaying, by at least one of the one or more first computing devices, the relevant information.

2. The method of claim 1 wherein receiving text provided by the user includes receiving text provided by the user as part of generating a document.

3. The method of claim 1, further comprising:

selecting at least one index to search; and

selecting the one or more second computing devices based at least in part upon the selected index.

4. The method of claim 1, wherein receiving text provided by the user includes receiving text optically captured by the one or more first computing devices.

5. The method of claim 1, wherein forming the query based at least in part upon the selected portion of the text includes forming a query that includes words other than the words in the selected portion of the text.

6. The method of claim 1, wherein forming the query based at least in part upon the selected portion of the text includes forming a first query associated with the selected portion of the text and forming a second query associated with text surrounding the selected portion of the text.

7. The method of claim 1, wherein selecting a portion of the text includes optically capturing text printed on a rendered document by the one or more first computing devices;

and

wherein forming the query based at least in part upon the selected portion of the text includes identifying the rendered document from the captured text and forming the query based at least in part upon the identified document.

8. The method of claim 1, wherein the text includes a sentence and wherein selecting a portion of the text includes identifying multiple components of the sentence and selecting at least one of the multiple components as the selected portion of the text.

9. A computer-readable medium whose contents cause one or more first computing devices to perform a method of providing relevant information, the method comprising:

monitoring receiving first information;

analyzing the first information;

based upon the analysis, forming a query;

transmitting, without having received an express request, the query to one or more indices;

receiving second information relevant to the first information from the one or more indices; and

providing the second information by the information output device of at least one of the one or more first computing devices.

10. The computer-readable medium of claim 9, wherein the first information is received by a word processing application.

11. A method in a mobile device for displaying content associated with a rendered document, the method comprising:

capturing a portion of text from a rendered document;

identifying the rendered document based on the captured text;

determining that the rendered document is associated with one or more electronic content sources; and

presenting content from the associated one or more electronic content sources.

12. The method of claim 11, wherein determining that the rendered document is associated with one or more electronic content sources includes identifying a tag associated with the identified rendered document and locating content sources that include content related to the identified tag.

13. The method of claim 11, wherein capturing a portion of text from a rendered document includes optically capturing the portion of text using an imaging component of the mobile device;

and

wherein presenting content from the associated one or more electronic content sources includes displaying via display component of the mobile device a graphical element representing the associated one or more electronic content sources.

14. The method of claim 11, wherein determining that the rendered document is associated with one or more electronic content sources includes automatically determining the one or more electronic content sources in response to the identification of the rendered document.

15. A system in a mobile device for interacting with an electronic content source, the system comprising:

an optical capture component, wherein the optical capture component is programmed to capture information from a rendered document using an imaging component of the mobile device;

a document identification component, wherein the document identification component is programmed to identify the rendered document based on the captured information; and

a content identification component, wherein the content identification component is programmed to automatically identify a content source that is associated with the identified rendered document in response to information received from the document identification component that indicates the rendered document is identified.

16. The system of claim 15, further comprising:

a content source component, wherein the content source component is programmed to update the identified content source based on input received from a user of the mobile device.

17. A method for providing supplemental information in response to audio information, the method comprising:

receiving audio information, wherein the audio information includes spoken information;

converting the spoken information into text;

processing the text to identify at least one content term; and

,

obtaining supplementary information based on the content term.

18. The method of claim 17, wherein the audio information is received in real time via a microphone, wherein the processing includes generating and submitting to a web-based search engine a query, and wherein the method includes providing for display at least of some supplementary information received from the web-based search engine in response to the query.

19. A computer-readable medium whose contents, when executed by a computing system, cause the computing system to perform a method for performing an action associated with an audio stream, the method comprising:

capturing information from the audio stream wherein the captured information includes spoken content;

identifying an action to perform based on the captured information, wherein the action is associated with the information from the audio stream or a source of the audio stream; and

performing the action.

20. The computer-readable medium of claim 19, wherein identifying an action to perform based on the captured information includes identifying visual content associated with the captured information;

and

wherein performing the action includes displaying the identified visual content via a display associated with the computing system.

21. The computer-readable medium of claim 19, wherein identifying an action to perform based on the captured information includes identifying content associated with the captured information;

and

wherein performing the action includes storing the identified content in a log associated with the audio stream that relates positions in time in the audio stream with identified content.