Triggering actions in response to optically or acoustically capturing keywords from a rendered document

• US 10,275,455 B2
• Filed: 03/17/2017
• Issued: 04/30/2019
• Est. Priority Date: 04/01/2004
• Status: Expired

• Current Assignee: Kyocera Corporation
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: King, Martin T., Grover, Dale L., Kushler, Clifford A., Stafford-Fraser, James Q.
• Primary Examiner(s): Couso, Jose L
• Application Number:
  US15/462,309
• Publication Number:
  US 20170192960A1
• Time in Patent Office: 774 Days
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/44 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06Q 30/0277 Online advertisement
• CPC Class Codes: G06V 30/153 using recognition of characters or words
• CPC Class Codes: G06V 30/412 Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
• CPC Class Codes: G06V 30/418 Document matching, e.g. of document images
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]

Abstract

A system for processing text captured from rendered documents is described. The system receives a sequence of one or more words optically or acoustically captured from a rendered document by a user. The system identifies among words of the sequence a word with which an action has been associated. The system then performs the associated action with respect to the user.

Claims

1. A computer-implemented method comprising:

receiving, from a mobile computing device that includes a capture device and is associated with an output device, a sequence of symbols optically captured from a rendered document by the capture device;

determining, by one or more processors, that the sequence of symbols includes a predetermined keyword or keyphrase;

selecting, by one or more processors, a content item that has been associated with the predetermined keyword or keyphrase, the content item being an electronic document that includes one or more of text, graphics, or audio, wherein selecting the content item comprises;

accessing a data structure that includes associations between content items, predetermined keywords or keyphrases, and conditions, the conditions including potential locations of mobile computing devices at which sequences of symbols are optically captured from rendered documents, each association in the data structure associating a content item with a different unique combination of predetermined keyword or keyphrase, and condition, wherein the predetermined keyword or keyphrase is associated with two or more different content items, each association being subject to a different condition;

determining a current condition that pertains to a location of the mobile computing device at which the sequence of symbols was optically captured from the rendered document, the current condition being different from the predetermined keyword or keyphrase included in the sequence of symbols; and

selecting the content item from among the two or more different content items, including selecting an identified content item that is associated in the data structure with the predetermined keyword or keyphrase and is associated in the data structure with the current condition that pertains to the mobile computing device; and

transmitting, to the mobile computing device, the selected content item that has been associated with the predetermined keyword or keyphrase and has been associated with the current condition that pertains to the mobile computing device, for presentation by the output device associated with the mobile computing device.

2. The computer-implemented method of claim 1, wherein the content item is identified by a link to a webpage that is transmitted to the mobile computing device.

3. The computer-implemented method of claim 1, wherein the output device includes one or more of a visual display or an audio speaker.

4. The computer-implemented method of claim 1, wherein the output device is a printer.

5. The computer-implemented method of claim 1, further comprising receiving, from the mobile computing device, an email address of a user of the mobile computing device, wherein the content item is transmitted to the mobile computing device using the email address.

6. The computer-implemented method of claim 1, wherein determining the current condition that pertains to the mobile computing device includes receiving, from the mobile computing device, location information that identifies a location of the mobile computing device at which the sequence of symbols was optically captured from the rendered document by the capture device, and wherein selecting the content item from among the two or more different content items includes selecting an identified content item that is associated in the data structure with the predetermined keyword or keyphrase and with the location of the mobile computing device at which the sequence of symbols was optically captured from the rendered document by the capture device.

7. The computer-implemented method of claim 1, wherein the conditions include potential times at which sequences of symbols are optically captured from rendered documents by capture devices of mobile computing devices, and wherein determining the current condition that pertains to the mobile device includes determining time information corresponding to a time at which the sequence of symbols was optically captured from the rendered document by the capture device of the mobile computing device, and wherein selecting the content item from among the two or more different content items includes selecting an identified content item that is associated in the data structure with the predetermined keyword or keyphrase and with the time at which the sequence of symbols was optically captured from the rendered document by the capture device of the mobile computing device.

8. The computer-implemented method of claim 1, wherein the conditions include potential rendered documents from which sequences of symbols are optically captured by capture devices of mobile computing devices, and wherein determining the current condition that pertains to the mobile computing device includes identifying the rendered document from which the sequence of symbols was optically captured by the capture device of the mobile computing device, and wherein selecting the content item from among the two or more different content items includes selecting an identified content item that is associated in the data structure with the predetermined keyword or keyphrase and with the rendered document from which the sequence of symbols was optically captured by the capture device of the mobile computing device.

9. A system, comprising:

one or more data processing apparatus; and

one or more computer-readable storage devices including instructions executable by the data processing apparatus and upon such execution cause the data processing apparatus to perform operations comprising;

receiving, from a mobile computing device that includes a capture device and is associated with an output device, a sequence of symbols optically captured from a rendered document by the capture device;

determining, by one or more processors, that the sequence of symbols includes a predetermined keyword or keyphrase;

selecting, by one or more processors, a content item that has been associated with the predetermined keyword or keyphrase, the content item being an electronic document that includes one or more of text, graphics, or audio, wherein selecting the content item comprises;

accessing a data structure that includes associations between content items, predetermined keywords or keyphrases, and conditions, the conditions including potential locations of mobile computing devices at which sequences of symbols are optically captured from rendered documents, each association in the data structure associating a content item with a different unique combination of predetermined keyword or keyphrase, and condition, wherein the predetermined keyword or keyphrase is associated with two or more different content items, each association being subject to a different condition;

determining a current condition that pertains to a location of the mobile computing device at which the sequence of symbols was optically captured from the rendered document, the current condition being different from the predetermined keyword or keyphrase included in the sequence of symbols; and

selecting the content item from among the two or more different content items, including selecting an identified content item that is associated in the data structure with the predetermined keyword or keyphrase and is associated in the data structure with the current condition that pertains to the mobile computing device; and

transmitting, to the mobile computing device, the selected content item that has been associated with the predetermined keyword or keyphrase and has been associated with the current condition that pertains to the mobile computing device, for presentation by the output device associated with the mobile computing device.

10. The system of claim 9, the operations further comprising receiving, from the mobile computing device, an email address of a user of the mobile computing device, wherein the content item is transmitted to the mobile computing device using the email address.

11. The system of claim 9, wherein determining the current condition that pertains to the mobile computing device includes receiving, from the mobile computing device, location information that identifies a location of the mobile computing device at which the sequence of symbols was optically captured from the rendered document by the capture device, and wherein selecting the content item from among the two or more different content items includes selecting an identified content item that is associated in the data structure with the predetermined keyword or keyphrase and with the location of the mobile computing device at which the sequence of symbols was optically captured from the rendered document by the capture device.

12. The system of claim 9, wherein the conditions include potential times at which sequences of symbols are optically captured from rendered documents by capture devices of mobile computing devices, and wherein determining the current condition that pertains to the mobile computing device includes determining time information corresponding to a time at which the sequence of symbols was optically captured from the rendered document by the capture device of the mobile computing device, and wherein selecting the content item from among the two or more different content items includes selecting an identified content item that is associated in the data structure with the predetermined keyword or keyphrase and with the time at which the sequence of symbols was optically captured from the rendered document by the capture device of the mobile computing device.

13. The system of claim 9, wherein the conditions include potential rendered documents from which sequences of symbols are optically captured by capture devices of mobile computing devices, and wherein determining the current condition that pertains to the mobile computing device includes identifying the rendered document from which the sequence of symbols was optically captured by the capture device of the mobile computing device, and wherein selecting the content item from among the two or more different content items includes selecting an identified content item that is associated in the data structure with the predetermined keyword or keyphrase and with the rendered document from which the sequence of symbols was optically captured by the capture device of the mobile computing device.

14. A computer-readable storage device storing software including instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, from a mobile computing device that includes a capture device and is associated with an output device, a sequence of symbols optically captured from a rendered document by the capture device;

determining, by one or more processors, that the sequence of symbols includes a predetermined keyword or keyphrase;

selecting, by one or more processors, a content item that has been associated with the predetermined keyword or keyphrase, the content item being an electronic document that includes one or more of text, graphics, or audio, wherein selecting the content item comprises;

accessing a data structure that includes associations between content items, predetermined keywords or keyphrases, and conditions, the conditions including potential locations of mobile computing devices at which sequences of symbols are optically captured from rendered documents, each association in the data structure associating a content item with a different unique combination of predetermined keyword or keyphrase, and condition, wherein the predetermined keyword or keyphrase is associated with two or more different content items, each association being subject to a different condition;

determining a current condition that pertains to a location of the mobile computing device at which the sequence of symbols was optically captured from the rendered document, the current condition being different from the predetermined keyword or keyphrase included in the sequence of symbols; and

selecting the content item from among the two or more different content items, including selecting an identified content item that is associated in the data structure with the predetermined keyword or keyphrase and is associated in the data structure with the current condition that pertains to the mobile computing device; and

transmitting, to the mobile computing device, the selected content item that has been associated with the predetermined keyword or keyphrase and has been associated with the current condition that pertains to the mobile computing device, for presentation by the output device associated with the mobile computing device.

15. The computer-readable storage device of claim 14, the operations further comprising receiving, from the mobile computing device, an email address of a user of the mobile computing device, wherein the content item is transmitted to the mobile computing device using the email address.

16. The computer-readable storage device of claim 14, wherein determining the current condition that pertains to the mobile computing device includes receiving, from the mobile computing device, location information that identifies a location of the mobile computing device at which the sequence of symbols was optically captured from the rendered document by the capture device, and wherein selecting the content item from among the two or more different content items includes selecting an identified content item that is associated in the data structure with the predetermined keyword or keyphrase and with the current location of the mobile computing device at which the sequence of symbols was optically captured from the rendered document by the capture device.

17. The computer-readable storage device of claim 14, wherein the conditions include potential times at which sequences of symbols are optically captured from rendered documents by capture devices of mobile computing devices, and wherein determining the current condition that pertains to the mobile computing device includes determining time information corresponding to a time at which the sequence of symbols was optically captured from the rendered document by the capture device of the mobile computing device, and wherein selecting the content item from among the two or more different content items includes selecting an identified content item that is associated in the data structure with the predetermined keyword or keyphrase and with the time at which the sequence of symbols was optically captured from the rendered document by the capture device of the mobile computing device.

18. The computer-readable storage device of claim 14, wherein the conditions include potential rendered document from which sequences of symbols are optically captured by capture devices of mobile computing devices, and wherein determining the current condition that pertains to the mobile computing device includes identifying the rendered document from which the sequence of symbols was optically captured by the capture device of the mobile computing device, and wherein selecting the content item from among the two or more different content items includes selecting an identified content item that is associated in the data structure with the predetermined keyword or keyphrase and with the rendered document from which the sequence of symbols was optically captured by the capture device of the mobile computing device.