Capturing text from rendered documents using supplemental information

• US 10,423,862 B2
• Filed: 09/13/2018
• Issued: 09/24/2019
• Est. Priority Date: 04/01/2004
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: King, Martin T., Grover, Dale L., Kushler, Clifford A., Stafford-Fraser, James Q.
• Primary Examiner(s): Mariam, Daniel G
• Application Number:
  US16/130,142
• Publication Number:
  US 20190156162A1
• Time in Patent Office: 376 Days
• Field of Search: 382181-182, 382209, 382218, 382229, 382305, 382317, 382321
• CPC Class Codes: G06F 40/00 Handling natural language data speech analysis or synthesis, speech recognition G10L
• CPC Class Codes: G06F 40/109 Font handling; Temporal or kinetic typography
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/418 Document matching, e.g. of document images

Abstract

A system for processing a text capture operation is described. The system receives text captured from a rendered document in the text capture operation. The system also receives supplemental information distinct from the captured text. The system determines an action to perform in response to the text capture operation based upon both the captured text and the supplemental information.

Claims

1. A method performed by a data processing apparatus, the method comprising:

receiving, at a computing device, data representing a symbol captured from a rendered document during a symbol capture operation;

receiving, at the computing device, supplemental information including information relating to circumstances under which the symbol capture operation was performed, the information relating to circumstances under which the symbol capture operation was performed comprising at least one of (i) a geographic location at which the symbol capture operation was performed to capture the symbol from the rendered document, (ii) actions performed by a user that initiated the symbol capture operation within a threshold amount of time of a time at which the symbol capture operation was performed or (iii) information describing an environment in which the symbol capture operation was performed;

determining, by the computing device and based on the supplemental information, an action to be performed; and

causing the action to be performed.

2. The method of claim 1, wherein the symbol comprises a bar code.

3. The method of claim 1, wherein the information describing the environment comprises at least one of information describing lighting, temperature, or sound at a location in which the symbol capture operation was performed.

4. The method of claim 3, wherein determining, by the computing device and based on the supplemental information, an action to be performed comprises:

determining whether to prevent use of a scanner that captured the symbol based on the information describing the environment.

5. The method of claim 1, wherein determining, by the computing device and based on the supplemental information, an action to be performed comprises:

identifying the document from which the symbol was captured; and

determining to provide content related to the document.

6. The method of claim 1, wherein determining, by the computing device and based on the supplemental information, an action to be performed comprises:

determining, based on the symbol being captured, that the user that initiated the symbol capture operation is in possession of a printed version of the document; and

determining, based on the user being in possession of the printed version of the document, to provide, to the user, access to electronic content related to the document.

7. The method of claim 1, wherein determining the action to be performed comprises determining the action based on actions being performed by users in the geographic location at which the symbol was captured from the rendered document.

8. A system, comprising:

a data processing apparatus; and

a memory apparatus in data communication with the data processing apparatus and storing instructions executable by the data processing apparatus and that upon such execution cause the data processing apparatus to perform operations comprising;

receiving, at a computing device, data representing a symbol captured from a rendered document during a symbol capture operation;

receiving, at the computing device, supplemental information including information relating to circumstances under which the symbol capture operation was performed, the information relating to circumstances under which the symbol capture operation was performed comprising at least one of (i) a geographic location at which the symbol capture operation was performed to capture the symbol from the rendered document, (ii) actions performed by a user that initiated the symbol capture operation within a threshold amount of time of a time at which the symbol capture operation was performed or (iii) information describing an environment in which the symbol capture operation was performed;

determining, by the computing device and based on the supplemental information, an action to be performed; and

causing the action to be performed.

9. The system of claim 8, wherein the symbol comprises a bar code.

10. The system of claim 8, wherein the information describing the environment comprises at least one of information describing lighting, temperature, or sound at a location in which the symbol capture operation was performed.

11. The system of claim 10, wherein determining, by the computing device and based on the supplemental information, an action to be performed comprises:

determining whether to prevent use of a scanner that captured the symbol based on the information describing the environment.

12. The system of claim 8, wherein determining, by the computing device and based on the supplemental information, an action to be performed comprises:

identifying the document from which the symbol was captured; and

determining to provide content related to the document.

13. The system of claim 8, wherein determining, by the computing device and based on the supplemental information, an action to be performed comprises:

determining, based on the symbol being captured, that the user that initiated the symbol capture operation is in possession of a printed version of the document; and

determining, based on the user being in possession of the printed version of the document, to provide, to the user, access to electronic content related to the document.

14. The system of claim 8, wherein determining the action to be performed comprises determining the action based on actions being performed by users in the geographic location at which the symbol was captured from the rendered document.

15. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:

receiving, at a computing device, data representing a symbol captured from a rendered document during a symbol capture operation;

receiving, at the computing device, supplemental information including information relating to circumstances under which the symbol capture operation was performed, the information relating to circumstances under which the symbol capture operation was performed comprising at least one of (i) a geographic location at which the symbol capture operation was performed to capture the symbol from the rendered document, (ii) actions performed by a user that initiated the symbol capture operation within a threshold amount of time of a time at which the symbol capture operation was performed or (iii) information describing an environment in which the symbol capture operation was performed;

determining, by the computing device and based on the supplemental information, an action to be performed; and

causing the action to be performed.

16. The non-transitory computer storage medium of claim 15, wherein the symbol comprises a bar code.

17. The non-transitory computer storage medium of claim 15, wherein the information describing the environment comprises at least one of information describing lighting, temperature, or sound at a location in which the symbol capture operation was performed.

18. The non-transitory computer storage medium of claim 17, wherein determining, by the computing device and based on the supplemental information, an action to be performed comprises:

determining whether to prevent use of a scanner that captured the symbol based on the information describing the environment.

19. The non-transitory computer storage medium of claim 15, wherein determining, by the computing device and based on the supplemental information, an action to be performed comprises:

identifying the document from which the symbol was captured; and

determining to provide content related to the document.

20. The non-transitory computer storage medium of claim 15, wherein determining, by the computing device and based on the supplemental information, an action to be performed comprises:

determining, based on the symbol being captured, that the user that initiated the symbol capture operation is in possession of a printed version of the document; and

determining, based on the user being in possession of the printed version of the document, to provide, to the user, access to electronic content related to the document.