Capturing text from rendered documents using supplemental information

• US 10,115,041 B2
• Filed: 09/05/2014
• Issued: 10/30/2018
• Est. Priority Date: 01/26/2004
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: King, Martin T., Grover, Dale L., Kushler, Clifford A., Stafford-Fraser, James Q.
• Primary Examiner(s): Mariam, Daniel
• Application Number:
  US14/479,145
• Publication Number:
  US 20150063714A1
• Time in Patent Office: 1,516 Days
• Field of Search: 382181-182, 382209, 382218, 382229, 382305, 382317, 382321
• CPC Class Codes: G06F 40/00 Handling natural language data speech analysis or synthesis, speech recognition G10L
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06V 10/768 using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition

Abstract

A system for processing a text capture operation is described. The system receives text captured from a rendered document in the text capture operation. The system also receives supplemental information distinct from the captured text. The system determines an action to perform in response to the text capture operation based upon both the captured text and the supplemental information.

Claims

1. A method performed by a data processing apparatus, the method comprising:

receiving, at a computing device, text captured from a rendered document during a text capture operation;

receiving, at the computing device, supplemental information including information relating to circumstances under which the text capture operation was performed, the information relating to circumstances under which the text capture operation was performed comprising a geographic location at which the text capture operation was performed to capture the text from the rendered document;

determining, by the computing device and based on the supplemental information including the geographic location at which the text capture operation was performed to capture the text from the rendered document, an action to be performed on the captured text; and

causing the action to be performed on the captured text.

2. The method of claim 1, wherein determining the action to be performed comprises determining the action based on actions being performed by users in the geographic location at which the text was captured from the rendered document.

3. The method of claim 1, wherein:

the text was captured by a user using a text capture device; and

the supplemental information comprises previous text captured by the user during one or more previous text capture operations.

4. The method of claim 3, wherein determining the action to be performed comprises determining the action based on additional supplemental information relating to circumstances under which the previous one or more text capture operations were performed.

5. The method of claim 4, wherein the additional supplemental information comprises information describing actions performed by the user within a threshold amount of time of a time at which the text capture operation was performed.

6. The method of claim 1, wherein the information relating to the circumstances under which the text capture operation was performed comprises information describing an environment in which the text capture operation was performed.

7. The method of claim 6, wherein the information describing the environment comprises at least one of information describing lighting, temperature, or sound at a location in which the text capture operation was performed.

8. The method of claim 1, wherein the supplemental information comprises a prediction of a user'"'"'s future context.

9. The method of claim 1, wherein determining the action to be performed comprises customizing presentation of content based on the supplemental information.

10. The method of claim 1, further comprising identifying a plurality of electronic documents that include the captured text, wherein the action comprises identifying a particular electronic document from the plurality of electronic documents based on the supplemental information.

11. The method of claim 1, wherein the supplemental information includes information regarding a user that caused the text capture operation to be performed.

12. The method of claim 1, wherein the information relating to the circumstances under which the text capture operation was performed comprises information describing circumstances at a time at which the text capture operation was performed.

13. The method of claim 1, wherein the supplemental information is independent of content of the rendered document.

14. A system, comprising:

a data processing apparatus; and

a memory apparatus in data communication with the data processing apparatus and storing instructions executable by the data processing apparatus and that upon such execution cause the data processing apparatus to perform operations comprising;

receiving text captured from a rendered document during a text capture operation;

receiving supplemental information including information relating to circumstances under which the text capture operation was performed, the information relating to circumstances under which the text capture operation was performed comprising a geographic location at which the text capture operation was performed to capture the text from the rendered document;

determining, based on the supplemental information including the geographic location at which the text capture operation was performed to capture the text from the rendered document, an action to be performed on the captured text; and

causing the action to be performed on the captured text.

15. The system of claim 14, wherein determining the action to be performed comprises determining the action based on actions being performed by users in the geographic location at which the text was captured from the rendered document.

16. The system of claim 14, wherein:

the text was captured by a user using a text capture device; and

the supplemental information comprises previous text captured by the user during one or more previous text capture operations.

17. The system of claim 16, wherein determining the action to be performed comprises determining the action based on additional supplemental information relating to circumstances under which the previous one or more text capture operations were performed.

18. The system of claim 17, wherein the additional supplemental information comprises information describing actions performed by the user within a threshold amount of time of a time at which the text capture operation was performed.

19. The system of claim 14, wherein the information relating to the circumstances under which the text capture operation was performed comprises information describing an environment in which the text capture operation was performed.

20. The system of claim 19, wherein the information describing the environment comprises at least one of information describing lighting, temperature, or sound at a location in which the text capture operation was performed.

21. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:

receiving text captured from a rendered document during a text capture operation;

receiving supplemental information including information relating to circumstances under which the text capture operation was performed, the information relating to circumstances under which the text capture operation was performed comprising a geographic location at which the text capture operation was performed to capture the text from the rendered document;

determining, based on the supplemental information including the geographic location at which the text capture operation was performed to capture the text from the rendered document, an action to be performed on the captured text; and

causing the action to be performed on the captured text.