Processing techniques for visual capture data from a rendered document

• US 8,874,504 B2
• Filed: 03/22/2010
• Issued: 10/28/2014
• Est. Priority Date: 12/03/2004
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: King, Martin T., Kushler, Clifford A., Stafford-Fraser, James Q., Grover, Dale L.
• Primary Examiner(s): BROWN, SHEREE N
• Application Number:
  US12/729,045
• Publication Number:
  US 20110022940A1
• Time in Patent Office: 1,681 Days
• Field of Search: 707/713, 707/608, 715/229, 715/273
• US Class Current: 707/608
• CPC Class Codes: G06F 16/382 using citations hypermedia G06F16/94
• CPC Class Codes: G06F 16/9032 Query formulation
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/9554 by using bar codes
• CPC Class Codes: G06F 40/12 Use of codes for handling textual entities
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06V 30/268 Lexical context

Abstract

A facility for navigating an electronic document is described. The facility receives user input selecting a portion of the content of a rendered document that constitutes a sentence fragment. In response to receiving the user input, the facility identifies an electronic document contained in a corpus of electronic documents, the identified electronic document containing the selected document portion. In response to receiving the user input, the facility further identifies a position within the identified electronic document at which the selected document portion occurs.

Claims

1. A method for processing data captured from a rendered document, the method comprising:

receiving data specifying a user input selecting a portion of content of a rendered document, the selected document portion containing text;

determining an encoded representation of text contained by the selected document portion, wherein determining the encoded representation of text contained by the selected document portion comprises determining a sequence of offsets that represent the text, and wherein each offset represents a number of character positions separating recurring instance of each letter within the text of the selected document portion;

searching for the encoded representation of text within a corpus of electronic documents to identify (1) an electronic document contained in the corpus of electronic documents, the identified electronic document containing the selected document portion, and (2) a position within the identified electronic document at which the selected document portion occurs; and

providing, for presentation on a display device, a menu of selectable user actions associated with the identified position in the identified electronic document, wherein at least one of the selectable user actions was associated with the identified position before the receipt of the data specifying the user input.

2. The method of claim 1, further comprising:

accessing a markup layer that relates to the identified electronic document; and

identifying actions defined by the markup layer for the identified position in the identified electronic document, wherein the identified actions are presented within the menu of selectable user inputs.

3. The method of claim 1, wherein the identified electronic document and the identified position are identified without determining a standard-character-set representation of the selected document portion.

4. The method of claim 3, wherein an image of the selected document portion is used to determine the encoded representation of text.

5. The method of claim 3, wherein the selected document portion is unique among the corpus of electronic documents, and a version of the selected document portion from which a single word is deleted is not unique among the corpus of electronic documents.

6. The method of claim 3, wherein the selected document portion is contained by a plurality of electronic documents of the corpus, the method further comprising:

identifying a user who provided the user input;

retrieving context information for the identified user; and

identifying a document among the plurality of electronic documents most likely to correspond to the rendered document based upon contents of the retrieved context information.

7. The method of claim 1, further comprising:

encoding both of two selected document portions into the encoded representation, the two selected document portions containing text of two different natural languages.

8. The method of claim 1, further comprising:

encoding data, independent of the text contained in the selected document portion, into the encoded representation by including an invalid value in the encoded representation.

9. The method of claim 1, further comprising:

receiving data specifying a user input that specifies an annotation relative to the identified position in the identified electronic document;

storing the specified annotation together with an indication of the identified electronic document and position in a location outside the identified electronic document;

in response to a user request for the specified annotation;

retrieving at least a portion of the identified electronic document containing the identified position;

providing, for display on the display device, a region of the identified electronic document containing the identified position;

retrieving the specified annotation; and

providing, for display on the display device, the specified annotation in connection with the displayed document region at a position proximate to the identified position.

10. The method of claim 1, wherein determining the sequence of offsets that represents the text comprises determining a first sequence of offsets and a second sequence of offsets that is a reduced version of the first sequence of offsets.

11. The method of claim 1, wherein, if a given letter occurs only once in the selected document portion of text, an offset for the given letter is zero (0).

12. The method of claim 1,

wherein the text contained by the selected document portion comprises one or more upper case letters, and

wherein determining the encoded representation of the text contained by the selected document portion comprises dropping the upper case letters so as to not include within the sequence of offsets any offset for the dropped upper case letters.

13. The method of claim 1, further comprising:

determining that the encoded representation of text contained by the selected document portion comprises two or more offsets that repeat,

wherein determining the sequence of offsets comprises adding symbols into the sequence of offsets that identify the repeating offsets so as to simplify the sequence of offsets.

26. The method of claim 1,

wherein receiving the data specifying the user input selecting the portion of the content of the rendered document comprises receiving scanned text resulting from scanning the text contained by the selected document portion with a scanner device, and

wherein determining the encoded representation of text contained by the selected document portion comprises generating a representation of the scanned text, and sliding a copy of the representation of the scanned text horizontally over the representation of the scanned text.

14. A non-transitory storage device, comprising:

program instructions readable by a logic device, wherein the program instructions are executable by the logic device to cause a computing system to perform operations comprising;

receiving data specifying a user input to select a portion of content of a rendered document, the selected document portion containing text;

determining an encoded representation of text contained by the selected document portion, wherein determining the encoded representation of text contained by the selected document portion comprises determining a sequence of offsets that represent the text, and wherein each offset represents a number of character positions separating recurring instance of each letter within the text of the selected document portion;

searching for the encoded representation of text within a corpus of electronic documents to identify (1) an electronic document contained in a corpus of electronic documents, the identified electronic document containing the selected document portion, and (2) a position within the identified electronic document at which the selected document portion occurs; and

providing, for presentation on a display device, a menu of selectable user actions associated with the identified position in the identified electronic document, wherein at least one of the selectable user actions was associated with the identified position before the receipt of the data specifying the user input.

15. The non-transitory storage device of claim 14, wherein the program instructions are further executable by the logic device to perform steps comprising:

accessing a markup layer that relates to the identified electronic document; and

identifying actions defined by the markup layer for the identified position in the identified electronic document, wherein the identified actions are presented within the menu of selectable user inputs.

16. The non-transitory storage device of claim 14, wherein the identified electronic document and the identified position are identified without determining a standard-character-set representation of the selected document portion.

17. The non-transitory storage device of claim 16, wherein an image of the selected document is used to determine the encoded representation of text.

18. The non-transitory storage device of claim 16, wherein the selected document portion is unique among the corpus of electronic documents, and a version of the selected document portion from which a single word is deleted is not unique among the corpus of electronic documents.

19. The non-transitory storage device of claim 16, wherein the selected document portion is contained by a plurality of electronic documents of the corpus, and

wherein the program instructions are further executable by the logic device to perform steps comprising:

identifying a user who provided the received user input;

retrieving context information for the identified user; and

identifying a document among the plurality of electronic documents most likely to correspond to the rendered document based upon contents of the retrieved context information.

20. The non-transitory storage device of claim 14, wherein the program instructions are further executable by the logic device to encode both of two selected document portions into the encoded representation, the two selected document portions containing text of two different natural languages.

21. The non-transitory storage device of claim 14, wherein the program instructions are further executable by the logic device to encode data, independent of the text contained in the selected document portion, into the encoded representation by including an invalid value in the encoded representation.

22. The non-transitory storage device of claim 14, wherein the program instructions are further executable by the logic device to perform steps comprising:

receiving data specifying a user input that specifies an annotation relative to the identified position in the identified electronic document;

storing the specified annotation together with an indication of the identified electronic document and position in a location outside the identified electronic document;

in response to a user request for the specified annotation;

retrieving at least a portion of the identified electronic document containing the identified position;

providing, for display on the display device, a region of the identified electronic document containing the identified position;

retrieving the specified annotation; and

providing, for display on the display device, the specified annotation in connection with the displayed document region at a position proximate to the identified position.

23. A computing system for processing data captured from a rendered document, the computing system comprising:

a receiver that receives data specifying a user input selecting a portion of content of a rendered document, the selected document portion containing text;

a processor that, in response to receiving the data specifying the user input at the receiver;

determines an encoded representation of text contained by the selected document portion, wherein determining the encoded representation of text contained by the selected document portion comprises determining a sequence of offsets that represent the text, and wherein each offset represents a number of character positions separating recurring instance of each letter within the text of the selected document portion,

searches for the encoded representation of text within a corpus of electronic documents to identify (1) an electronic document contained in a corpus of electronic documents, the identified electronic document containing the selected document portion, and (2) a position within the identified electronic document at which the selected document portion occurs, and

provides, for presentation by a display device, a menu of selectable user actions associated with the identified position in the identified electronic document, wherein at least one of the selectable user actions was associated with the identified position in the identified electronic document before the receiver received the user input.

24. The computing system of claim 23, wherein the rendered document is a printed document.

25. The computing system of claim 23, wherein the rendered document is a displayed document.