Processing techniques for visual capture data from a rendered document

• US 20070011140A1
• Filed: 04/19/2005
• Published: 01/11/2007
• Est. Priority Date: 02/15/2004
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Exbiblio BV
• Inventors: King, Martin, Kushler, Clifford, Stafford-Fraser, James, Grover, Dale
• Granted Patent: US 7,702,624 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/316 Indexing structures
• CPC Class Codes: G06F 16/332 Query formulation
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 16/9554 by using bar codes
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06F 40/14 Tree-structured documents parsing G06F40/205; validation G06F40/226
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06K 17/0016 : Selecting or retrieving of images by means of their associated code-marks, e.g. coded microfilm or microfiche microfilm reading G03B21/11; computers therefor G06F16/00; teaching devices G09B5/02; accessing record carriers G11B27/002; static digital stores using optical elements G11C13/04; intermediate storage of facsimile picture signals H04N1/21
• CPC Class Codes: G06Q 10/00 : Administration; Management
• CPC Class Codes: G06Q 10/10 : Office automation; Time management
• CPC Class Codes: G06Q 20/102 : Bill distribution or payments
• CPC Class Codes: G06Q 20/12 : specially adapted for electronic shopping systems
• CPC Class Codes: G06Q 20/4012 : Verifying personal identification numbers [PIN]
• CPC Class Codes: G06Q 30/00 : Commerce
• CPC Class Codes: G06Q 30/02 : Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0207 : Discounts or incentives, e.g. coupons or rebates
• CPC Class Codes: G06Q 30/0241 : Advertisements
• CPC Class Codes: G06Q 30/0283 : Price estimation or determination
• CPC Class Codes: G06Q 30/04 : Billing or invoicing
• CPC Class Codes: G06Q 30/06 : Buying, selling or leasing transactions
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0603 : Catalogue ordering
• CPC Class Codes: G06Q 30/0623 : Item investigation
• CPC Class Codes: G06Q 30/0641 : Shopping interfaces
• CPC Class Codes: G06Q 30/0643 : Graphical representation of items or shoppers
• CPC Class Codes: G06Q 40/04 : Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06V 30/10 : Character recognition
• CPC Class Codes: G06V 30/142 : using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/1444 : Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields
• CPC Class Codes: G06V 30/1448 : based on markings or identifiers characterising the document or the area
• CPC Class Codes: G06V 30/248 : involving plural approaches, e.g. verification by template match; Resolving confusion among similar patterns, e.g. "O" versus "Q" G06V30/242 takes precedence
• CPC Class Codes: G06V 30/262 : using context analysis, e.g. lexical, syntactic or semantic context
• CPC Class Codes: G06V 30/40 : Document-oriented image-based pattern recognition
• CPC Class Codes: G07F 17/40 : for devices for accepting orders, advertisements, or the like
• CPC Class Codes: G07G 1/00 : Cash registers alarm indicators G07G3/00
• CPC Class Codes: H04N 1/00204 : with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00244 : with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 1/00326 : with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus arrangements for the associated working of recording or reproducing apparatus with related apparatus G11B31/00
• CPC Class Codes: H04N 1/0036 : Alphanumeric symbols
• CPC Class Codes: H04N 1/00363 : Bar codes or the like
• CPC Class Codes: H04N 1/00374 : on the same page as at least a part of the image
• CPC Class Codes: H04N 1/00381 : Input by recognition or interpretation of visible user gestures manual input means, e.g. digitisers, writing tablets H04N1/00392; gesture input to computers G06F3/017; recognition algorithms G06V40/20
• CPC Class Codes: H04N 1/107 : with manual scanning
• CPC Class Codes: H04N 1/2166 : for mass storage, e.g. in document filing systems information retrieval G06F16/00
• CPC Class Codes: H04N 1/2187 : with image input from a plurality of different locations or from a non-central location, e.g. from one or more users
• CPC Class Codes: H04N 1/32128 : attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 1/34 : for coin-freed systems ; Pay systems
• CPC Class Codes: H04N 2201/0081 : Image reader H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/3215 : of a time or duration
• CPC Class Codes: H04N 2201/3249 : data relating to a linked page or object, e.g. hyperlink
• CPC Class Codes: H04N 2201/3253 : Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 2201/3263 : of a graphical motif or symbol, e.g. Christmas symbol, logo H04N2201/3211 takes precedence
• CPC Class Codes: H04N 2201/3266 : of text or character information, e.g. text accompanying an image H04N2201/3226 takes precedence
• CPC Class Codes: H04N 2201/3277 : The additional information being stored in the same storage device as the image data
• CPC Class Codes: H04N 2201/3278 : Transmission
• CPC Class Codes: Y02A 90/10 : Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
• CPC Class Codes: Y10S 707/99931 : Database or file accessing
• CPC Class Codes: Y10S 707/99934 : Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99936 : Pattern matching access
• CPC Class Codes: Y10S 707/99943 : Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99945 : Object-oriented database structure processing

Abstract

A facility for navigating an electronic document is described. The facility receives user input selecting a portion of the content of a rendered document that constitutes a sentence fragment. In response to receiving the user input, the facility identifies an electronic document contained in a corpus of electronic documents, the identified electronic document containing the selected document portion. In response to receiving the user input, the facility further identifies a position within the identified electronic document at which the selected document portion occurs.

Claims

1. A method in a computing system for navigating an electronic document, comprising:

receiving user input selecting a portion of the content of a rendered document, the portion constituting a sentence fragment; and

in response to receiving the user input, identifying (1) an electronic document contained in a corpus of electronic documents, the identified electronic document containing the selected document portion, and (2) a position within the identified electronic document at which the selected document portion occurs.

2. The method of claim 1, further comprising:

accessing a markup layer that relates to the identified electronic document;

identifying actions defined by the markup layer for the identified position in the identified electronic document; and

making the identified actions available to a user who provided the received user input.

3. The method of claim 1 wherein the identified electronic document and the identified position are identified without determining a standard-character-set representation of the selected document portion.

4. The method of claim 3 wherein the identification is performed using an image of the selected document portion.

5. The method of claim 3 wherein the identification is performed using a convolution-encoded representation of text contained by the selected document portion.

6. The method of claim 5, further comprising:

encoding selected document portions each into a convolution-encoded representation, the two selected document portions containing text of two different natural languages.

7. The method of claim 5, further comprising;

encoding data that is independent of textual content of the selected document portion in the convolution-encoded representation, wherein encoding data comprises including an invalid value in the convolution-encoded representation.

8. The method of claim 3 wherein the selected document portion is unique among the corpus of electronic documents, and a version of the selected document portion from which a single word is deleted is not unique among the corpus of electronic documents.

9. The method of claim 3 wherein the selected document portion is contained by a plurality of electronic documents of the corpus, the method further comprising:

identifying a user who provided the received user input;

retrieving context information for the identified user; and

identifying a document among the plurality of electronic documents most likely to correspond to the rendered document based upon contents of the retrieved context information.

10. The method of claim 1, further comprising:

receiving user input specifying an annotation relative to the identified position in the identified electronic document;

storing the specified annotation together with an indication of the identified electronic document and identified position in a location outside the identified electronic document;

in response to a user request for the specified annotation;

retrieving at least a portion of the identified electronic document containing the identified position;

displaying a region of the identified electronic document containing the identified position;

retrieving the specified annotation; and

displaying the specified annotation in connection with the displayed document region at a position proximate to the identified position.

11. (canceled)

12. A method in a portable optical text capture device for capturing text from a rendered document, comprising:

capturing from the rendered document a first image of text in the rendered document;

uploading the captured first image of text to a computer system separate from the portable optical text capture device;

receiving from the computer system a mapping of character images in a typeface used in the rendered document to character identities;

capturing from the rendered document a second image of text in the rendered document; and

using the received mapping to determine the identity of characters whose images occur in the second image of text.

13. (canceled)

14. A method in a portable optical text capture device for capturing text from a rendered document, comprising:

capturing from the rendered document an image of text in the rendered document;

identifying a distinguished character contained in the captured image whose appearance is not directly associated with its identity in the portable optical text capture device by determining that the distinguished character can be formed by combining a plurality of characters other than the distinguished character whose appearances are directly associated with its identity in the portable optical text capture device in accordance with a character relationship rule stored in the portable optical text capture device.

15. (canceled)

16. A method in a computing system for accessing an electronic copy of a rendered document on behalf of a user, comprising:

receiving visual content captured by the user from the rendered document;

using the captured content to identify an electronic document corresponding to the rendered document;

retrieving a copy of the identified electronic document; and

granting the user access to the retrieved copy of the identified electronic document based upon the captured content.

17. The method of claim 16 wherein the captured content is text from the body of the rendered document.

18. The method of claim 16 wherein the captured content is a non-textual symbol contained in the rendered document.

19. A computer-readable medium whose contents cause a computing system to perform a method for navigating an electronic document, the method comprising:

receiving user input selecting a portion of the content of a rendered document, the portion constituting a sentence fragment; and

in response to receiving the user input, identifying (1) an electronic document contained in a corpus of electronic documents, the identified electronic document containing the selected document portion, and (2) a position within the identified electronic document at which the selected document portion occurs.

20. The computer-readable medium of claim 19, further comprising:

receiving user input specifying an annotation relative to the identified position in the identified electronic document;

storing the specified annotation together with an indication of the identified electronic document and identified position in a location outside the identified electronic document;

in response to a user request for the specified annotation;

retrieving at least a portion of the identified electronic document containing the identified position;

displaying a region of the identified electronic document containing the identified position;

retrieving the specified annotation; and

displaying the specified annotation in connection with the displayed document region at a position proximate to the identified position.

21. A portable optical text capture device for capturing text from a rendered document, comprising:

a capture component that captures from the rendered document a first image of text in the rendered document and a second image of text in the rendered document;

an upload component that uploads the captured first image of text to a computer system separate from the portable optical text capture device;

a reception component that receives from the computer system a mapping of character images in a typeface used in the rendered document to character identities; and

a determination component that uses the received mapping to determine the identity of characters whose images occur in the second image of text.

22. One or more computer memories collectively containing a data structure, the data structure capable of performing a method of capturing text from a rendered document, the method comprising:

receiving an image of text from a portable optical text capture device, the image of text captured from the rendered document; and

identifying a distinguished character contained in the received image whose appearance is not directly associated with its identity in the portable optical text capture device by determining that the distinguished character can be formed by combining a plurality of characters other than the distinguished character whose appearances are directly associated with its identity in the portable optical text capture device in accordance with a character relationship rule stored in the portable optical text capture device.