Establishing an interactive environment for rendered documents

• US 8,799,303 B2
• Filed: 10/13/2010
• Issued: 08/05/2014
• Est. Priority Date: 02/15/2004
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: King, Martin T., Grover, Dale L., Kushler, Clifford A., Stafford-Fraser, James Q.
• Primary Examiner(s): Xue, Belinda
• Application Number:
  US12/904,064
• Publication Number:
  US 20110154507A1
• Time in Patent Office: 1,392 Days
• Field of Search: None
• US Class Current: 707/758
• CPC Class Codes: G06F 16/316 Indexing structures
• CPC Class Codes: G06F 16/332 Query formulation
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/483 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 16/9554 by using bar codes
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 21/1083 Partial license transfers
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06K 17/0016 Selecting or retrieving of images by means of their associated code-marks, e.g. coded microfilm or microfiche microfilm reading G03B21/11; computers therefor G06F16/00; teaching devices G09B5/02; accessing record carriers G11B27/002; static digital stores using optical elements G11C13/04; intermediate storage of facsimile picture signals H04N1/21
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 20/102 : Bill distribution or payments
• CPC Class Codes: G06Q 20/12 : specially adapted for electronic shopping systems
• CPC Class Codes: G06Q 20/4012 : Verifying personal identification numbers [PIN]
• CPC Class Codes: G06Q 30/00 : Commerce
• CPC Class Codes: G06Q 30/02 : Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0207 : Discounts or incentives, e.g. coupons or rebates
• CPC Class Codes: G06Q 30/0241 : Advertisements
• CPC Class Codes: G06Q 30/0283 : Price estimation or determination
• CPC Class Codes: G06Q 30/04 : Billing or invoicing
• CPC Class Codes: G06Q 30/06 : Buying, selling or leasing transactions
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0603 : Catalogue ordering
• CPC Class Codes: G06Q 30/0623 : Item investigation
• CPC Class Codes: G06Q 30/0641 : Shopping interfaces
• CPC Class Codes: G06Q 30/0643 : Graphical representation of items or shoppers
• CPC Class Codes: G06Q 40/04 : Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06V 30/10 : Character recognition
• CPC Class Codes: G06V 30/142 : using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/1444 : Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields
• CPC Class Codes: G06V 30/1448 : based on markings or identifiers characterising the document or the area
• CPC Class Codes: G06V 30/248 : involving plural approaches, e.g. verification by template match; Resolving confusion among similar patterns, e.g. "O" versus "Q" G06V30/242 takes precedence
• CPC Class Codes: G06V 30/262 : using context analysis, e.g. lexical, syntactic or semantic context
• CPC Class Codes: G06V 30/40 : Document-oriented image-based pattern recognition
• CPC Class Codes: G07F 17/40 : for devices for accepting orders, advertisements, or the like
• CPC Class Codes: H04N 1/00204 : with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00244 : with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 1/00326 : with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus arrangements for the associated working of recording or reproducing apparatus with related apparatus G11B31/00
• CPC Class Codes: H04N 1/0036 : Alphanumeric symbols
• CPC Class Codes: H04N 1/00363 : Bar codes or the like
• CPC Class Codes: H04N 1/00374 : on the same page as at least a part of the image
• CPC Class Codes: H04N 1/00381 : Input by recognition or interpretation of visible user gestures manual input means, e.g. digitisers, writing tablets H04N1/00392; gesture input to computers G06F3/017; recognition algorithms G06V40/20
• CPC Class Codes: H04N 1/107 : with manual scanning
• CPC Class Codes: H04N 1/2166 : for mass storage, e.g. in document filing systems information retrieval G06F16/00
• CPC Class Codes: H04N 1/2187 : with image input from a plurality of different locations or from a non-central location, e.g. from one or more users
• CPC Class Codes: H04N 1/32128 : attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 1/34 : for coin-freed systems ; Pay systems
• CPC Class Codes: H04N 2201/0081 : Image reader H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/3215 : of a time or duration
• CPC Class Codes: H04N 2201/3249 : data relating to a linked page or object, e.g. hyperlink
• CPC Class Codes: H04N 2201/3253 : Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 2201/3263 : of a graphical motif or symbol, e.g. Christmas symbol, logo H04N2201/3211 takes precedence
• CPC Class Codes: H04N 2201/3266 : of text or character information, e.g. text accompanying an image H04N2201/3226 takes precedence
• CPC Class Codes: H04N 2201/3277 : The additional information being stored in the same storage device as the image data
• CPC Class Codes: H04N 2201/3278 : Transmission
• CPC Class Codes: Y10S 707/99931 : Database or file accessing
• CPC Class Codes: Y10S 707/99934 : Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99936 : Pattern matching access
• CPC Class Codes: Y10S 707/99943 : Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99945 : Object-oriented database structure processing

Abstract

A system for identifying an electronic counterpart for a rendered document is described. The system receives an indication of a text capture operation performed from a rendered document. The indication identifies a text sequence captured as part of the text capture operation, the identified text sequence comprising fewer than nine words. In response to receiving the indication, the system uniquely identifies an electronic document from which the rendered document was rendered.

Claims

1. A method in a computing system for regulating access to supplemental electronic materials associated with a distinguished rendered document, the computing system comprising a memory and a plurality of processing components, the method comprising:

receiving into the memory a human-readable text sequence captured as part of a text capture operation performed by a distinguished user on a print version of the distinguished rendered document using a scanner, wherein the text capture operation captures the human-readable text sequence and the human-readable text sequence includes text that is perceptible to a human on the print version of the distinguished rendered document;

processing data from the text capture operation to recognize the text;

determining, from the recognized text, a text string corresponding to the captured human-readable text sequence;

identifying the distinguished rendered document using the text string;

identifying supplemental electronic materials associated with the distinguished rendered document, the supplemental electronic materials being materials presentable using an audio or visual medium;

determining that a request to access the supplemental electronic materials associated with the distinguished rendered document is implicit in receiving the human-readable text sequence;

retrieving a set of access rules for the supplemental electronic materials associated with the distinguished rendered document; and

resolving the request by enabling access to the supplemental electronic materials when the request satisfies the access rules or denying access to the supplemental electronic materials when the request does not satisfy the access rules, wherein resolving the request by enabling access to the supplemental electronic materials when the request satisfies the access rules includes permitting access to the supplemental electronic materials when the received human-readable text sequence includes an indication that the human-readable text sequence was captured from the print version of the distinguished rendered document.

2. The method of claim 1, wherein resolving the request by enabling access to the supplemental electronic materials when the request satisfies the access rules includes permitting full access to the supplemental electronic materials in response to a determination that the distinguished user is in possession of an authorized copy of the distinguished rendered document.

3. The method of claim 1, wherein resolving the request by enabling access to the supplemental electronic materials when the request satisfies the access rules includes permitting access to the supplemental electronic materials when the received human-readable text sequence includes an indication that the human-readable text sequence was captured from the distinguished rendered document by a subscriber of the distinguished rendered document.

4. The method of claim 1, wherein resolving the request by enabling access to the supplemental electronic materials when the request satisfies the access rules includes permitting access to the supplemental electronic materials when the received human-readable text sequence includes an indication from printed characteristics of the human-readable text sequence that the human-readable text sequence was captured from an authorized version of the distinguished rendered document.

5. A system on a mobile device for providing access to supplemental electronic materials associated with a document, the system comprising:

a text capture component, wherein the text capture component includes a scanner programmed to capture a human-readable text sequence of a print version of a document, wherein the human-readable text sequence includes text that is perceptible to a human on the print version of the document;

a processing component, wherein the processing component is programmed to process data from the text capture component to recognize the text;

an identification component, wherein the identification component is programmed to determine, from the recognized text, a text string corresponding to the captured human-readable text sequence and identify the document using the text string;

a content access component, wherein the content access component is programmed to;

identify supplemental electronic materials associated with the document, the supplemental electronic materials being materials presentable using an audio or visual medium;

determine that a request to access the supplemental electronic materials associated with the document is implicit in receiving the human-readable text sequence;

receive a set of access rules for the supplemental electronic materials associated with the document; and

resolve the request by enabling access to the supplemental electronic materials when the request satisfies the access rules or denying access to the supplemental electronic materials when the request does not satisfy the access rules, wherein resolving the request by enabling access to the supplemental electronic materials when the request satisfies the access rules includes permitting access to the supplemental electronic materials when the received human-readable text sequence includes an indication that the human-readable text sequence was captured from the print version of the document; and

a content presentation component, wherein the content presentation component is programmed to present the supplemental electronic materials associated with the document to a user associated with the mobile device.

6. The system of claim 5, wherein the content access component is programmed to enable access to additional supplemental electronic materials associated with the document upon determining that a user of the mobile device is a subscriber of the document.

7. The system of claim 5, wherein the content presentation component is programmed to present the supplemental electronic materials associated with the document to the user via a display or speaker of the mobile device.

8. The system of claim 5, wherein the content presentation component is programmed to present the supplemental electronic materials associated with the document to the user via a computing system in communication with the mobile device.

9. A non-transitory computer-readable storage medium whose contents, when executed by a mobile device, cause the mobile device to perform a method of enabling access to supplemental electronic materials associated with a document, the method comprising:

capturing a human-readable text sequence of a print version of the document using an imaging component of the mobile device, wherein the human-readable text sequence includes text that is perceptible by a human on the print version of the document;

processing data from the imaging component to recognize the text;

identifying, within the human-readable text sequence of the print version of the document, a text fragment of the print version of the document;

identifying the document based on the text fragment;

identifying supplemental electronic materials associated with the document, the supplemental electronic materials being materials presentable using an audio or visual medium;

determining that a request to access the supplemental electronic materials associated with the document is implicit in receiving the human-readable text sequence;

retrieving a set of access rules for the supplemental electronic materials associated with the document;

resolving the request by enabling access to the supplemental electronic materials when the request satisfies the access rules or denying access to the supplemental electronic materials when the request does not satisfy the access rules, wherein resolving the request by enabling access to the supplemental electronic materials when the request satisfies the access rules includes permitting access to the supplemental electronic materials when the human-readable text sequence includes an indication that the human-readable text sequence was captured from the print version of the document.

10. The non-transitory computer-readable storage medium of claim 9, wherein resolving the request includes applying the access rules associated with the supplemental electronic materials to the received request.

11. The non-transitory computer-readable storage medium of claim 9, wherein resolving the request includes determining that the user captured the human-readable text sequence of the print version of the document.

12. The non-transitory computer-readable storage medium of claim 9, wherein resolving the request includes determining that the user is a subscriber of the document.

13. The non-transitory computer-readable storage medium of claim 9, wherein resolving the request includes determining that the user has purchased access rights to the supplemental electronic materials.