Processing techniques for text capture from a rendered document

• US 20060294094A1
• Filed: 05/17/2005
• Published: 12/28/2006
• Est. Priority Date: 02/15/2004
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kushler, Clifford A., Stafford-Fraser, James Q., King, Martin T., Grover, Dale L.
• Granted Patent: US 7,818,215 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/316 Indexing structures
• CPC Class Codes: G06F 16/332 Query formulation
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 16/9554 by using bar codes
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06F 40/14 Tree-structured documents parsing G06F40/205; validation G06F40/226
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06K 17/0016 : Selecting or retrieving of images by means of their associated code-marks, e.g. coded microfilm or microfiche microfilm reading G03B21/11; computers therefor G06F16/00; teaching devices G09B5/02; accessing record carriers G11B27/002; static digital stores using optical elements G11C13/04; intermediate storage of facsimile picture signals H04N1/21
• CPC Class Codes: G06Q 10/00 : Administration; Management
• CPC Class Codes: G06Q 10/10 : Office automation; Time management
• CPC Class Codes: G06Q 20/102 : Bill distribution or payments
• CPC Class Codes: G06Q 20/12 : specially adapted for electronic shopping systems
• CPC Class Codes: G06Q 20/4012 : Verifying personal identification numbers [PIN]
• CPC Class Codes: G06Q 30/00 : Commerce
• CPC Class Codes: G06Q 30/02 : Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0207 : Discounts or incentives, e.g. coupons or rebates
• CPC Class Codes: G06Q 30/0241 : Advertisements
• CPC Class Codes: G06Q 30/0283 : Price estimation or determination
• CPC Class Codes: G06Q 30/04 : Billing or invoicing
• CPC Class Codes: G06Q 30/06 : Buying, selling or leasing transactions
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0603 : Catalogue ordering
• CPC Class Codes: G06Q 30/0623 : Item investigation
• CPC Class Codes: G06Q 30/0641 : Shopping interfaces
• CPC Class Codes: G06Q 30/0643 : Graphical representation of items or shoppers
• CPC Class Codes: G06Q 40/04 : Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06V 30/10 : Character recognition
• CPC Class Codes: G06V 30/142 : using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/1444 : Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields
• CPC Class Codes: G06V 30/1448 : based on markings or identifiers characterising the document or the area
• CPC Class Codes: G06V 30/248 : involving plural approaches, e.g. verification by template match; Resolving confusion among similar patterns, e.g. "O" versus "Q" G06V30/242 takes precedence
• CPC Class Codes: G06V 30/262 : using context analysis, e.g. lexical, syntactic or semantic context
• CPC Class Codes: G06V 30/40 : Document-oriented image-based pattern recognition
• CPC Class Codes: G07F 17/40 : for devices for accepting orders, advertisements, or the like
• CPC Class Codes: G07G 1/00 : Cash registers alarm indicators G07G3/00
• CPC Class Codes: H04N 1/00204 : with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00244 : with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 1/00326 : with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus arrangements for the associated working of recording or reproducing apparatus with related apparatus G11B31/00
• CPC Class Codes: H04N 1/0036 : Alphanumeric symbols
• CPC Class Codes: H04N 1/00363 : Bar codes or the like
• CPC Class Codes: H04N 1/00374 : on the same page as at least a part of the image
• CPC Class Codes: H04N 1/00381 : Input by recognition or interpretation of visible user gestures manual input means, e.g. digitisers, writing tablets H04N1/00392; gesture input to computers G06F3/017; recognition algorithms G06V40/20
• CPC Class Codes: H04N 1/107 : with manual scanning
• CPC Class Codes: H04N 1/2166 : for mass storage, e.g. in document filing systems information retrieval G06F16/00
• CPC Class Codes: H04N 1/2187 : with image input from a plurality of different locations or from a non-central location, e.g. from one or more users
• CPC Class Codes: H04N 1/32128 : attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 1/34 : for coin-freed systems ; Pay systems
• CPC Class Codes: H04N 2201/0081 : Image reader H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/3215 : of a time or duration
• CPC Class Codes: H04N 2201/3249 : data relating to a linked page or object, e.g. hyperlink
• CPC Class Codes: H04N 2201/3253 : Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 2201/3263 : of a graphical motif or symbol, e.g. Christmas symbol, logo H04N2201/3211 takes precedence
• CPC Class Codes: H04N 2201/3266 : of text or character information, e.g. text accompanying an image H04N2201/3226 takes precedence
• CPC Class Codes: H04N 2201/3277 : The additional information being stored in the same storage device as the image data
• CPC Class Codes: H04N 2201/3278 : Transmission
• CPC Class Codes: Y02A 90/10 : Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
• CPC Class Codes: Y10S 707/99931 : Database or file accessing
• CPC Class Codes: Y10S 707/99934 : Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99936 : Pattern matching access
• CPC Class Codes: Y10S 707/99943 : Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99945 : Object-oriented database structure processing

Abstract

A facility for initiating a purchase is described. The facility receives a text sequence captured by a user from a rendered document using a handheld text capture device. The facility identifies in the received text sequence a reference to a distinguished product. In response to identifying the reference, the facility presents to the user an opportunity to place an order for the established product. If the user accepts the presented opportunity to order the distinct product, the facility orders the distinct product on behalf of the user.

Claims

1. A method in a computing system for initiating a purchase, comprising:

receiving a text sequence captured by a user from a rendered document using a handheld text capture device;

identifying in the received text sequence a reference to a distinguished product;

in response to the identifying, presenting to the user an opportunity to purchase the distinguished product; and

if the user accepts the opportunity to purchase the distinguished product, purchasing the distinguished product on behalf of the user.

2. The method of claim 1 wherein both a reference to a first product and a reference to a second product are identified in the received text sequence, the method further comprising:

assessing availability of each of the referenced products for purchase, wherein the first product is assessed to be available in the second product is assessed to be unavailable, and wherein an opportunity to purchase the first product is presented to the user and no opportunity to purchase the second product is presented to the user.

3. The method of claim 1, further comprising retrieving terms for purchasing the distinguished product, wherein the retrieved terms are presented to the user as part of presenting the opportunity to purchase the distinguished product to the user.

4. The method of claim 1 wherein the identifying and presenting are performed at a time contemporaneous with the capture of the text sequence.

5. The method of claim 4 wherein the opportunity to purchase the distinguished product is presented within a context menu of actions that the user may take relating to the capture of the text sequence.

6. The method of claim 4 wherein the opportunity to purchase the distinguished product is presented on a display integrated into the handheld text capture device.

7. The method of claim 4 wherein the opportunity to purchase the distinguished product is presented on a display that is separate from but physically near the handheld text capture device.

8. The method of claim 1, further comprising storing an indication of the text sequence capture in a capture log, wherein the presenting is performed in connection with later retrieving and displaying contents of the capture log including the stored indication.

9. The method of claim 1 wherein the distinguished product is purchased from a distinguished seller, the method further comprising collecting a referral fee from the distinguished seller in connection with the purchase.

10. The method of claim 9, further comprising:

identifying the document from which the text sequence capture was performed;

identifying the source of the identified document; and

initiating payment to the identified source of a portion of the collected referral fee.

11. The method of claim 10 wherein the identified source is a publisher.

12. The method of claim 10 wherein the identified source is an author.

13. A computer-readable medium whose contents cause a computing system to perform a method for initiating a purchase, the method comprising:

receiving a text sequence captured by a user from a rendered document using a handheld text capture device;

identifying in the received text sequence a reference to a distinguished product;

in response to the identifying, presenting to the user an opportunity to order the distinguished product; and

if the user accepts the opportunity to order the distinguished product, ordering the distinguished product on behalf of the user.

14. A method in a computing system for document identification, comprising:

receiving a first text sequence captured by a user from a first rendered document using a handheld text capture device;

using explicit context information to identify among a corpus of documents a first proper subset of documents from which the first text sequence capture could have been performed;

identifying a first document among the first proper subset of documents from which the first text sequence capture was performed;

receiving a second text sequence captured by a user from a second rendered document using a handheld text capture device;

using implicit context information to identify among the corpus of documents a second proper subset of documents from which the second text sequence capture could have been performed; and

identifying a second document among the second proper subset of documents from which the second text sequence capture was performed.

15. The method of claim 14 wherein the explicit context information is a unique document identifier captured from the identified first document.

16. The method of claim 14 wherein the implicit context information is a text sequence that occurs only in the second subset of documents.

17. The method of claim 14 wherein the implicit context information is a captured watermark that occurs only in the second subset of documents.

18. The method of claim 14 wherein the implicit context information is a captured text spacing pattern that occurs only in the second subset of documents.

19. The method of claim 14 wherein the implicit context information is a captured ink color that occurs only in the second subset of documents.

20. The method of claim 14 wherein the implicit context information is a captured font that occurs only in the second subset of documents.

21. The method of claim 14 wherein the implicit context information is the elapsed amount of time between the capture of the second text sequence and the text sequence capture occurring immediately before the capture of the second text sequence.

22. The method of claim 14 wherein the implicit context information is a comparison of a font encountered in the second text sequence to a font encountered in the text sequence captured immediately before the second text sequence.

23. The method of claim 14 wherein the implicit context information is a comparison of an ink color encountered in the second text sequence to an ink color encountered in the text sequence captured immediately before the second text sequence.

24. The method of claim 14 wherein the implicit context information is a comparison of a text spacing pattern encountered in the second text sequence to a text spacing pattern encountered in the text sequence captured immediately before the second text sequence.

25. The method of claim 14 wherein the implicit context information is an assessment of the user'"'"'s reading habits.

26. The method of claim 14 wherein the implicit context information is a list of periodicals to which the user subscribes.

27. The method of claim 14 wherein the implicit context information is a measure of availability of documents among the second subset.

28. The method of claim 14 wherein the implicit context information is a measure of popularity among readers of documents among the second subset.

29. The method of claim 14 wherein the implicit context information is the user'"'"'s current physical circumstance.

30. The method of claim 14, further comprising ascribing a reliability factor to the implicit context information on which the second proper subset of documents is based.

31. A computing system for document identification, comprising:

a text sequence receiving subsystem that receives text sequences each captured by a user from a rendered document using a handheld text capture device;

an explicit context derivation subsystem that derives explicit context information relating to the capture of at least a subset of the text sequences received by the text sequence receiving subsystem;

an implicit context derivation subsystem that derives implicit context information relating to the capture of at least a subset of the text sequences received by the text sequence receiving subsystem; and

a document identification subsystem that, for each text sequence received by the text sequence receiving subsystem, identifies documents among a corpus of documents from which the text sequence could have been captured, based both upon the explicit context information derived by the explicit context duration subsystem and the implicit context information derived by the implicit context derivation subsystem.

32. A handheld text capture device, comprising:

a text capture subsystem for capturing text sequences from one or more rendered documents as directed by a user;

a memory containing at least one parametric context description that can be applied to text sequences captured by the text capture subsystem to determine context information for particular text sequences; and

a processor for applying the parametric context descriptions contained in the memory to text sequences captured by the text capture subsystem to determine context information for particular text sequences.

33. The handheld text capture device of claim 32, further comprising a communication subsystem for communicating at least a portion of the text sequences captured by the text capture subsystem to an external computer system.

34. The handheld text capture device of claim 32 wherein a parametric context description contained in the memory indicates a minimum size at which captured text sequences are expected to be unique.

35. The handheld text capture device of claim 32 wherein a parametric context description contained in the memory indicates a URL to be dereferenced or the captured text sequence matches a particular pattern.

36. The handheld text capture device of claim 32 wherein a parametric context description contained in the memory indicates that it is to be applied to text sequences captured from all documents.

37. The handheld text capture device of claim 32 wherein a parametric context description contained in the memory indicates that it is to be applied to text sequences captured from an identified subset of documents.

38. The handheld text capture device of claim 37 wherein the processor determines whether each text sequence was captured from a document among the identified subset of documents based upon a unique identifier for one of the documents of the identified subset contained in an earlier-captured text sequence.

39. A method in a portable device, comprising:

capturing text sequences from one or more rendered documents as directed by a user;

applying at least one parametric context description to the captured text sequences; and

determining context information for particular text sequences based on the applied parametric context descriptions.

40. The method of claim 39, further comprising communicating at least a portion of the captured text sequences to an external computer system.

41. The method of claim 39 wherein a parametric context description indicates a minimum size at which captured text sequences are expected to be unique.

42. The method of claim 39 wherein a parametric context description indicates a URL to be dereferenced or the captured text sequence matches a particular pattern.

43. The method of claim 39 wherein a parametric context description indicates that it is to be applied to text sequences captured from all documents.

44. The method of claim 39 wherein a parametric context description indicates that it is to be applied to text sequences captured from an identified subset of documents.

45. The method of claim 44, further comprising determining whether each text sequence was captured from a document among the identified subset of documents based upon a unique identifier for one of the documents of the identified subset contained in an earlier-captured text sequence.