Archive of text captures from rendered documents

• US 7,421,155 B2
• Filed: 04/01/2005
• Issued: 09/02/2008
• Est. Priority Date: 02/15/2004
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Exbiblio BV
• Inventors: Stafford-Fraser, James Q., Kushler, Clifford A., Grover, Dale L., King, Martin T.
• Primary Examiner(s): Couso, Jose L.
• Application Number:
  US11/098,016
• Publication Number:
  US 20060050996A1
• Time in Patent Office: 1,250 Days
• Field of Search: 382305-306, 382312-313, 382317-319, 382321-324, 382/187, 358/462, 358/473, 358/474, 705/26, 705/27, 707/1, 707/3, 707 9- 10, 707/100, 707/101, 707/104.1, 715/500, 715/503, 715/531, 715/700, 715/733, 715/741
• US Class Current: 382/312
• CPC Class Codes: G06F 16/316 Indexing structures
• CPC Class Codes: G06F 16/332 Query formulation
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 16/9554 by using bar codes
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06F 40/14 Tree-structured documents parsing G06F40/205; validation G06F40/226
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06K 17/0016 : Selecting or retrieving of images by means of their associated code-marks, e.g. coded microfilm or microfiche microfilm reading G03B21/11; computers therefor G06F16/00; teaching devices G09B5/02; accessing record carriers G11B27/002; static digital stores using optical elements G11C13/04; intermediate storage of facsimile picture signals H04N1/21
• CPC Class Codes: G06Q 10/00 : Administration; Management
• CPC Class Codes: G06Q 10/10 : Office automation; Time management
• CPC Class Codes: G06Q 20/102 : Bill distribution or payments
• CPC Class Codes: G06Q 20/12 : specially adapted for electronic shopping systems
• CPC Class Codes: G06Q 20/4012 : Verifying personal identification numbers [PIN]
• CPC Class Codes: G06Q 30/00 : Commerce
• CPC Class Codes: G06Q 30/02 : Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0207 : Discounts or incentives, e.g. coupons or rebates
• CPC Class Codes: G06Q 30/0241 : Advertisements
• CPC Class Codes: G06Q 30/0283 : Price estimation or determination
• CPC Class Codes: G06Q 30/04 : Billing or invoicing
• CPC Class Codes: G06Q 30/06 : Buying, selling or leasing transactions
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0603 : Catalogue ordering
• CPC Class Codes: G06Q 30/0623 : Item investigation
• CPC Class Codes: G06Q 30/0641 : Shopping interfaces
• CPC Class Codes: G06Q 30/0643 : Graphical representation of items or shoppers
• CPC Class Codes: G06Q 40/04 : Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06V 30/10 : Character recognition
• CPC Class Codes: G06V 30/142 : using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/1444 : Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields
• CPC Class Codes: G06V 30/1448 : based on markings or identifiers characterising the document or the area
• CPC Class Codes: G06V 30/248 : involving plural approaches, e.g. verification by template match; Resolving confusion among similar patterns, e.g. "O" versus "Q" G06V30/242 takes precedence
• CPC Class Codes: G06V 30/262 : using context analysis, e.g. lexical, syntactic or semantic context
• CPC Class Codes: G06V 30/40 : Document-oriented image-based pattern recognition
• CPC Class Codes: G07F 17/40 : for devices for accepting orders, advertisements, or the like
• CPC Class Codes: G07G 1/00 : Cash registers alarm indicators G07G3/00
• CPC Class Codes: H04N 1/00204 : with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00244 : with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 1/00326 : with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus arrangements for the associated working of recording or reproducing apparatus with related apparatus G11B31/00
• CPC Class Codes: H04N 1/0036 : Alphanumeric symbols
• CPC Class Codes: H04N 1/00363 : Bar codes or the like
• CPC Class Codes: H04N 1/00374 : on the same page as at least a part of the image
• CPC Class Codes: H04N 1/00381 : Input by recognition or interpretation of visible user gestures manual input means, e.g. digitisers, writing tablets H04N1/00392; gesture input to computers G06F3/017; recognition algorithms G06V40/20
• CPC Class Codes: H04N 1/107 : with manual scanning
• CPC Class Codes: H04N 1/2166 : for mass storage, e.g. in document filing systems information retrieval G06F16/00
• CPC Class Codes: H04N 1/2187 : with image input from a plurality of different locations or from a non-central location, e.g. from one or more users
• CPC Class Codes: H04N 1/32128 : attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 1/34 : for coin-freed systems ; Pay systems
• CPC Class Codes: H04N 2201/0081 : Image reader H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/3215 : of a time or duration
• CPC Class Codes: H04N 2201/3249 : data relating to a linked page or object, e.g. hyperlink
• CPC Class Codes: H04N 2201/3253 : Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 2201/3263 : of a graphical motif or symbol, e.g. Christmas symbol, logo H04N2201/3211 takes precedence
• CPC Class Codes: H04N 2201/3266 : of text or character information, e.g. text accompanying an image H04N2201/3226 takes precedence
• CPC Class Codes: H04N 2201/3277 : The additional information being stored in the same storage device as the image data
• CPC Class Codes: H04N 2201/3278 : Transmission
• CPC Class Codes: Y02A 90/10 : Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
• CPC Class Codes: Y10S 707/99931 : Database or file accessing
• CPC Class Codes: Y10S 707/99934 : Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99936 : Pattern matching access
• CPC Class Codes: Y10S 707/99943 : Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99945 : Object-oriented database structure processing

Abstract

A facility for storing a text capture data structure for a particular user is described. The data structure comprises a number of entries. Each entry corresponds to a text capture operation performed by the user from a rendered document. Each entry contains information specifying the text captured in the text capture operation.

Claims

1. One or more computer memories collectively containing a text capture data structure for a distinguished user, comprising a plurality of entries each corresponding to a text capture operation performed by the distinguished user from a rendered document using a handheld capture device, each entry containing information specifying text captured in the text capture operation and being usable to perform an action with respect to an electronic document found to contain the specified text and generated before the text capture operation was performed.

2. The computer memories of claim 1 wherein each entry further contains information identifying an electronic document containing the text captured in the text capture operation.

3. The computer memories of claim 1 wherein each entry further contains information specifying a time at which the text capture operation was performed.

4. The computer memories of claim 1 wherein each entry further contains information specifying a location at which the text capture operation was performed.

5. The computer memories of claim 1 wherein the data structure is accessible to the distinguished user.

6. The computer memories of claim 1 wherein the data structure is available to a user other than the distinguished user.

7. The computer memories of claim 1 wherein the data structure is accessible to a plurality of users.

11. The computer memories of claim 1 wherein the text capture operations to which the entries correspond were performed using one or more text capture devices, and wherein the computer memories are physically distinct from any text capture device.

47. The computer memories of claim 1 wherein the text capture data structure further comprises information identifying at least one user of other than the distinguished user and indicating that the distinguished user has authorized access to contents of the text capture data structure by any of the identified users.

8. A system for constructing a text capture history, comprising:

a handheld text capture device that captures textual passages from rendered documents in response to user control; and

a computer memory that is distinct from the handheld text capture device, in which is stored both (a) information identifying each textual passage captured by the handheld text capture device, and (b) information identifying, for at least one of the textual passages captured by the handheld text capture device, an electronic document and a point in the document at which the capture of the textual passage was automatically determined to have been performed by the handheld text capture device, the electronic document being generated before the textual passage was captured by the handheld text capture device.

9. The system of claim 8 wherein the handheld text capture device captures textual passages from rendered documents by visually capturing images of the textual passages.

10. The computing system of claim 8 wherein the handheld text capture device captures textual passages from rendered documents by capturing audio clips of a user reading aloud from the rendered documents.

48. The system of claim 8, further comprising a processor that is distinct from a handheld text capture device on which executes an electronic mail client that receives from the handheld text capture device textual passages captured from rendered documents, and that stores in the computer memory information identifying each textual passage captured by the handheld text capture device.

49. The system of claim 8 wherein the handheld text capture device is a wireless phone.

50. The system of claim 8, further comprising an RSS feed generator that generates an RSS feed to update feed subscribers on changes to the information contained in the memory.

12. A method in a computing system for enabling a distinguished user to interact with a text capture history, comprising:

displaying an indication of each of a plurality of text capture operations performed by the distinguished user from rendered documents using a handheld text capture device, each indicated text capture operation designating less than a whole page of text as captured;

receiving user input with respect to a distinguished one of the displayed indications; and

performing an action relative to the text capture operation indicated by the distinguished indication in accordance with the received user input.

13. The method of claim 12 wherein each displayed indication includes an indication of a time at which the text capture operation was performed.

14. The method of claim 12 wherein each displayed indication includes an indication of a geographic location at which the text capture operation was performed.

15. The method of claim 12 wherein each displayed indication includes an indication of text captured during the text capture operation.

16. The method of claim 12 wherein each displayed indication includes information identifying a document containing the text captured during the text capture operation.

17. The method of claim 12 wherein each displayed indication includes information identifying a location in a document -where the text capture operation occurs.

18. The method of claim 12 wherein each displayed indication includes an indication of raw information captured during the text capture operation.

19. The method of claim 18 when the raw information is an image captured during the text capture operation.

20. The method of claim 18 when the raw information is an audio clip captured during the text capture operation.

21. The method of claim 12 wherein each displayed indication includes a contextual passage of an electronic document containing the text captured during the text capture operation.

22. The method of claim 12 wherein the action is performed with respect to a document determined to contain the text captured during the text capture operation.

23. The method of claim 12 wherein the action is performed with respect to literal contents of the text captured during the text capture operation.

24. The method of claim 12 wherein the performed action is displaying a menu of actions available relative to the text capture operation.

25. The method of claim 24 wherein the performed action is displaying a menu of actions available relative to a document determined to contain the text captured during the text capture operation.

26. The method of claim 24 wherein the performed action is displaying a menu of actions available relative to literal contents of the text captured during the text capture operation.

27. The method of claim 12 wherein the indications are displayed as part of a web page.

28. The method of claim 12 wherein the indications are displayed as part of a general-purpose electronic mail client application.

29. The method of claim 12 wherein the indications are displayed as part of a dedicated client application.

30. The method of claim 12 wherein the indications are displayed as part of a desktop search application.

31. The method of claim 12 wherein the indications are displayed as part of an Internet search application.

51. The method of claim 12 wherein the action performed is purchasing a product relating to the text capture indication indicated by the distinguished indication.

52. The method of claim 12 wherein the action performed is attaching an annotation to the distinguished indication.

53. The method of claim 12 wherein the action performed is launching an application program to display contents of an electronic document from which the rendered document in which the text capture operation indicated by the distinguished indication was performed.

54. The method of claim 12 wherein the action performed is launching an application program to edit contents of an electronic document from which the rendered document in which the text capture operation indicated by the distinguished indication was performed.

55. The method of claim 12 wherein the action performed is sending an email message containing text from the rendered document in which the text capture operation indicated by the distinguished indication was performed that includes the text designated as captured by the text capture operation indicated by the distinguished indication.

56. The method of claim 12 wherein the action performed is registering the distinguished user'"'"'s interest in a topic to which the text capture operation indicated by the distinguished indication relates.

57. The method of claim 12 wherein the action performed is displaying supplemental materials related to the text capture operation indicated by the distinguished indication.

58. The method of claim 12 wherein an operating system is executing in the computing system and wherein the indications are displayed by the operating system as part of a list of most recently used documents that also includes an indication of at least one electronic document stored in the computing system.

59. The method of claim 12 wherein an operating system is executing in the computing system and wherein the indications are displayed in a manner that resembles a list of electronic documents stored in the computer system.

60. The method of claim 17 wherein information identifying a location in a document where the text capture operation occurred and the displayed indication of a distinguished text capture operation of the plurality is generated at least in part based upon text capture operations performed by the distinguished user before the distinguished text capture operation.

61. The method of claim 17 wherein information identifying a location in a document where the text capture operation occurred and the displayed indication of a distinguished text capture operation of the plurality is generated at least in part based upon text capture operations performed by the distinguished user after the distinguished text capture operation.

62. The method of claim 17 wherein information identifying a location in a document where the text capture operation occurred and the displayed indication of a distinguished text capture operation of the plurality is generated at least in part based upon disambiguating input received from the distinguished user with respect to the displayed indication for the distinguished text capture operation.

32. A method in a computing system for providing access to information related to text capture operations performed by a user, comprising:

receiving indications, each indication indicating a text capture operation performed by the user from a rendered document designating less than a whole page of text as captured;

storing the received indications;

generating a display for the user based upon the stored indications; and

receiving payment from the user authorizing the user to access an electronic document related to one of the performed text capture operations.

33. The method of claim 32 wherein the generating is performed in response to receiving payment.

34. The method of claim 32 wherein the storing is performed in response to receiving payment.

35. The method of claim 32, further comprising maintaining the stored indications in response to receiving payment.

36. The method of claim 32, further comprising, for a distinguished one of the received indications:

identifying an electronic document corresponding to the rendered document from which the indicated text capture was performed; and

insuring that a copy of the identified electronic document remains accessible to the user for a future period of time.

37. The method of claim 32 wherein each of the text captures indicated by a received indication yielded a text sequence, and wherein the rendered document contains at least one line of text, and wherein the text sequence yielded by the text capture operation is a proper subset of the text contained in a single line of the rendered document.

38. The method of claim 32 wherein each of the text captures indicated by a received indication yielded a text sequence, and wherein the yielded text sequence is comprised of words, and wherein the text capture operation involved specific user interactions with each of the words of the yielded text sequence.

39. The method of claim 38 wherein the user spoke each of the words of the yielded text sequence.

40. The method of claim 38 wherein the user directed an optical sensor at each of the words of the yielded text sequence.

41. The method of claim 32 wherein each of the text captures indicated by a received indication yielded a text sequence, and wherein the yielded text sequence is comprised of ordered words, and wherein the text capture operation involved capturing physical phenomenon corresponding to each of the words of the yielded text sequence in the order of the yielded text sequence.

42. The method of claim 32 wherein each of the text captures indicated by a received indication yielded a text sequence, and wherein the yielded text sequence is comprised of ordered words, and wherein the text capture operation involved capturing physical phenomenon corresponding to each of the words of the yielded text sequence in the reverse order of the yielded text sequence.

43. The method of claim 32 wherein each of the text captures indicated by a received indication yielded a text sequence, and wherein the text capture operation involved manually moving an optical sensor across the rendered document.

44. The method of claim 32 wherein each of the text captures indicated by a received indication yielded a text sequence, and wherein the text capture operation involved capturing image data from a non-rectangular region of the rendered document.

45. The method of claim 32 wherein each of the text captures indicated by a received indication yielded a text sequence, and wherein the text sequence yielded by the text capture operation comprises fewer than ten words.

46. The method of claim 32 wherein each of the text captures indicated by a received indication yielded a text sequence, and wherein the text capture operation involved capturing an image of fewer than ten words.

63. A method and a computing system for presenting advertising messages, comprising:

receiving a plurality of text capture operations performed by a distinguished user from rendered documents using a handheld text capture device, each indicated text capture operation designating less than a whole page of text as captured;

analyzing the plurality of text capture operations to identify a likely interest of the distinguished user;

selecting an advertising message relating to the identified interest; and

presenting the selected advertising message to the distinguished user.

64. A method in a computing system for facilitating social interactions, comprising:

for each of a plurality of users, receiving a plurality of text capture operations performed by the user from rendered documents using a handheld text capture device, each indicated text capture operation designating less than a whole page of text as captured;

received from the user for each of the plurality of users, analyzing the plurality of text capture operations to identify a likely interest of the user;

identifying a pair of users for which the same likely interest is identified; and

presenting to a first member of the identified pair of users an indication that the other member of the identified pair of users shares a likely interest.