Search engines and systems with handheld document data capture devices

• US 10,635,723 B2
• Filed: 07/20/2011
• Issued: 04/28/2020
• Est. Priority Date: 02/15/2004
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: King, Martin T., Grover, Dale L., Kushler, Clifford A., Stafford-Fraser, James Q.
• Primary Examiner(s): Torgrimson, Tyler J
• Application Number:
  US13/186,908
• Publication Number:
  US 20120041941A1
• Time in Patent Office: 3,205 Days
• Field of Search: 707706, 707707, 707708, 707709, 707710, 707712
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/316 Indexing structures
• CPC Class Codes: G06F 16/332 Query formulation
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/483 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06F 7/00 Methods or arrangements for processing data by operating upon the order or content of the data handled logic circuits H03K19/00
• CPC Class Codes: G06K 17/0016 Selecting or retrieving of images by means of their associated code-marks, e.g. coded microfilm or microfiche microfilm reading G03B21/11; computers therefor G06F16/00; teaching devices G09B5/02; accessing record carriers G11B27/002; static digital stores using optical elements G11C13/04; intermediate storage of facsimile picture signals H04N1/21
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 20/102 Bill distribution or payments
• CPC Class Codes: G06Q 20/12 : specially adapted for electronic shopping systems
• CPC Class Codes: G06Q 20/4012 : Verifying personal identification numbers [PIN]
• CPC Class Codes: G06Q 30/00 : Commerce
• CPC Class Codes: G06Q 30/02 : Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0207 : Discounts or incentives, e.g. coupons or rebates
• CPC Class Codes: G06Q 30/0241 : Advertisements
• CPC Class Codes: G06Q 30/0283 : Price estimation or determination
• CPC Class Codes: G06Q 30/04 : Billing or invoicing
• CPC Class Codes: G06Q 30/06 : Buying, selling or leasing transactions
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0603 : Catalogue ordering
• CPC Class Codes: G06Q 30/0623 : Item investigation
• CPC Class Codes: G06Q 30/0641 : Shopping interfaces
• CPC Class Codes: G06Q 30/0643 : Graphical representation of items or shoppers
• CPC Class Codes: G06Q 40/04 : Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06V 30/10 : Character recognition
• CPC Class Codes: G06V 30/142 : using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/1444 : Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields
• CPC Class Codes: G06V 30/1448 : based on markings or identifiers characterising the document or the area
• CPC Class Codes: G06V 30/248 : involving plural approaches, e.g. verification by template match; Resolving confusion among similar patterns, e.g. "O" versus "Q" G06V30/242 takes precedence
• CPC Class Codes: G06V 30/262 : using context analysis, e.g. lexical, syntactic or semantic context
• CPC Class Codes: G06V 30/40 : Document-oriented image-based pattern recognition
• CPC Class Codes: G07F 17/40 : for devices for accepting orders, advertisements, or the like
• CPC Class Codes: H04N 1/00204 : with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00244 : with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 1/00326 : with a data reading, recognizing or recording apparatus, e.g. with a bar-code apparatus arrangements for the associated working of recording or reproducing apparatus with related apparatus G11B31/00
• CPC Class Codes: H04N 1/0036 : Alphanumeric symbols
• CPC Class Codes: H04N 1/00363 : Bar codes or the like
• CPC Class Codes: H04N 1/00374 : on the same page as at least a part of the image
• CPC Class Codes: H04N 1/00381 : Input by recognition or interpretation of visible user gestures manual input means, e.g. digitisers, writing tablets H04N1/00392; gesture input to computers G06F3/017; recognition algorithms G06V40/20
• CPC Class Codes: H04N 1/107 : with manual scanning
• CPC Class Codes: H04N 1/2166 : for mass storage, e.g. in document filing systems information retrieval G06F16/00
• CPC Class Codes: H04N 1/2187 : with image input from a plurality of different locations or from a non-central location, e.g. from one or more users
• CPC Class Codes: H04N 1/32128 : attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 1/34 : for coin-freed systems ; Pay systems
• CPC Class Codes: H04N 2201/0081 : Image reader H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/3215 : of a time or duration
• CPC Class Codes: H04N 2201/3249 : data relating to a linked page or object, e.g. hyperlink
• CPC Class Codes: H04N 2201/3253 : Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 2201/3263 : of a graphical motif or symbol, e.g. Christmas symbol, logo H04N2201/3211 takes precedence
• CPC Class Codes: H04N 2201/3266 : of text or character information, e.g. text accompanying an image H04N2201/3226 takes precedence
• CPC Class Codes: H04N 2201/3277 : The additional information being stored in the same storage device as the image data
• CPC Class Codes: H04N 2201/3278 : Transmission
• CPC Class Codes: Y10S 707/99931 : Database or file accessing
• CPC Class Codes: Y10S 707/99934 : Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99936 : Pattern matching access
• CPC Class Codes: Y10S 707/99943 : Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99945 : Object-oriented database structure processing

Abstract

Embodiments of the disclosed innovations provide systems and methods for locating data associated with rendered documents. Some embodiments support the use of a handheld document data capture device.

Claims

1. A system for locating and retrieving information, comprising:

an aggregated search index comprising document indexes from multiple restricted sources, wherein the aggregated search index also comprises indications of whether indexed electronic content is an electronic counterpart to one or more rendered documents; and

one or more computing servers comprising a search engine, wherein the search engine is operatively coupled with the aggregated search index, wherein the search engine is configured to determine whether information associated with a search query is based on information captured by a capture device from a rendered document and to search the aggregated search index in response to receiving the search query.

2. The system of claim 1, wherein the indexed electronic content comprises:

at least one of a web page, image, text, or word processing file.

3. The system of claim 1, wherein the document indexes from multiple restricted sources comprise:

indexes specific to at least one of particular publications or periodicals.

4. The system of claim 1, wherein the search engine is configured to receive search queries comprising at least one selected from the group of a time stamp, a location stamp, a capture device identifier, a billing identifier, and a subscriber identifier.

5. The system of claim 1, wherein the aggregated search index is stored in a first storage device, wherein at least a portion of the aggregated search index is copied from the first storage device to a second storage device that is accessible to the capture device.

6. The system of claim 1, wherein the search engine is configured to perform operations further comprising:

determining a scope of a search, wherein the determining comprises using scan context information to select an index or a portion of the aggregated index applied to the search, wherein the scan context information comprises supplemental information relevant to a capture of information included with the search query.

7. The system of claim 1, wherein the aggregated search index further comprises:

indications of whether indexed electronic content is access-restricted.

8. The system of claim 1, wherein the search engine comprises an access component, wherein the access component is configured to perform operations comprising:

receiving a user selection of a search result;

determining that the search result corresponds to restricted content;

determining that the user is a subscriber or other party having authorization to access the restricted content; and

providing the user with access to the restricted content.

9. The system of claim 1, wherein the search engine is configured to perform operations further comprising:

determining which access permissions for electronic content are available to a user or users of the capture device.

10. The system of claim 9, wherein determining which access permissions for the electronic content are available to a user or users of the capture device comprises:

determining that the user is a subscriber to a publisher of the electronic content.

11. The system of claim 9, wherein determining which access permissions for the electronic content are available to a user or users of the capture device comprises:

determining that registration with a content publisher is required to access the electronic content; and

assisting the user of the capture device with registration with the content publisher.

12. The system of claim 1, wherein the aggregated search index further comprises statistical information indicating a popularity of a rendered document based on an amount of capture activity from the rendered document, wherein the popularity is used to determine a priority of electronic content associated with the rendered document.

13. A method in a computing system of locating information, comprising:

receiving a search query initiated by a capture device;

determining whether the search query results from information captured from a rendered document;

searching an aggregated search index in response to receiving the search query, wherein the aggregated search index comprises document indexes from multiple restricted sources and indications of whether indexed electronic content is an electronic counterpart to one or more rendered documents; and

providing, to the capture device, information associated with search results corresponding to the search query.

14. The method of claim 13, further comprising determining a scope of a search, wherein the determining comprises using scan context information to select an index or a portion of the aggregated search index applied to the search, wherein the scan context information comprises supplemental information relevant to a capture of information included with the search query.

15. The method of claim 13, wherein the aggregated search index further comprises indications of whether electronic content is access-restricted, the method further comprising:

receiving information associated with an identification of the capture device;

determining that the capture device is authorized to access one or more restricted sources; and

in response to the determining, searching one or more of the document indexes from the one or more restricted sources.

16. The method of claim 13, further comprising determining whether registration with a content publisher is required for accessing the information associated with results of the search.

17. A non-transitory computer-readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions of:

receiving a search query initiated by a capture device;

determining whether the search query results from information captured from a rendered document;

searching an aggregated search index in response to receiving the search query, wherein the aggregated search index comprises document indexes from multiple restricted sources and indications of whether indexed electronic content is an electronic counterpart to one or more rendered documents; and

providing, to the capture device, information associated with search results corresponding to the search query.

18. The computer-readable medium of claim 17, further comprising instructions executable for:

receiving information associated with an identification of a rendered document from which the search query is based.

19. The computer-readable medium of claim 18, further comprising instructions executable for:

searching the aggregated index for electronic content associated with the rendered document using the information associated with the identity of the rendered document and the search query.

20. The computer-readable medium of claim 17, further comprising instructions executable for:

receiving information associated with an identification of the capture device; and

searching the aggregated index using the search query and information associated with the identification of the capture device.

21. A system for locating and retrieving information, comprising:

an aggregated search index comprising;

document indexes from multiple restricted sources; and

indications of whether indexed electronic content is an electronic counterpart to one or more rendered documents; and

one or more computing servers comprising a search engine, wherein the search engine is operatively coupled with the aggregated search index, and wherein the search engine is configured to perform operations comprising;

receiving a search query initiated by a capture device;

searching the aggregated search index in response to receiving the search query and determining that an access condition is satisfied, wherein determining that the access condition is satisfied comprises determining that the search query includes information indicating that the search query is based on information captured from a rendered document;

determining one or more search results that correspond to the search query and to the electronic content; and

providing the one or more search results to the capture device.

22. The system of claim 21, wherein the electronic content comprises at least one of a web page, image, text, or word processing file.

23. The system of claim 21, wherein the document indexes from multiple restricted sources comprise indexes specific to at least one of particular publications or periodicals.

24. The system of claim 21, wherein the search engine is configured to receive search queries comprising at least one selected from the group of a time stamp, a location stamp, a capture device identifier, a billing identifier, and a subscriber identifier.

25. The system of claim 21, wherein the aggregated search index is stored in a first storage device, wherein the search engine is configured to perform operations further comprising copying at least a portion of the aggregated search index from the first storage device to a second storage device that is more readily accessible to the capture device initiating the search query.

26. The system of claim 21, wherein the search engine is configured to perform operations further comprising:

determining a scope of the search query based on scan context information related to a scan of the rendered document by the capture device; and

selecting a portion of the aggregated index to be searched over, based on the scope of the search query.

27. The system of claim 21, wherein the aggregated search index further comprises indications of whether indexed electronic content is access-restricted.

28. The system of claim 21, wherein the search engine comprises an access component, wherein the access component is configured to perform operations comprising:

receiving a user selection of a search result from the capture device;

determining that the search result corresponds to restricted content;

determining that the user is a subscriber or other party having authorization to access the restricted content; and

providing the user with access to the restricted content.

29. The system of claim 21, wherein the search engine is configured to perform operations further comprising:

determining which access permissions for electronic content are available to a user or users of the capture device.

30. The system of claim 29, wherein determining which access permissions for the electronic content are available to a user or users of the capture device comprises:

determining that the user is a subscriber to a publisher of the electronic content.

31. The system of claim 29, wherein determining which access permissions for the electronic content are available to a user or users of the capture device comprises:

determining that registration with a content publisher is required to access the electronic content; and

assisting the user of the capture device with registration with the content publisher.

32. The system of claim 21, wherein the aggregated search index further comprises statistical information indicating a popularity of the rendered document based on an amount of capture activity from the rendered document, wherein the search engine assigns a higher priority to a search result associated with electronic content corresponding to the rendered document.