Data capture from rendered documents using handheld device

• US 9,143,638 B2
• Filed: 04/29/2013
• Issued: 09/22/2015
• Est. Priority Date: 04/01/2004
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: King, Martin Towle, Grover, Dale L., Kushler, Clifford A., Stafford-Fraser, James Quentin
• Primary Examiner(s): MARIAM, DANIEL G
• Application Number:
  US13/872,850
• Publication Number:
  US 20140120981A1
• Time in Patent Office: 876 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: H04M 1/21 Combinations with auxiliary equipment, e.g. with clocks or memoranda pads
• CPC Class Codes: H04N 1/00334 with an apparatus processing barcodes or the like arrangements for sensing record carriers G06K7/00

Abstract

A portable device having scanning, imaging or other data-capture capability is described. In some cases, the portable device can indicate to the user when enough information has been captured to uniquely identify a source document. In some cases, the portable device calculates timestamps and location-stamps indicating when and where a data capture occurred. In some cases, the portable device is controlled by gestures. In some cases, the portable scanning device has associated billing and content/service subscription information.

Claims

1. A method comprising:

capturing, using a mobile phone, information from a rendered document;

obtaining, using the mobile phone, a location-stamp indicating a location of the mobile phone at which the information captured from the rendered document was captured;

storing, in a memory of the mobile phone, the information captured from the rendered document and the location-stamp;

transmitting, from the mobile phone to a search engine, a search query to identify a source document of the rendered document in a set of source documents, wherein the search query includes the information captured from the rendered document and the location-stamp; and

receiving, by the mobile device and from the search engine, an electronic version of the source document based on the search engine identifying the source document from a subset of source documents of the set of source documents by the information captured from the rendered document and the location-stamp, wherein the subset of source documents is determined by the location-stamp.

2. The method of claim 1, further comprising:

the mobile phone constructing the search query,

wherein the search query further includes an identifier of the mobile phone.

3. The method of claim 2, further comprising:

the mobile phone storing, within the memory, the search query at a time when no communication between the mobile phone and a wireless network is available,

wherein transmitting the search query occurs after communication between the mobile phone and the wireless network resumes.

4. The method of claim 1, further comprising:

obtaining, using the mobile phone, a time-stamp indicating a time and a date that the information captured from the rendered document was captured,

wherein the search query further includes the time-stamp.

5. The method of claim 4,

wherein obtaining the location-stamp comprises using a location module of the mobile phone to obtain the location-stamp, and

wherein the location module uses a location technology selected from the group

consisting of (i) global positioning system (GPS), (ii) enhanced observed time difference, (iii) assisted GPS, (iv) differential GPS, (v) time difference of arrival, (vi) angle of arrival, and (vii) triangulation and monitoring of local transceiver pilot signals.

6. The method of claim 1,

wherein the search query further includes a factor selected from the group consisting of (i) an identity of a user using the mobile phone, (ii) information regarding a reading habit of the user, (iii) information regarding a capture history of the user, and (iv) information regarding a previous capture of information using the mobile phone.

7. The method of claim 1, wherein capturing the information from the rendered document comprises capturing text typed via a keypad or keyboard of the mobile phone.

8. The method of claim 1,

wherein capturing the information from the rendered document comprises capturing, using a microphone of the mobile phone, spoken words being read from text of the rendered document,

the method further comprising:

converting, using a speech-to-text application, the captured spoken words to a text form of the spoken words;

wherein the search query includes the text form of the spoken words.

9. The method of claim 1,

wherein capturing the information from the rendered document comprises capturing the information optically using a lens and an image sensor of the mobile phone.

10. The method of claim 9, wherein capturing the information optically comprises capturing hidden control symbols printed with ink having UV or infrared properties.

11. The method of claim 1,

wherein capturing the information from the rendered document comprising capturing the information from a magnetic strip.

12. The method of claim 1,

wherein the captured information comprises a trademark symbol,

the method further comprising:

recognizing, using the mobile phone, the trademark symbol within the captured information; and

substituting, using the mobile phone, a code or text for the trademark symbol,

wherein transmitting the search query comprises transmitting the code or text for trademark symbol in place of the trademark symbol.

13. The method of claim 1, further comprising:

receiving, using the mobile phone, text entered via a keypad of the mobile phone,

modifying, using the mobile phone, the search query to include the text entered via the keypad of the mobile phone.

14. The method of claim 1, wherein capturing the information comprises capturing multiple lines of text from the rendered document.

15. The method of claim 1, wherein the search query includes an identifier of the rendered document to restrict search results returned to the mobile phone.

16. A system comprising:

a user interface to capture information from a rendered document;

a location module to obtain a location-stamp indicating a location of the mobile phone at which the information captured from the rendered document was captured;

a memory to store the information captured from the rendered document and the location-stamp; and

a communications interface including a transceiver that transmits, to a wireless network, a search query to identify a source document of the rendered document in a set of source documents, wherein the search query includes the information captured from the rendered document and the location-stamp, wherein the communications interface is configured to receive an electronic version of the source document, from a search engine, based on the search engine identifying the source document from a subset of source documents of the set of source documents by the information captured from the rendered document and the location-stamp, wherein the subset of source documents is determined by the location-stamp.

17. The system of claim 16, wherein the user interface comprises a keypad or keyboard to type text shown on the rendered document.

18. The system of claim 16,

wherein the user interface comprises a microphone configured to capture spoken words being read from text of the rendered document,

the mobile phone further comprising:

a speech-to-text application configured to convert the captured spoken words to a text form of the spoken words,

wherein the search query includes the text form of the spoken words.

19. The system of claim 16,

wherein the user interface comprises an optical sensing system including a lens and an image sensor configured to capture the information from the rendered document optically.

20. A non-transitory computer-readable medium comprising computer instructions executable by a processor of a mobile phone to cause performance of the following operations:

capturing, using a mobile phone, information from a rendered document;

obtaining, using the mobile phone, a location-stamp indicating a location of the mobile phone at which the information captured from the rendered document was captured;

storing, in a memory of the mobile phone, the information captured from the rendered document and the location-stamp;

transmitting, from the mobile phone to a search engine, a search query to identify a source document of the rendered document in a set of source documents, wherein the search query includes the information captured from the rendered document and the location-stamp; and

receiving, by the mobile device and from the search engine, an electronic version of the source document based on the search engine identifying the source document from a subset of source documents of the set of source documents by the information captured from the rendered document and the location-stamp, wherein the subset of source documents is determined by the location-stamp.

21. A method, comprising:

receiving, by a search engine and from a mobile device, a search query including information from a rendered document and a location-stamp indicating a location of the mobile phone at which the information captured from the rendered document was captured;

identifying, by the search engine, a subset of source documents from a set of source documents based on the location-stamp of the rendered document, wherein the rendered document is included in at least one of the source documents of the subset of source documents;

determining, by the search engine, the source document from the subset of source documents based on the information captured from the rendered document and the location-stamp; and

sending, by the search engine to the mobile device, an electronic version of the source document.