Handheld device for capturing text from both a document printed on paper and a document displayed on a dynamic display device

• US 9,319,555 B2
• Filed: 04/14/2014
• Issued: 04/19/2016
• Est. Priority Date: 04/19/2004
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: King, Martin T., Grover, Dale L., Kushler, Clifford A., Stafford-Fraser, James Q.
• Primary Examiner(s): HANNETT, JAMES M
• Application Number:
  US14/252,395
• Publication Number:
  US 20140253977A1
• Time in Patent Office: 736 Days
• Field of Search: 348/207.1
• US Class Current: 1/1
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/00 Commerce
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 20/66 Trinkets, e.g. shirt buttons or jewellery items recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/36 Matching; Classification
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
• CPC Class Codes: G06V 30/418 Document matching, e.g. of document images
• CPC Class Codes: H04N 1/00241 using an image reading device as a local input to a computer input via network, e.g. network scanners H04N1/00225
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/00488 providing an audible output to the user
• CPC Class Codes: H04N 1/0049 providing a visual indication to the user, e.g. using a lamp H04N1/00408 takes precedence
• CPC Class Codes: H04N 1/107 with manual scanning
• CPC Class Codes: H04N 1/32101 Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
• CPC Class Codes: H04N 2201/0041 Point to point H04N2201/0039 takes precedence
• CPC Class Codes: H04N 2201/0055 By radio
• CPC Class Codes: H04N 2201/0058 Docking-station, cradle or the like
• CPC Class Codes: H04N 2201/0081 Image reader H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/0096 : Portable devices
• CPC Class Codes: H04N 2201/3249 : data relating to a linked page or object, e.g. hyperlink
• CPC Class Codes: H04N 2201/3264 : of sound signals
• CPC Class Codes: H04N 2201/3269 : of machine readable codes or marks, e.g. bar codes or glyphs

Abstract

A device for capturing rendered text is described. The device incorporates one or more visual sensors that receive visual information as a part of capturing rendered text. The visual sensors are collectively capable of capturing both text that is permanently printed on a page, and text that is displayed transitorily on a dynamic device. The device further incorporates a visual information disposition subsystem for disposing of visual information received by the visual sensors. The device further incorporates a package that bears the visual sensors and the visual information disposition subsystem, and is suitable to be held in a human hand.

Claims

1. A computer-implemented method comprising:

receiving document capture data that represents an output of performing a document capture process on a rendered document;

determining, based at least on the document capture data, that the rendered document includes a particular symbol;

accessing a registry that associates symbols with predetermined actions, wherein each symbol is associated with a different predetermined action;

determining a particular predetermined action associated with the particular symbol; and

providing, to a document management system, (i) at least a portion of the document capture data, and (ii) an indication of the predetermined action associated with the particular symbol.

2. The computer-implemented method of claim 1, further comprising:

identifying, from a corpus of electronic documents, an electronic document that corresponds to the rendered document, based on the portion of the document capture data; and

performing the predetermined action associated with the particular symbol.

3. The computer-implemented method of claim 2, wherein performing the predetermined action comprises causing a control program to be executed.

4. The computer-implemented method of claim 2, wherein performing the predetermined action comprises causing the electronic document to be printed.

5. The computer-implemented method of claim 2, further comprising:

receiving information associated with a user initiating the document capture process on the rendered document; and

identifying the user, based on the received information.

6. The computer-implemented method of claim 5, wherein performing the predetermined action comprises providing the electronic document for display on a display device to the user.

7. The computer-implemented method of claim 5, wherein performing the predetermined action comprises sending the electronic document to an email address associated with the user.

8. The computer-implemented method of claim 5, further comprising:

maintaining a record of performing the predetermined action, wherein the record comprises a user identifier associated with the user, an electronic document identifier associated with the electronic document, and an action identifier associated with the performed action.

9. The computer-implemented method of claim 5, further comprising:

determining that the user is authorized to access a document archive that is accessible to a plurality of users of a user group; and

storing information associated with the rendered document in the document archive.

10. A computer-readable storage device storing software including instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving document capture data that represents an output of performing a document capture process on a rendered document;

determining, based at least on the document capture data, that the rendered document includes a particular symbol;

accessing a registry that associates symbols with predetermined actions, wherein each symbol is associated with a different predetermined action;

determining a particular predetermined action associated with the particular symbol; and

providing, to a document management system, (i) at least a portion of the document capture data, and (ii) an indication of the predetermined action associated with the particular symbol.

11. The computer-readable storage device of claim 10, wherein the operations further comprise:

identifying, from a corpus of electronic documents, an electronic document that corresponds to the rendered document, based on the portion of the document capture data; and

performing the predetermined action associated with the particular symbol.

12. The computer-readable storage device of claim 11, wherein the operations further comprise:

receiving information associated with a user initiating the document capture process on the rendered document; and

identifying the user, based on the received information.

13. The computer-readable storage device of claim 12, wherein the operations further comprise:

maintaining a record of performing the predetermined action, wherein the record comprises a user identifier associated with the user, an electronic document identifier associated with the electronic document, and an action identifier associated with the performed action.

14. The computer-readable storage device of claim 12, wherein the operations further comprise:

determining that the user is authorized to access a document archive that is accessible to a plurality of users of a user group; and

storing information associated with the rendered document in the document archive.

15. A system, comprising:

one or more data processing apparatus; and

a computer-readable storage device including instructions executable by the data processing apparatus and upon such execution cause the data processing apparatus to perform operations comprising;

receiving document capture data that represents an output of performing a document capture process on a rendered document;

determining, based at least on the document capture data, that the rendered document includes a particular symbol;

accessing a registry that associates symbols with predetermined actions, wherein each symbol is associated with a different predetermined action;

determining a particular predetermined action associated with the particular symbol; and

providing, to a document management system, (i) at least a portion of the document capture data, and (ii) an indication of the predetermined action associated with the particular symbol.

16. The system of claim 15, wherein the operations further comprise:

identifying, from a corpus of electronic documents, an electronic document that corresponds to the rendered document, based on the portion of the document capture data; and

performing the predetermined action associated with the particular symbol.

17. The system of claim 16, wherein the operations further comprise:

receiving information associated with a user initiating the document capture process on the rendered document; and

identifying the user, based on the received information.

18. The system of claim 17, wherein the operations further comprise:

maintaining a record of performing the predetermined action, wherein the record comprises a user identifier associated with the user, an electronic document identifier associated with the electronic document, and an action identifier associated with the performed action.

19. The system of claim 17, wherein the operations further comprise:

determining that the user is authorized to access a document archive that is accessible to a plurality of users of a user group; and

storing information associated with the rendered document in the document archive.

20. The system of claim 17, wherein performing the predetermined action comprises one or more of causing the electronic document to be printed, providing the electronic document for display on a display device to the user, or sending the electronic document to an email address associated with the user.