Handheld device for capturing text from both a document printed on paper and a document displayed on a dynamic display device

• US 10,769,431 B2
• Filed: 09/17/2018
• Issued: 09/08/2020
• Est. Priority Date: 09/27/2004
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: King, Martin T., Grover, Dale L., Kushler, Clifford A., Stafford-Fraser, James Q.
• Primary Examiner(s): Hannett, James M
• Application Number:
  US16/133,654
• Publication Number:
  US 20190087656A1
• Time in Patent Office: 722 Days
• Field of Search: None
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/00 Commerce
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 20/66 Trinkets, e.g. shirt buttons or jewellery items recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/36 Matching; Classification
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
• CPC Class Codes: G06V 30/418 Document matching, e.g. of document images
• CPC Class Codes: H04N 1/00241 using an image reading device as a local input to a computer input via network, e.g. network scanners H04N1/00225
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/00488 providing an audible output to the user
• CPC Class Codes: H04N 1/0049 providing a visual indication to the user, e.g. using a lamp H04N1/00408 takes precedence
• CPC Class Codes: H04N 1/107 with manual scanning
• CPC Class Codes: H04N 1/32101 Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
• CPC Class Codes: H04N 2201/0041 Point to point H04N2201/0039 takes precedence
• CPC Class Codes: H04N 2201/0055 By radio
• CPC Class Codes: H04N 2201/0058 Docking-station, cradle or the like
• CPC Class Codes: H04N 2201/0081 Image reader H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/0096 : Portable devices
• CPC Class Codes: H04N 2201/3249 : data relating to a linked page or object, e.g. hyperlink
• CPC Class Codes: H04N 2201/3264 : of sound signals
• CPC Class Codes: H04N 2201/3269 : of machine readable codes or marks, e.g. bar codes or glyphs

Abstract

A device for capturing rendered text is described. The device incorporates one or more visual sensors that receive visual information as a part of capturing rendered text. The visual sensors are collectively capable of capturing both text that is permanently printed on a page, and text that is displayed transitorily on a dynamic device. The device further incorporates a visual information disposition subsystem for disposing of visual information received by the visual sensors. The device further incorporates a package that bears the visual sensors and the visual information disposition subsystem, and is suitable to be held in a human hand.

Claims

1. A computer-implemented method to capture data from digital images, comprising:

receiving, by one or more computing devices, an input to capture digital images;

providing, by the one or more computing devices, a user interface that displays instructions to capture digital images;

initiating, by the one or more computing devices, an image capturing module of the one or more computing devices;

receiving, by the one or more computing devices, a digital image from the image capturing module;

determining, by the one or more computing devices, that at least a portion of a set of text in the digital image is readable;

performing, by the one or more computing devices, an optical character recognition algorithm on text in the digital image; and

providing, by the one or more computing devices, a display of at least a portion of the results of the optical character recognition algorithm as an overlay to the digital image on the user interface.

2. The computer implemented method of claim 1, further comprising:

receiving, by the one or more computing devices, an input of an annotation to the digital image; and

communicating, by the one or more computing devices, the annotation with the digital image to a processing system.

3. The computer implemented method of claim 1, further comprising:

determining, by the one or more computing devices, that at least a portion of a set of text in the digital image is not viewable; and

providing, by the one or more computing devices, a notification on the user interface to obtain a second digital image.

4. The computer implemented method of claim 1, further comprising:

providing, by the one or more computing devices, results of the optical character recognition algorithm to a processing system.

5. The computer implemented method of claim 1, wherein the instructions are provided in a pop-up menu.

6. The computer implemented method of claim 1, further comprising communicating, by the one or more computing devices, a request to a processing system to initiate a financial transaction in connection with the digital image.

7. The computer implemented method of claim 1, wherein the one or more computing devices comprise a mobile computing device.

8. The computer implemented method of claim 7, wherein the image capturing module is a function of the mobile computing device.

18. The computer implemented method of claim 1, further comprising:

communicating, by the one or more computing devices, the digital image to a processing system in response to determining that at least a portion of the set of text in the digital image is readable.

9. A computer program product, comprising:

a non-transitory computer-readable storage device having computer-executable program instructions embodied thereon that when executed by one or more computing devices cause the computer to capture data from digital images, the computer-readable program instructions comprising computer-executable instructions to;

receive an input to capture digital images;

provide a user interface that displays instructions to capture digital images;

initiate an image capturing module of the one or more computing devices;

receive a digital image from the image capturing module;

determine that at least a portion of a set of text in the digital image is readable;

perform an optical character recognition algorithm on text in the digital image; and

provide a display of at least a portion of the results of the optical character recognition algorithm as an overlay to the digital image on the user interface.

10. The computer program product of claim 9, further comprising computer-executable instructions to:

receive an input of an annotation to the digital image; and

communicate the annotation with the digital image to a processing system.

11. The computer program product of claim 9, further comprising computer-executable instructions to:

determine that at least a portion of a set of text in the digital image is not viewable; and

provide a notification on the user interface to obtain a second digital image.

12. The computer program product of claim 10, further comprising computer-executable instructions to:

provide the results of the optical character recognition algorithm to a processing system.

19. The computer program product of claim 9, further comprising computer-executable instructions to communicate the digital image to a processing system in response to determining that at least a portion of the set of text in the digital image is readable.

13. A system to capture data from digital images, comprising:

a storage device; and

a processor communicatively coupled to the storage device, wherein the processor executes application code instructions that are stored in the storage device to cause the system to;

receive an input to capture digital images;

provide a user interface that displays instructions to capture digital images;

initiate an image capturing module of the processor;

receive a digital image from the image capturing module;

determine that at least a portion of a set of text in the digital image is readable;

perform an optical character recognition algorithm on text in the digital image; and

provide a display of at least a portion of the results of the optical character recognition algorithm as an overlay to the digital image on the user interface.

14. The system of claim 13, wherein the instructions are provided in a pop-up menu.

15. The system of claim 13, wherein the processor executes application code instructions that are stored in the storage device to cause the system to communicate a request to a processing system to initiate a financial transaction in connection with the digital image.

16. The system of claim 13, wherein the system comprises a mobile computing device.

17. The system of claim 13, wherein the processor executes application code instructions that are stored in the storage device to cause the system to communicate the digital image to a processing system in response to determining that at least a portion of the set of text in the digital image is readable.