Processing techniques for text capture from a rendered document

• US 10,504,162 B2
• Filed: 05/15/2017
• Issued: 12/10/2019
• Est. Priority Date: 05/17/2004
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: King, Martin T., Grover, Dale L., Kushler, Clifford A., Stafford-Fraser, James Q.
• Primary Examiner(s): Garg, Yogesh C
• Application Number:
  US15/594,932
• Publication Number:
  US 20170249507A1
• Time in Patent Office: 939 Days
• Field of Search: 705 261, 707758
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06K 7/1404 Methods for optical code recognition
• CPC Class Codes: G06Q 30/0222 During e-commerce, i.e. online transactions
• CPC Class Codes: G06Q 30/0255 based on user history
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1456 based on user interactions
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 30/2247 Characters composed of bars, e.g. CMC-7
• CPC Class Codes: G06V 30/32 Digital ink
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 30/418 Document matching, e.g. of document images
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules
• CPC Class Codes: H04N 1/00331 with an apparatus performing optical character recognition character recognition G06V30/10

Abstract

A facility for initiating a purchase is described. The facility receives a text sequence captured by a user from a rendered document using a handheld text capture device. The facility identifies in the received text sequence a reference to a distinguished product. In response to identifying the reference, the facility presents to the user an opportunity to place an order for the established product. If the user accepts the presented opportunity to order the distinct product, the facility orders the distinct product on behalf of the user.

Claims

1. A method implemented by data processing apparatus, the method comprising:

receiving, by a server system, from a wireless device including a camera, an image of text captured by the camera from a rendered document;

performing, by the server system, an optical character recognition process on the image of text, and generating, by the optical character recognition process, a corresponding representation of the text;

determining, by the server system, that the representation of the text generated by the optical character recognition process is insufficient for recognition of the rendered document; and

providing, by the server system and for receipt by the wireless device, an indication that a more expansive or different image of text is required for recognition of the rendered document.

2. The method of claim 1, wherein the wireless device includes a display, and wherein the method further comprises:

presenting, on the display, the image of the text to be captured by the camera.

3. The method of claim 2, further comprising:

presenting, on the display, information about the quality of the image and its suitability for recognition.

4. The method of claim 2, further comprising:

in response to the wireless device receiving the indication, presenting, on the display, a visual indication that the more expansive or different image of text is required for recognition of the rendered document.

5. The method of claim 1, further comprising:

receiving, by the server system, from the wireless device including the camera, a subsequent image of text captured by the camera from the rendered document;

performing, by the server system, the optical character recognition process on the subsequent image of text, and generating, by the optical character recognition process, a corresponding subsequent representation of the text;

determining, by the server system, that the subsequent representation of the text generated by the optical character recognition process is sufficient for recognition of the rendered document; and

providing, by the server system and for receipt by the wireless device, a subsequent indication that the subsequent representation of the text is sufficient for recognition of the rendered document.

6. The method of claim 5, wherein the wireless device includes a display, and wherein the method further comprises:

in response to the wireless device receiving the subsequent indication, presenting, on the display, a subsequent visual indication that the subsequent representation of the text is sufficient for recognition of the rendered document.

7. The method of claim 5, further comprising:

based on the subsequent representation of the text, retrieving, by the server system, a digital document that matches the rendered document; and

providing, by the server system and for receipt by the wireless device, the digital document that matches the rendered document.

8. The method of claim 7, wherein the wireless device includes a display, and wherein the method further comprises:

in response to the wireless device receiving the digital document that matches the rendered document, presenting, on the display, a visual notification that the digital document has been received.

9. The method of claim 7, wherein the wireless device includes a display, and wherein the method further comprises:

after the wireless device is provided with the digital document that matches the rendered document, presenting the digital document on the display.

10. A system comprising:

one or more computing devices; and

one or more memory storage apparatus in data communication with the one or more computing devices and storing instructions executable by the one or more computing devices and that upon such execution cause the one or more computing devices to perform operations comprising;

receiving, by a server system, from a wireless device including a camera, an image of text captured by the camera from a rendered document;

performing, by the server system, an optical character recognition process on the image of text, and generating, by the optical character recognition process, a corresponding representation of the text;

determining, by the server system, that the representation of the text generated by the optical character recognition process is insufficient for recognition of the rendered document; and

providing, by the server system and for receipt by the wireless device, an indication that a more expansive or different image of text is required for recognition of the rendered document.

11. The system of claim 10, wherein the operations further comprise:

receiving, by the server system, from the wireless device including the camera, a subsequent image of text captured by the camera from the rendered document;

performing, by the server system, the optical character recognition process on the subsequent image of text, and generating, by the optical character recognition process, a corresponding subsequent representation of the text;

determining, by the server system, that the subsequent representation of the text generated by the optical character recognition process is sufficient for recognition of the rendered document; and

providing, by the server system and for receipt by the wireless device, a subsequent indication that the subsequent representation of the text is sufficient for recognition of the rendered document.

12. The system of claim 11, wherein the wireless device includes a display, and wherein the operations further comprise:

in response to the wireless device receiving the subsequent indication, presenting, on the display, a subsequent visual indication that the subsequent representation of the text is sufficient for recognition of the rendered document.

13. The system of claim 11, wherein the operations further comprise:

based on the subsequent representation of the text, retrieving, by the server system, a digital document that matches the rendered document; and

providing, by the server system and for receipt by the wireless device, the digital document that matches the rendered document.

14. The system of claim 13, wherein the wireless device includes a display, and wherein the operations further comprise:

in response to the wireless device receiving the digital document that matches the rendered document, presenting, on the display, a visual notification that the digital document has been received.

15. The system of claim 13, wherein the wireless device includes a display, and wherein the operations further comprise:

after the wireless device is provided with the digital document that matches the rendered document, presenting the digital document on the display.

16. A non-transitory computer readable medium storing instructions executable by one or more computing devices and that upon such execution cause the one or more computing devices to perform operations comprising:

receiving, by a server system, from a wireless device including a camera, an image of text captured by the camera from a rendered document;

performing, by the server system, an optical character recognition process on the image of text, and generating, by the optical character recognition process, a corresponding representation of the text;

determining, by the server system, that the representation of the text generated by the optical character recognition process is insufficient for recognition of the rendered document; and

providing, by the server system and for receipt by the wireless device, an indication that a more expansive or different image of text is required for recognition of the rendered document.

17. The non-transitory computer readable medium of claim 16, wherein the operations further comprise:

receiving, by the server system, from the wireless device including the camera, a subsequent image of text captured by the camera from the rendered document;

performing, by the server system, the optical character recognition process on the subsequent image of text, and generating, by the optical character recognition process, a corresponding subsequent representation of the text;

determining, by the server system, that the subsequent representation of the text generated by the optical character recognition process is sufficient for recognition of the rendered document; and

providing, by the server system and for receipt by the wireless device, a subsequent indication that the subsequent representation of the text is sufficient for recognition of the rendered document.

18. The non-transitory computer readable medium of claim 17, wherein the wireless device includes a display, and wherein the operations further comprise:

in response to the wireless device receiving the subsequent indication, presenting, on the display, a subsequent visual indication that the subsequent representation of the text is sufficient for recognition of the rendered document.

19. The non-transitory computer readable medium of claim 16, wherein the operations further comprise:

based on the subsequent representation of the text, retrieving, by the server system, a digital document that matches the rendered document; and

providing, by the server system and for receipt by the wireless device, the digital document that matches the rendered document.