Processing Techniques for Text Capture From a Rendered Document

• US 20170249507A1
• Filed: 05/15/2017
• Published: 08/31/2017
• Est. Priority Date: 05/17/2004
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: King, Martin T., Grover, Dale L., Kushler, Clifford A., Stafford-Fraser, James Q.
• Granted Patent: US 10,504,162 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06K 7/1404 Methods for optical code recognition
• CPC Class Codes: G06Q 30/0222 During e-commerce, i.e. online transactions
• CPC Class Codes: G06Q 30/0255 based on user history
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1456 based on user interactions
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 30/2247 Characters composed of bars, e.g. CMC-7
• CPC Class Codes: G06V 30/32 Digital ink
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 30/418 Document matching, e.g. of document images
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules
• CPC Class Codes: H04N 1/00331 with an apparatus performing optical character recognition character recognition G06V30/10

Abstract

A facility for initiating a purchase is described. The facility receives a text sequence captured by a user from a rendered document using a handheld text capture device. The facility identifies in the received text sequence a reference to a distinguished product. In response to identifying the reference, the facility presents to the user an opportunity to place an order for the established product. If the user accepts the presented opportunity to order the distinct product, the facility orders the distinct product on behalf of the user.

Claims

1-38. -38. (canceled)

39. A method implemented by data processing apparatus, the method comprising:

receiving, by a server system, from a wireless device including a camera, an image of text captured by the camera from a rendered document;

performing, by the server system, a recognition process on the image of text, and generating, by the recognition process, a corresponding representation of the text;

determining, by the server system, that the representation of the text does not include sufficient context data for recognition of the rendered document; and

providing, by the server system and for receipt by the wireless device, an indication that additional context data is required for recognition of the rendered document.

40. The method of claim 39, wherein the wireless device includes a display, and wherein the method further comprises:

presenting, on the display, the image of the text to be captured by the camera.

41. The method of claim 40, further comprising:

presenting, on the display, information about the quality of the image and its suitability for recognition.

42. The method of claim 40, further comprising:

in response to the wireless device receiving the indication, presenting, on the display, a visual indication that additional context data is required for recognition of the rendered document.

43. The method of claim 42, wherein the visual indication indicates that a more expansive or different image of text is required for recognition of the rendered document.

44. The method of claim 39, further comprising:

receiving, by the server system, from the wireless device including the camera, a subsequent image of text captured by the camera from the rendered document;

performing, by the server system, the recognition process on the subsequent image of text, and generating, by the recognition process, a corresponding subsequent representation of the text;

determining, by the server system, that the subsequent representation of the text includes sufficient context data for recognition of the rendered document; and

providing, by the server system and for receipt by the wireless device, a subsequent indication that sufficient context data has been determined for recognition of the rendered document.

45. The method of claim 44, wherein the wireless device includes a display, and wherein the method further comprises:

in response to the wireless device receiving the subsequent indication, presenting, on the display, a subsequent visual indication that sufficient context data has been determined for recognition of the rendered document.

46. The method of claim 44, further comprising:

based on the subsequent representation of the text, retrieving, by the server system, a digital document that matches the rendered document; and

providing, by the server system and for receipt by the wireless device, the digital document that matches the rendered document.

47. The method of claim 46, wherein the wireless device includes a display, and wherein the method further comprises:

in response to the wireless device receiving the digital document that matches the rendered document, presenting, on the display, a visual notification that the digital document has been received.

48. The method of claim 46, wherein the wireless device includes a display, and wherein the method further comprises:

after the wireless device is provided with the digital document that matches the rendered document, presenting the digital document on the display.

49. A system comprising:

one or more computing devices; and

one or more memory storage apparatus in data communication with the one or more computing devices and storing instructions executable by the one or more computing devices and that upon such execution cause the one or more computing devices to perform operations comprising;

receiving, by a server system, from a wireless device including a camera, an image of text captured by the camera from a rendered document;

performing, by the server system, a recognition process on the image of text, and generating, by the recognition process, a corresponding representation of the text;

determining, by the server system, that the representation of the text does not include sufficient context data for recognition of the rendered document; and

providing, by the server system and for receipt by the wireless device, an indication that additional context data is required for recognition of the rendered document.

50. The system of claim 49, wherein the operations further comprise:

receiving, by the server system, from the wireless device including the camera, a subsequent image of text captured by the camera from the rendered document;

performing, by the server system, the recognition process on the subsequent image of text, and generating, by the recognition process, a corresponding subsequent representation of the text;

determining, by the server system, that the subsequent representation of the text includes sufficient context data for recognition of the rendered document; and

providing, by the server system and for receipt by the wireless device, a subsequent indication that sufficient context data has been determined for recognition of the rendered document.

51. The system of claim 50, wherein the wireless device includes a display, and wherein the operations further comprise:

in response to the wireless device receiving the subsequent indication, presenting, on the display, a subsequent visual indication that sufficient context data has been determined for recognition of the rendered document.

52. The system of claim 50, wherein the operations further comprise:

based on the subsequent representation of the text, retrieving, by the server system, a digital document that matches the rendered document; and

providing, by the server system and for receipt by the wireless device, the digital document that matches the rendered document.

53. The system of claim 52, wherein the wireless device includes a display, and wherein the operations further comprise:

in response to the wireless device receiving the digital document that matches the rendered document, presenting, on the display, a visual notification that the digital document has been received.

54. The system of claim 52, wherein the wireless device includes a display, and wherein the operations further comprise:

after the wireless device is provided with the digital document that matches the rendered document, presenting the digital document on the display.

55. A non-transitory computer readable medium storing instructions executable by one or more computing devices and that upon such execution cause the one or more computing devices to perform operations comprising:

receiving, by a server system, from a wireless device including a camera, an image of text captured by the camera from a rendered document;

performing, by the server system, a recognition process on the image of text, and generating, by the recognition process, a corresponding representation of the text;

determining, by the server system, that the representation of the text does not include sufficient context data for recognition of the rendered document; and

providing, by the server system and for receipt by the wireless device, an indication that additional context data is required for recognition of the rendered document.

56. The non-transitory computer readable medium of claim 55, wherein the operations further comprise:

receiving, by the server system, from the wireless device including the camera, a subsequent image of text captured by the camera from the rendered document;

performing, by the server system, the recognition process on the subsequent image of text, and generating, by the recognition process, a corresponding subsequent representation of the text;

determining, by the server system, that the subsequent representation of the text includes sufficient context data for recognition of the rendered document; and

providing, by the server system and for receipt by the wireless device, a subsequent indication that sufficient context data has been determined for recognition of the rendered document.

57. The non-transitory computer readable medium of claim 56, wherein the wireless device includes a display, and wherein the operations further comprise:

in response to the wireless device receiving the subsequent indication, presenting, on the display, a subsequent visual indication that sufficient context data has been determined for recognition of the rendered document.

58. The non-transitory computer readable medium of claim 55, wherein the operations further comprise:

based on the subsequent representation of the text, retrieving, by the server system, a digital document that matches the rendered document; and

providing, by the server system and for receipt by the wireless device, the digital document that matches the rendered document.