Selective display of OCR'ed text and corresponding images from publications on a client device

• US 8,675,012 B2
• Filed: 06/06/2013
• Issued: 03/18/2014
• Est. Priority Date: 01/28/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ratnakar, Viresh, Haugen, Frances Bordwell, Popat, Ashok
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US13/911,762
• Publication Number:
  US 20130265325A1
• Time in Patent Office: 285 Days
• Field of Search: 345/619, 382/321
• US Class Current: 345/619
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/127 with the intervention of an operator
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

Text is extracted from a source image of a publication using an Optical Character Recognition (OCR) process. A document is generated containing text segments of the extracted text. The document includes a control module that responds to user interactions with the displayed document. Responsive to a user selection of a displayed text segment, a corresponding image segment from the source image containing the text is retrieved and rendered in place of the selected text segment. The user can select again to toggle the display back to the text segment. Each text segment can be tagged with a garbage score indicating its quality. If the garbage score of a text segment exceeds a threshold value, the corresponding image segment can be automatically displayed instead.

Claims

1. A computer-implemented method comprising:

responsive to a quality measure for a text segment generated via an Optical Character Recognition (OCR) process meeting a quality threshold, displaying the text segment on a display of a client device; and

responsive to the quality measure of the text segment not meeting the quality threshold, displaying an image segment from which the text segment was generated via the OCR process on the display of the client device.

2. The computer-implemented method of claim 1, further comprising:

responsive to a selection of the image segment by a user, replacing the selected image segment with the text segment on the display of the client device.

3. The computer-implemented method of claim 1, wherein the text segment is replaced with the image segment from which the text segment was generated via the OCR process on the display of the client device responsive to a selection of the text segment by a user.

4. The computer-implemented method of claim 1:

wherein the text segment is associated with positional information, and

wherein displaying the image segment comprises transmitting a request for the image segment to a document serving system, the request including the identified positional information.

5. The computer-implemented method of claim 1, wherein the quality measure is based on a language-conditional character probability associated with one or more characters of the text segment.

6. The computer-implemented method of claim 5:

wherein the language-conditional character probability indicates how well the one or more characters and a set of characters preceding the one or more characters in the text segment concord with a language model.

7. The computer-implemented method of claim 1, wherein displaying the text segment on the display of the client device comprises executing a control module received from a document serving system, wherein the control module is implemented using browser-executable code written in one or more programming languages.

8. A non-transitory computer-readable storage medium encoded with executable computer program code comprising program code for:

responsive to a quality measure of a text segment generated via an Optical Character Recognition (OCR) process meeting a quality threshold, displaying the text segment on a display of a client device; and

responsive to the quality measure of the text segment not meeting the quality threshold, displaying an image segment from which the text segment was generated via the OCR process on the display of the client device.

9. The non-transitory computer-readable storage medium of claim 8, wherein the computer program code further comprises program code for:

responsive to a selection of the image segment by a user, replacing the selected image segment with the text segment on the display of the client device.

10. The non-transitory computer-readable storage medium of claim 8, wherein the text segment is replaced with the image segment from which the text segment was generated via the OCR process on the display of the client device responsive to a selection of the text segment by a user.

11. The non-transitory computer-readable storage medium of claim 8:

wherein the text segment is associated with positional information, and

wherein the program code for displaying the image segment comprises program code for transmitting a request for the image segment to a document serving system, the request including the identified positional information.

12. The non-transitory computer-readable storage medium of claim 8, wherein the quality measure is based on a language-conditional character probability associated with one or more characters of the text segment.

13. The non-transitory computer-readable storage medium of claim 12:

wherein the language-conditional character probability indicates how well the one or more characters and a set of characters preceding the one or more characters in the text segment concord with a language model.

14. The non-transitory computer-readable storage medium of claim 8, wherein the program code for displaying the text segment on the display of the client device comprises program code for executing a control module received from a document serving system, wherein the control module is implemented using browser-executable code written in one or more programming languages.

15. A client device comprising:

a non-transitory computer-readable storage medium storing executable computer program instructions comprising instructions for;

responsive to a quality measure of a text segment generated via an Optical Character Recognition (OCR) process meeting a quality threshold, displaying the text segment on a display of a client device; and

responsive to the quality measure of the text segment not meeting the quality threshold, displaying an image segment from which the text segment was generated via the OCR process on the display of the client device; and

a processor for executing the program instructions.

16. The client device of claim 15, wherein the computer program instructions further comprise computer program instructions for:

responsive to a selection of the image segment by a user, replacing the selected image segment with the text segment on the display of the client device.

17. The client device of claim 15, wherein the text segment is replaced with the image segment from which the text segment was generated via the OCR process on the display of the client device responsive to a selection of the text segment by a user.

18. The client device of claim 15, wherein the quality measure is based on a language-conditional character probability associated with one or more characters of the text segment.

19. The client device of claim 18:

wherein the language-conditional character probability indicates how well the one or more characters and a set of characters preceding the one or more characters in the text segment concord with a language model.

20. The client device of claim 15, wherein the computer program instructions for displaying the text segment on the display of the client device comprise computer program instructions for executing a control module received from a document serving system, wherein the control module is implemented using browser-executable code written in one or more programming languages.