Selective display of OCR'ed text and corresponding images from publications on a client device

• US 9,280,952 B2
• Filed: 01/10/2014
• Issued: 03/08/2016
• Est. Priority Date: 01/28/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ratnakar, Viresh, Haugen, Frances Bordwell, Popat, Ashok
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US14/152,893
• Publication Number:
  US 20140125693A1
• Time in Patent Office: 788 Days
• Field of Search: 345/619
• US Class Current: 1/1
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/127 with the intervention of an operator
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

Text is extracted from a source image of a publication using an Optical Character Recognition (OCR) process. A document is generated containing text segments of the extracted text. The document includes a control module that responds to user interactions with the displayed document. Responsive to a user selection of a displayed text segment, a corresponding image segment from the source image containing the text is retrieved and rendered in place of the selected text segment. The user can select again to toggle the display back to the text segment. Each text segment can be tagged with a garbage score indicating its quality. If the garbage score of a text segment exceeds a threshold value, the corresponding image segment can be automatically displayed instead.

Claims

1. A computer-implemented method for displaying a document, the method comprising:

identifying a document including at least one text segment generated responsive to an Optical Character Recognition (OCR) process performed on an image segment, wherein the text segment includes a plurality of characters;

generating a quality score for each of the plurality of characters;

generating a segment quality measure for the text segment by averaging the generated quality scores, the segment quality measure indicating a quality of the text segment; and

responsive to the segment quality measure not meeting a quality threshold, displaying the image segment instead of the text segment on a display of a client device.

2. The method of claim 1, further comprising:

responsive to the segment quality measure meeting the quality threshold, displaying the text segment on the display of the client device.

3. The method of claim 2, further comprising:

responsive to a selection of the text segment by a user, replacing the text segment with the image segment on the display of the client device.

4. The method of claim 1, further comprising:

responsive to a selection of the image segment by a user, replacing the image segment with the text segment on the display of the client device.

5. The method of claim 1, wherein the document includes positional information relating the text segment to the image segment, wherein displaying the image segment further comprises:

identifying the positional information in the document for the text segment; and

transmitting a request for the image segment, the request including the identified positional information.

6. The method of claim 5, wherein the request for the image segment is transmitted to a remote server and the image segment is obtained from the remote server.

7. The method of claim 5, wherein the positional information describes a region of a source image including text contained in the text segment.

8. A non-transitory computer-readable storage medium encoded with executable computer program code for:

identifying a document including at least one text segment generated responsive to an Optical Character Recognition (OCR) process performed on an image segment, wherein the text segment includes a plurality of characters;

generating a quality score for each of the plurality of characters;

generating a segment measure for text segment by averaging the generated quality scores, the segment quality measure indicating a quality of the text segment; and

responsive to the segment quality measure not meeting a quality threshold, displaying the image segment instead of the text segment on a display of a client device.

9. The computer-readable storage medium of claim 8, wherein the computer program code is further for:

responsive to the segment quality measure meeting the quality threshold, displaying the text segment on the display of the client device.

10. The computer-readable storage medium of claim 9, wherein the computer program code is further for:

responsive to a selection of the text segment by a user, replacing the text segment with the image segment on the display of the client device.

11. The computer-readable storage medium of claim 8, wherein the computer program code is further for:

responsive to a selection of the image segment by a user, replacing the image segment with the text segment on the display of the client device.

12. The computer-readable storage medium of claim 8, wherein the document includes positional information relating the text segment to the image segment, and wherein displaying the image segment comprises:

identifying the positional information in the document for the text segment; and

transmitting a request for the image segment, the request including the identified positional information.

13. The computer-readable storage medium of claim 12, wherein the request for the image segment is transmitted to a remote server and the image segment is obtained from the remote server.

14. The computer-readable storage medium of claim 12, wherein the positional information describes a region of a source image including text contained in the text segment.

15. A system for displaying a publication, the system comprising:

a computer processor; and

a non-transitory computer-readable storage medium encoded with computer program code adapted to execute on the computer processor for;

identifying a document including at least one text segment generated responsive to an Optical Character Recognition (OCR) process performed on an image segment, wherein the text segment includes a plurality of characters;

generating a quality score for each of the plurality of characters;

generating a segment quality measure for the text segment by averaging the generated quality scores, the segment quality measure indicating a quality of the text segment; and

responsive to the segment quality measure not meeting a quality threshold, displaying the image segment instead of the text segment on a display of a client device.

16. The system of claim 15, wherein the computer program code is further adapted to execute on the computer processor for

responsive to the segment quality measure meeting the quality threshold, displaying the text segment on the display of the client device.

17. The system of claim 16, wherein the computer program code is further adapted to execute on the computer processor for:

responsive to a selection of the text segment by a user, replacing the text segment with the image segment on the display of the client device.

18. The system of claim 15, wherein the computer program code is further adapted to execute on the computer processor for

responsive to a selection of the image segment by a user, replacing the image segment with the text segment on the display of the client device.

19. The system of claim 15, wherein the document includes positional information relating the text segment to the image segment, and wherein displaying the image segment comprises:

identifying the positional information in the document for the text segment; and

transmitting a request for the image segment, the request including the identified positional information.

20. The system of claim 19, wherein the positional information describes a region of a source image including text contained in the text segment.