Verification of optical character recognition results

• US 10,068,155 B2
• Filed: 09/26/2016
• Issued: 09/04/2018
• Est. Priority Date: 09/16/2016
• Status: Expired due to Fees

• Current Assignee: ABBYY Development LLC
• Original Assignee: ABBYY Development LLC
• Inventors: Kalyuzhny, Aleksey Ivanovich
• Primary Examiner(s): YEH, EUENG NAN
• Application Number:
  US15/275,990
• Publication Number:
  US 20180082154A1
• Time in Patent Office: 708 Days
• CPC Class Codes: G06F 18/2178 based on feedback of a supervisor
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/12 Detection or correction of errors, e.g. by rescanning the pattern
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors

Abstract

A method of verifying optical character recognition (OCR) results may involve: performing OCR on one or more initial images of a document and displaying initial OCR results of the document to a user; receiving a feedback from the user regarding an error location in the initial OCR results, the error location being a location of a misspelled character sequence; receiving an additional image of the document, which corresponds to the error location, and performing OCR of the additional image to produce additional OCR results; identifying a cluster of character sequences, which correspond to the error location, using the initial OCR results and the additional OCR results; identifying an order of character sequences in the cluster of character sequences based on their respective probability values; and displaying to the user modified optical character recognition results, which contain in the error location a corrected character sequence.

Claims

1. A method comprising:

performing optical character recognition on one or more initial images of a document to produce initial optical character recognition results and displaying the initial optical character recognition results of the document to a user;

receiving a feedback from the user regarding an error location in the initial optical character recognition results, wherein the error location is a location of a misspelled character sequence in the initial optical character recognition results;

receiving an additional image of the document, wherein the additional image contains a portion of the document, which corresponds to the error location;

performing optical character recognition of the additional image to produce additional optical character recognition results;

identifying a cluster of character sequences corresponding to the error location by matching the initial optical character recognition results and the additional optical character recognition results;

performing, for each of the cluster of characters, a probability evaluation to determine a plurality of probability values for the cluster of characters;

identifying, based on the probability values, a character sequence of the cluster of character sequences as a corrected character sequence; and

displaying to the user modified optical character recognition results, which contain in the error location the corrected character sequence, wherein the corrected character sequence is different from the misspelled character sequence.

2. The method of claim 1, wherein the additional image differs from the one or more initial images in at least one of:

an image noise, an image scale, a shooting angle and an image brightness.

3. The method of claim 1, wherein said identifying the cluster of character sequences, which correspond to the error location, comprises:

identifying in the initial optical character recognition results and the additional optical character recognition results a plurality of common features to identify reference points.

4. The method of claim 3, further comprising identifying using coordinates of the reference points parameters of a coordinate transformation converting coordinates of the additional optical character recognition results into coordinates of the initial optical character recognition results.

5. The method of claim 1, wherein said identifying, based on the probability values, the character sequence of the cluster of character sequences as the corrected character sequence comprises determining a pre-determined metric for each character sequence of the cluster and resorting the character sequences of the cluster according to the probability values.

6. The method of claim 5, wherein the pre-determined metric is a sum of edit distances between a character sequence of the cluster and each of the other character sequences of the cluster and wherein the character sequences are resorted so that a character sequence with a lowest value of the metric goes is the top of the cluster, while a character sequence with a highest value of the metric is on the bottom of the cluster.

7. The method of claim 1, wherein the one or more initial images and the additional image are obtained in a continuous sequence of images.

8. The method of claim 1, wherein displaying to user the modified optical character recognition results comprises highlighting in the displayed modified optical character recognition results the corrected character sequence.

9. The method of claim 1, further comprising, after said receiving the feedback from the user, highlighting the error location in the displayed initial optical character recognition results.

10. The method of claim 1, which performed by a mobile device.

11. The method of claim 10, wherein the mobile device is one of a mobile phone, a tablet, a laptop, a smartphone or a PDA.

12. The method of claim 10, wherein said receiving the feedback from the user comprises receiving a feedback from a location of a touch screen display of the mobile device, which corresponds to the error location.

13. The method of claim 10, wherein the mobile device comprises a camera and the one or more initial images and the additional image are acquired by said camera.

14. A system comprising:

a memory;

a display; and

a processing device, which is coupled to the memory and the display, the processing device is configured to;

perform optical character recognition on one or more initial images of a document to produce initial optical character recognition results and display the initial optical character recognition results to a user;

receive a feedback from the user regarding an error location in the initial optical character recognition results, wherein the error location is a location of a misspelled character sequence in the initial optical character recognition results;

receive an additional image of the document, wherein the additional image contains a portion of the document, which corresponds the error location;

perform optical character recognition on the additional image to produce additional optical character recognition results;

identify a cluster of character sequences corresponding to the error location by matching the initial optical character recognition results and the additional optical character recognition results;

perform, for each of the cluster of characters, a probability evaluation to determine a plurality of probability values for the cluster of characters;

identify, based on the probability values, a character sequence of the cluster of character sequences as a corrected character sequence; and

display to the user modified optical character recognition results, which contain in the error location the corrected character sequence, wherein the corrected character sequence is different from the misspelled character sequence.

15. The system of claim 14, which is a mobile device.

16. The system of claim 15, wherein the mobile device is a one of a mobile phone, a tablet, a laptop, a smartphone or a PDA.

17. The system of claim 14, wherein the display is a touch-screen display and wherein the processing device receives the feedback from the user regarding the error location by receiving a feedback from a location of the touch-screen display touched by the user.

18. The system of claim 14, further comprising a camera configured to acquire the one or more images and the next image and to transfer each of the one or more images and the next image to the processing device.

19. A computer-readable non-transitory storage medium comprising executable instructions that, when executed by a processing device, cause the processing device to:

perform optical character recognition on one or more initial images of a document to produce initial optical character recognition results and display the initial optical character recognition results of the document to a user;

receive a feedback from the user regarding an error location in the initial optical character recognition results, wherein the error location is a location of a misspelled character sequence in the initial optical character recognition results;

receive an additional image of the document, wherein the additional image contains a portion of the document, which corresponds to the error location;

perform optical character recognition on the additional image to produce additional optical character recognition results;

identify a cluster of character sequences corresponding to the error location by matching the initial optical character recognition results and the additional optical character recognition results;

perform, for each of the cluster of characters, a probability evaluation to determine a plurality of probability values for the cluster of characters;

identify, based on the probability values, a character sequence of the cluster of character sequences as a corrected character sequence; and

display to the user modified optical character recognition results, which contain in the error location the corrected character sequence, wherein the corrected character sequence is different from the misspelled character sequence.