Document comparison and analysis for improved OCR

• US 8,472,727 B2
• Filed: 01/07/2011
• Issued: 06/25/2013
• Est. Priority Date: 01/07/2011
• Status: Expired due to Fees

• Current Assignee: Ecortext, Ltd.
• Original Assignee: Ecortext, Ltd.
• Inventors: Gronau, Yuval, Likhovski, Edo
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US12/929,200
• Publication Number:
  US 20120177295A1
• Time in Patent Office: 900 Days
• Field of Search: 382/218
• US Class Current: 382/218
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/12 Detection or correction of errors, e.g. by rescanning the pattern
• CPC Class Codes: G06V 30/248 involving plural approaches, e.g. verification by template match; Resolving confusion among similar patterns, e.g. "O" versus "Q" G06V30/242 takes precedence

Abstract

A method for enhancing the accuracy of Optical Character Recognition (OCR) algorithms by detection of differences between a digital image of a document and a text file corresponding to the digital image, created by the OCR algorithm.

The method includes calculating the transformation between the first and second digital images such as geometrical distortion, local brightness and contrast differences and blurring due to the optical imaging process. The method estimates the parameters of these transformations so that the transformations can be applied to at least one of the images, rendering it as similar as possible to the other image. The method further compares the two images in order to find differences. The method further displays the differences on a display device and analyzes the differences. The analysis results are fed back to the OCR algorithm.

Claims

1. A computerized method comprising:

a. obtaining a first digital image of a first text;

b. applying a character recognition algorithm to said first digital image to obtain a text representation of said first digital image;

c. transforming said text representation into a second digital image;

d. comparing said first and second digital images to obtain a difference image; and

e. analyzing said difference image.

2. The method of claim 1, additionally comprising displaying said analysis results.

3. The method of claim 1, additionally comprising:

a. feeding said analysis results to said character recognition algorithm;

b. using said analysis results to apply said character recognition algorithm to said first digital image to obtain an improved text representation of said first digital image;

c. transforming said improved text representation into a second digital image;

d. comparing said first and second digital images to obtain an improved difference image;

e. analyzing said improved difference image; and

f. repeating steps (a) through (e) until a predetermined criterion is satisfied.

4. The method of claim 3, wherein said predefined criterion comprises a number of iterations.

5. The method of claim 3, wherein said predefined criterion comprises a difference level between said first and second digital images.

6. The method of claim 1, wherein said analyzing comprises detecting dots or lines in said difference image, signifying one of:

at least one character is missing in said first digital image, at least one character was mistakenly added to said first digital image and a character in said first digital image was erroneous detected by said character recognition algorithm.

7. The method of claim 1, wherein said analyzing comprises detecting dots or lines in said difference image, signifying that a character said first digital image is correct but is of the wrong font type, font size or color.

8. The method of claim 1, wherein said analyzing comprises detecting dots or lines in said difference image, signifying that at least one of borders of a table and underlining of words are missing from said first digital image or mistakenly added to it.

9. The method of claim 1, wherein said analyzing comprises detecting misalignments between said first and second digital images.

10. The method of claim 1, wherein said analyzing comprises applying one of a minimum difference criterion and a maximum likelihood criterion for replacing a mismatched character or word.

11. The method of claim 10, wherein the maximum likelihood criterion comprises probabilities.

12. The method of claim 11, wherein said probabilities comprise a-priori probabilities of character combinations.

13. The method of claim 11, wherein said probabilities comprise higher probabilities for words appearing in a dictionary or lexicon.

14. The method of claim 11, wherein said probabilities comprise the probability of a character or word to be the correct character or word, based on a comparison between said first and second digital images.

15. The method of claim 1, wherein said comparing said first and second digital images to obtain a difference image comprises:

a. performing global geometrical transformation on at least one of said first and second digital images to obtain first and second aligned digital images;

b. selecting a window of said first aligned digital image;

c. selecting a window of said second aligned digital image;

d. estimating residual local geometrical shifts, radiometric differences and blurring differences between said selected windows to obtain a second transformation;

e. applying said second transformation to at least one of said windows;

f. repeating steps (d) through (g) for consecutive windows of said first and second aligned images to obtain first and second similar images; and

g. comparing said first and second similar images to obtain a difference image.

16. The method of claim 1, wherein said obtaining a first digital image comprises scanning a paper document.

17. The method of claim 1, wherein said obtaining a first digital image comprises capturing the image with a digital camera.

18. The method of claim 17, wherein said digital camera is a handheld camera.

19. The method of claim 18, wherein said handheld digital camera is part of a handheld processing device.

20. The method of claim 19, wherein said handheld processing device is one of a smartphone and a tablet PC.

21. The method of claim 1, wherein said obtaining a first digital image comprises retrieving the image from storage.

22. The method of claim 15, wherein said global geometrical transformation comprises at least one of translation in the vertical and horizontal axes, magnification, rotation, skewing and optical distortion correction.

23. The method of claim 15, wherein the global geometrical transformation comprises:

a. detecting features in said first and second digital images; and

b. matching between said detected features.

24. The method of claim 15, wherein said estimating residual geometrical shifts, radiometric differences and blurring differences comprises applying a linear convolution and adding a constant to the window of said second aligned digital image.

25. The method of claim 24, wherein the convolution coefficients and constant level are calculated as a linear combination of several windows, said windows comprising translations of said selected window of said second aligned digital image and a constant level.

26. The method of claim 15, wherein said estimating residual geometrical shifts, radiometric differences and blurring differences comprises applying a local Fourier analysis to said first selected window of said first aligned digital image and comparing it to the Fourier analysis of said selected window of said second aligned digital image.

27. The method of claim 26, wherein said residual geometrical shifts between features in said selected window of said first aligned image and corresponding features in said selected window of said second aligned image are derived by finding the linear shift of the phase in the frequency analysis.

28. The method of claim 26, wherein said blurring and radiometric differences between said selected window of said first aligned image and said selected window of said second aligned image are estimated by calculating the ratio of the amplitudes of the coefficients in the frequency analysis and fitting an appropriate model.

29. The method of claim 15, wherein said residual geometrical shifts between features in said selected window of said first aligned image and corresponding features in said selected window of said second aligned image are derived by:

a. calculating the integer pixel shift between said selected window of said first aligned image and said selected window of said second aligned image; and

b. calculating the sub-pixel shift between said selected window of said first aligned image and said selected window of said second aligned image.

30. The method of claim 29, wherein said calculating the integer pixel shift comprises calculating one of the maximum correlation and the minimum difference between said windows in the vertical and horizontal directions and finding the shift at which the highest correlation or minimum difference is found.

31. The method of claim 30, wherein said calculating the sub-pixel shift comprises fitting a second order curve to the values of the correlation function at the best fit position and at positions that are shifted by one pixel in both directions.

32. The method of claim 29, wherein said calculating the sub-pixel shift comprises calculating the mean square error of the two image windows, fitting a second order curve to the values of the mean square error function at the best fit position and at positions that are shifted by one pixel in both directions and finding the minimum value position of said second order curve.

33. The method of claim 29, wherein said calculating the sub-pixel shift comprises calculating the mean absolute difference between the two image windows, fitting a V shape curve to the values of the mean absolute difference function at the best fit position and at positions that are shifted by one pixel in both directions and finding the minimum value position of said V shape curve.

34. The method of claim 15, wherein calculating at least one said global and local geometrical transformations comprises calculating optical flow.

35. The method of claim 15, wherein calculating at least one said global and local geometrical transformations comprises interpolating one or both of the images to increase their resolution and performing an integer pixel translation on the images of increased resolution.

36. A system comprising:

a processor;

communication means connected with the processor;

display means;

means for obtaining a first digital image of a first text; and

memory means coupled to the processor, said memory means configured to store a plurality of modules for execution by the processor, the plurality of modules comprising;

logic for performing a character recognition algorithm on said first digital image to obtain a text representation of said first digital image;

logic for transforming said text representation into a second digital image;

logic for comparing said first and second digital images to obtain a difference image; and

logic for analyzing said difference image.

37. The system of claim 36, additionally comprising logic for displaying said analysis results.

38. The system of claim 36, additionally comprising:

logic for feeding said analysis results to said character recognition algorithm;

logic for using said analysis results to apply said character recognition algorithm to said first digital image to obtain an improved text representation of said first digital image;

logic for transforming said improved text representation into a second digital image;

logic for comparing said first and second digital images to obtain an improved difference image;

logic for analyzing said improved difference image; and

logic for repeating said using, transforming, comparing and analyzing until a predetermined criterion is satisfied.

39. The system of claim 38, wherein said predefined criterion comprises a number of iterations.

40. The system of claim 38, wherein said predefined criterion comprises a difference level between said first and second digital images.

41. The system of claim 36, wherein said logic for analyzing comprises logic for detecting dots or lines in said difference image, signifying one of:

at least one character is missing in said first digital image, at least one character was mistakenly added to said first digital image and a character in said first digital image was erroneous detected by said character recognition algorithm.

42. The system of claim 36, wherein said logic for analyzing comprises logic for detecting dots or lines in said difference image, signifying that a character in said first digital image is correct but is of the wrong font type, font size or color.

43. The system of claim 36, wherein said logic for analyzing comprises logic for detecting dots or lines in said difference image, signifying that at least one of borders of a table and underlining of a word are missing from said first digital image or mistakenly added to it.

44. The system of claim 36, wherein said logic for analyzing comprises logic for detecting misalignments between said first and second digital images.

45. The system of claim 36, wherein said logic for analyzing comprises logic for applying one of a minimum difference criterion and a maximum likelihood criterion for replacing a mismatched character or word.

46. The system of claim 45, wherein the maximum likelihood criterion comprises probabilities.

47. The system of claim 46, wherein said probabilities comprise a-priori probabilities of character combinations.

48. The system of claim 46, wherein said probabilities comprise higher probabilities for words appearing in a dictionary or lexicon.

49. The system of claim 46, wherein said probabilities comprise the probability of a character or word to be the correct character or word, based on a comparison between said first and second digital images.

50. The system of claim 36, wherein said logic for comparing said first and second digital images to obtain a difference image comprises:

logic for performing global geometrical transformation on at least one of said first and second digital images to obtain first and second aligned digital images;

logic for selecting a window of said first aligned digital image;

logic for selecting a window of said second aligned digital image;

logic for estimating local residual geometrical shifts, radiometric differences and blurring differences between said selected windows to obtain a second transformation;

logic for applying said second transformation to at least one of said windows;

logic for repeating said estimating and applying for consecutive windows of said first and second aligned images to obtain first and second similar images; and

logic for comparing said first and second similar images to obtain a difference image.

51. The system of claim 36, wherein said means obtaining a first digital image of a first text comprise a scanner.

52. The system of claim 36, wherein said means obtaining a first digital image of a first text comprise a digital camera.

53. The system of claim 52, wherein said digital camera is a handheld camera.

54. The system of claim 53, wherein said handheld digital camera is part of a handheld processing device.

55. The system of claim 54, wherein said handheld processing device is a smartphone.

56. The system of claim 36, wherein said means obtaining a first digital image of a first text comprise means for retrieving the image from storage.

57. The system of claim 50, wherein said logic for performing global to geometrical transformation comprises at least one of translation in the vertical and horizontal axes, magnification, rotation, skewing and optical distortion correction.

58. The system of claim 50, wherein said logic for performing global geometrical transformation comprises:

a. logic for detecting features in said first and second digital images; and

b. logic for matching between said detected features.

59. The system of claim 50, wherein said logic for estimating residual geometrical shifts, radiometric differences and blurring differences comprises logic for applying a linear convolution and adding a constant to the window of said second aligned digital image.

60. The system of claim 59, wherein said logic for applying a linear convolution and adding a constant comprises logic for calculating the convolution coefficients and constant as a linear combination of several windows, said windows comprising translations of said selected window of said second aligned digital image and a constant.

61. The system of claim 50, wherein said logic for estimating residual geometrical shifts, radiometric differences and blurring differences comprises logic for applying a local Fourier analysis to said first selected window of said first aligned digital image and comparing it to the Fourier analysis of said selected window of said second aligned digital image.

62. The system of claim 61, wherein said logic for estimating residual geometrical shifts between features in said selected window of said first aligned image and corresponding features in said selected window of said second aligned image comprises logic for finding the linear shift of the phase in the frequency analysis.

63. The system of claim 61, wherein said logic for estimating blurring and radiometric differences between said selected window of said first aligned image and said selected window of said second aligned image comprises logic for calculating the ratio of the amplitudes of the coefficients in the frequency analysis and fitting an appropriate model.

64. The system of claim 50, wherein said logic for estimating residual geometrical shifts between features in said selected window of said first aligned image and corresponding features in said selected window of said second aligned image comprises:

logic for calculating the integer pixel shift between said selected window of said first aligned image and said selected window of said second aligned image; and

logic for calculating the sub-pixel shift between said selected window of said first aligned image and said selected window of said second aligned image.

65. The system of claim 64, wherein said logic for calculating the integer pixel shift comprises logic for calculating one of the maximum correlation and the minimum difference between said windows in the vertical and horizontal directions and finding the shift at which the highest correlation or minimum difference is found.

66. The system of claim 65, wherein said logic for calculating the sub-pixel shift comprises logic for fitting a second order curve to the values of the correlation function at the best fit position and at positions that are shifted by one pixel in both directions and finding the maximum value position of said second order curve.

67. The system of claim 65, wherein said logic for calculating the sub-pixel shift comprises logic for calculating the mean square error of the two image windows, fitting a second order curve to the values of the mean square error function at the best fit position and at positions that are shifted by one pixel in both directions and finding the minimum value position of said second order curve.

68. The system of claim 65, wherein said logic for calculating the sub-pixel shift comprises logic for calculating the mean absolute difference between the two image windows, fitting a V shape curve to the values of the mean absolute difference function at the best fit position and at positions that are shifted by one pixel in both directions and finding the minimum value position of said V shape curve.

69. The system of claim 50, wherein at least one said logic for performing global and local geometrical transformations comprises logic for calculating an optical flow.

70. The system of claim 50, wherein at least one said logic for performing global and local geometrical transformations comprises logic for interpolating one or both of the images to increase their resolution and performing an integer pixel translation on the images of increased resolution.