Local connectivity feature transform of binary images containing text characters for optical character/word recognition

• US 10,521,697 B2
• Filed: 09/29/2017
• Issued: 12/31/2019
• Est. Priority Date: 09/29/2017
• Status: Active Grant

• Current Assignee: Konica Minolta Laboratory U.S.A., Inc. (Konica Minolta Inc.)
• Original Assignee: Konica Minolta Laboratory U.S.A., Inc. (Konica Minolta Inc.)
• Inventors: Agarwal, Shubham, Mesmakhosroshahi, Maral, Zhang, Yongmian
• Primary Examiner(s): Entezari, Michelle M
• Application Number:
  US15/721,610
• Publication Number:
  US 20190102653A1
• Time in Patent Office: 823 Days
• Field of Search: None
• CPC Class Codes: G06F 18/21322 Rendering the within-class scatter matrix non-singular
• CPC Class Codes: G06F 18/2414 Smoothing the distance, e.g. radial basis function networks [RBFN]
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06T 7/45 using co-occurrence matrix computation
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/15 Cutting or merging image elements, e.g. region growing, watershed or clustering-based techniques
• CPC Class Codes: G06V 30/153 using recognition of characters or words
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/194 References adjustable by an adaptive method, e.g. learning

Abstract

A local connectivity feature transform (LCFT) is applied to binary document images containing text characters, to generate transformed document images which are then input into a bi-directional Long Short Term Memory (LSTM) neural network to perform character/word recognition. The LCFT transformed image is a gray scale image where the pixel values encode local pixel connectivity information of corresponding pixels in the original binary image. The transform is one that provides a unique transform score for every possible shape represented as a 3×3 block. In one example, the transform is computed using a 3×3 weight matrix that combines bit coding with a zigzag pattern to assign weights to each element of the 3×3 block, and by summing up the weights for the non-zero elements of the 3×3 block shape.

Claims

1. A method for processing a binary document image containing text characters, the method comprising:

(a) obtaining the binary document image, the document image having a plurality of pixels, each pixel having either a first pixel value representing content of the document or a second pixel value representing background;

(b) assigning the second pixel value to all pixels located on a boundary of the document image;

(c) generating a transformed document image, the transformed document image being a grayscale image having a same size as the binary document image, including;

(c1) for each pixel (i,j) of the document image that has the second pixel value, where i and j denote position indices of the document image respectively, assigning a fixed transform score to the pixel,

(c2) for each pixel (i,j) of the document image that has the first pixel value, computing a transform score using

T(i,j)=Σ

<sub>m=−

1</sub>⁺¹Σ

<sub>n=−

1</sub>⁺¹W(m, n)*P(i+m,j+n)

where T(i,j) is the transform score of the pixel (i,j), m and n are integers and m, n ∈

{−

1, 0, +1}, W(m,n) is a 3×

3 weight matrix, and P(i+m,j+n) is the pixel value of pixel (i+m,j+n),

wherein a center element of the 3×

3 weight matrix W(m,n) has a value of zero, and each one of eight non-center elements of the 3×

3 weight matrix W(m,n) has a value which is a different one of eight numbers 2^q, q=0, 1, 2, . . . 7; and

wherein the transform scores of all pixels of the document image form the transformed image; and

(d) processing the transformed image using a bi-directional Long Short Term Memory (LSTM) neural network for character/word recognition to recognize characters or words in the transformed image.

2. The method of claim 1, wherein the values of the eight non-center elements of the 3×

3 weight matrix are assigned using a zigzag pattern.

3. The method of claim 1, wherein the 3×

3 weight matrix W(m,n) is;

4. The method of claim 1, wherein the first pixel value is 1 and the second pixel value is 0, and wherein the fixed transform score is 0.

5. The method of claim 1, wherein steps (a), (b) and (c) are performed for a plurality of binary document images to generate a plurality of transformed images, and wherein step (d) includes training the bi-directional LSTM neural network for character/word recognition using the plurality of transformed image and corresponding label data.

6. The method of claim 5, wherein steps (a), (b) and (c) are performed for additional binary document images to generate additional transformed images, and wherein step (d) includes processing the additional transformed images using the trained bi-directional LSTM neural network to recognize characters/words in the additional transformed images.

7. A method for processing a binary document image containing text characters, the method comprising:

(a) obtaining the binary document image, the document image having a plurality of pixels, each pixel having either a first pixel value representing content of the document or a second pixel value representing background;

(b) generating a transformed document image by transforming the binary document image into the transformed document image, the transformed document image being a grayscale image having a same size as the binary document image, each pixel of the transformed image representing a transform score that encodes local pixel connectivity information of the corresponding pixel in the binary document image; and

(c) processing the transformed image using a bi-directional Long Short Term Memory (LSTM) neural network for character/word recognition.

8. The method of claim 7, wherein in step (b), for each pixel of the binary document image having a first pixel value, the transform score is generated using a 3×

3 weight matrix configured to generate a unique transform score for each of a plurality of different connectivity shapes, each connectivity shapes being represented by a 3×

3 pixel block, a center pixel of the 3×

3 pixel block having the first pixel value and at least one other pixels having the first pixel value, and

wherein for each pixel of the binary document image having a second pixel value, the transform score is a fixed value.

9. The method of claim 8, wherein the transform score for each connectivity shape is a sum, over all pixels of the 3×

3 pixel block, of a product of the pixel value of the 3×

3 pixel block and the corresponding element of the 3×

3 weight matrix.

10. The method of claim 8, wherein a center element of the 3×

3 weight matrix has a value of zero, and each one of eight non-center elements of the 3×

3 weight matrix has a value which is a different one of eight numbers 2^q, q=0, 1, 2, . . . 7.

11. The method of claim 10, wherein the values of the eight non-center elements of the 3×

3 weight matrix are assigned using a zigzag pattern.

12. The method of claim 8, wherein the 3×

3 weight matrix is;

13. The method of claim 8, wherein the first pixel value is 1 and the second pixel value is 0, and wherein the fixed transform score is 0.

14. The method of claim 7, wherein steps (a) and (b) are performed for a plurality of binary document images to generate a plurality of transformed images, and wherein step (c) includes training the bi-directional LSTM neural network for character/word recognition using the plurality of transformed image and corresponding label data.

15. The method of claim 14, wherein steps (a) and (b) are performed for additional binary document images to generate additional transformed images, and wherein step (c) includes processing the additional transformed images using the trained bi-directional LSTM neural network to recognize characters/words in the additional transformed images.