Character segmentation using an associative memory for optical character recognition

• US 5,299,269 A
• Filed: 12/20/1991
• Issued: 03/29/1994
• Est. Priority Date: 12/20/1991
• Status: Expired due to Term

• Current Assignee: Eastman Kodak Company
• Original Assignee: Eastman Kodak Company
• Inventors: Barski, Lori L., Gaborski, Roger S.
• Primary Examiner(s): Razavi, Michael T.
• Assistant Examiner(s): KLOCINSKI, STEVEN
• Application Number:
  US07/811,477
• Time in Patent Office: 830 Days
• Field of Search: 382/14, 382/9, 382/36, 382/34, 382/18, 382/30, 382/15
• US Class Current: 382/156
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 30/19147 Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 30/194 References adjustable by an adaptive method, e.g. learning

Abstract

Character segmentation apparatus for an optical character recognition system for segmenting individual character images in an image of a document having many characters prior to performing character identification, including a movable kernel for capturing a sub-image framed within a window having an area corresponding to an area occupied by an individual character, the window being movable in the document image in pixel-by-pixel steps to capture a sub-image for each step of the window, an associative memory for responding to the captured sub-image by producing a corresponding one of a set of images of known characters with which the associative memory has been trained, and a sensor responsive to the behavior of the associative memory for determining whether the sub-image is the image of an individual character or a non-character.

Claims

1. Character segmentation apparatus for an optical character recognition system for segmenting individual character images in an image of a document comprising plural characters, prior to performing character identification, said image of a document comprising an array of pixels, said apparatus comprising:

kernel means for capturing, in said image of a document, a sub-image framed within a window having an area corresponding to an area occupied by an individual character of said document, said window being movable in said image of a document in pixel-by-pixel steps whereby said kernel means captures a sub-image for each step of said window;

associative memory means for responding to said sub-image by producing a corresponding one of a set of training images of known characters with which said associative memory has been trained; and

sensor means, responsive to said associative memory means producing said corresponding one of a set of training images of known characters, for determining whether said sub-image is the image of an individual character, and segmenting said sub-image as an unidentified individual character image in response to said sub-image corresponding to the image of an individual character.

2. The character segmentation apparatus of claim 1 wherein:

said associative memory means has been trained with said set training of images of known characters and in addition thereto has been trained with a set of training images of known non-characters corresponding to positions of said window framing more than a single one of said characters in said image of said document;

each image in said set of training images of known characters and in said set of images of training known non-characters with which said associative memory has been trained comprises an image portion corresponding to the area of said window and a tag portion containing information identifying whether said image with which said associative memory has been trained belongs to said set of training images of known characters or belongs to said set of training images of known non-characters; and

said sensor means comprises means for reading the tag portion of said image produced by said associative memory.

3. The apparatus of claim 2 wherein said window comprises m pixel rows and n pixel columns, said image portion of each image with which said associative memory has been trained comprises m pixel rows and n pixel columns and said tag portion comprises an extra m+1^st row adjacent the m^th row of said image portion, said tag portion comprising pixels having a predetermined pattern uniquely defining said image portion as being one of a character or a non-character.

4. The apparatus of claim 3 wherein said associative memory comprises an (m+1)×

n by (m+1)×

n matrix multiplier, said sub-image comprises m pixel rows and n pixel columns, said apparatus further comprising;

means for converting each m by n pixel sub-image to a vector comprising m×

n elements;

means for matching the dimensionality of said vector to the dimensionality of said matrix multiplier by appending n additional elements to said vector corresponding to the location of said tag portion of each image with which said associative memory has been trained whereby to form an augmented vector, and for transmitting said augmented vector to said matrix multiplier for multiplication of said augmented vector by a reference vector.

5. The apparatus of claim 4 wherein each of said n additional elements has a zero value.

6. The apparatus of claim 1 wherein said associative memory means exhibits the emergent behavior in producing said corresponding image and wherein sensor means comprises means for determining from emergent behavior exhibited by said associative memory whether said sub-image is the image of one of said known characters.

7. The apparatus of claim 6 wherein:

said associative memory means comprises;

means for modifying said sub-image to produce a transformed sub-image which resembles one of said images with which said associative memory has been trained more closely than said sub-image;

means for determining whether a difference between said sub-image and said transformed sub-image exceeds a predetermined threshold;

means responsive to said means for determining for feeding back said transformed sub-image to said means for modifying whereby said transformed sub-image is modified to produce a next transformed sub-image, whereby said means for modifying produces an iterative series of transformed sub-images in a series of iterative repetitions until performing a repetition for which said difference does not exceed said predetermined threshold; and

said sensor means comprises;

means for counting the number of said iterative repetitions performed by said means for modifying and for comparing said number with a threshold count.

8. The apparatus of claim 7 wherein said means for modifying comprises:

means for transforming said sub-image into a vector;

means for multiplying said vector by a matrix whose elements each comprise a sum of corresponding elements of outer product matrices of a set of vectors corresponding to said set of images of known characters.

9. The apparatus of claim 8 wherein said window is characterized by m pixel rows and n pixel columns and wherein each vector comprises m×

n elements, and wherein each of said matrices comprises m×

n rows and n×

m columns of matrix elements.

10. The apparatus of claim 1 wherein said associative memory means comprises first and second associative memories, said first associative memory trained with said set of training images of known characters and said second associative memory trained with a set of training images of non-characters corresponding to sub-images, captured in said window, of more than one character in said document, said apparatus further comprising means for transmitting said sub-image captured in said window to said first and second associative memories, said sensor means comprising means for comparing emergent behavior responses to said sub-image of said first and second associative memories.

11. The apparatus of claim 10 wherein:

each one of said first and second associative memories comprises;

means for modifying said sub-image to produce a transformed sub-image which resembles one of said images with which said each one of said associative memories has been trained more closely than said sub-image;

means for determining whether a difference between said sub-image and said transformed sub-image exceeds a predetermined threshold;

means responsive to said means for determining for feeding back said transformed sub-image to said means for modifying whereby said transformed sub-image is modified to produce a next transformed sub-image, whereby said means for modifying produces an iterative series of transformed sub-images in a series of iterative repetitions until performing a repetition for which said difference does not exceed said predetermined threshold; and

said sensor means comprises;

means for counting the number of said iterative repetitions performed by said means for modifying and for comparing said number with a threshold count.

12. The apparatus of claim 11 wherein said means for modifying comprises:

means for transforming said sub-image into a vector;

means for multiplying said vector by a matrix, whose elements each comprise a sum of corresponding elements of outer product matrices of a set of vectors, corresponding to one of said set of training images of known characters or said set of training images of non-characters.

13. A character segmentation method in optical character recognition for segmenting individual character images in an image of a document comprising plural characters in preparation for performing character identification, said image of a document comprising an array of pixels, said method comprising:

capturing, in said image of a document, a sub-image framed within a window having an area corresponding to an area occupied by an individual character of said document, said window being movable in said image of a document in pixel-by-pixel steps whereby a sub-image is captured for each step of said window;

responding to said sub-image by producing a corresponding one of a set of training images of known characters;

determining from results of said producing step whether said sub-image is the image of an individual character; and

in response to said determining step determining that said sub-image is the image of an individual character, segmenting such sub-image as an unidentified individual character image.

14. The character segmentation method of claim 13 wherein:

wherein said set of training images includes a first set of images of known characters and a second set of images of known non-characters corresponding to positions of said window framing more than a single one of said characters in said image of said document;

each training image in said set of images of known characters and in said set of known non-characters comprises an image portion corresponding to the area of said window and a tag portion containing information classifying said training image as belonging to one of said first and second sets of images;

said determining step comprises reading the tag portion of said training image.

15. The method of claim 14 wherein said window comprises m pixel rows and n pixel columns, said image portion of each training image comprises m pixel rows and n pixel columns and said tag portion comprises an extra ^m+1st row adjacent the m^th row of said image portion, said tag portion comprising pixels having a predetermined pattern uniquely defining said image portion as being one of a character or a non-character.

16. The method of claim 15 wherein said producing step comprises transforming said sub-image with an (m+1)×

n by (m+1)×

n matrix, said sub-image comprises m pixel rows and n pixel columns, said method further comprising;

converting each m by n pixel sub-image to a vector comprising m×

n elements;

matching the dimensionality of said vector to the dimensionality of said matrix by appending n additional

elements to said vector corresponding to the location of said tag portion in each of said training images whereby to form an augmented vector, and multiplying said augmented vector by a reference vector.

17. The method of claim 16 wherein each of said n additional elements has a zero value.

18. The method of claim 13 wherein said producing step exhibits emergent behavior in producing said corresponding training image and wherein said determining step comprises determining from emergent behavior exhibited by said producing step whether said sub-image is the training image of one of said known characters.

19. The method of claim 18 wherein:

said producing step comprises;

treating said sub-image as an input image and modifying said input image to produce an output image which resembles one of said training images more closely than said input image;

deciding whether a difference between said input an output images exceeds a predetermined threshold;

in response to said deciding step, feeding back said output image as a next input image and repeating said modifying step to produce a next output image and then repeating said deciding step with said next input and output images, whereby an iterative series of repetitions of said modifying and deciding steps are performed until a repetition is performed for which said difference does not exceed said predetermined threshold; and

said determining step comprises;

counting the number of said iterative repetitions of said modifying step and comparing said number with a threshold count.

20. The method of claim 19 wherein said modifying step comprises:

transforming said image into a vector;

multiplying said vector by a matrix whose elements each comprise a sum of corresponding elements of outer product matrices of a set of vectors corresponding to said set of training images of known characters.

21. The method of claim 20 wherein said window is characterized by m pixel rows and n pixel columns and wherein each vector comprises m×

n elements, and wherein said matrices comprises m×

n rows and n×

m columns of matrix elements.

22. The method of claim 13 wherein said producing step comprises transmitting said sub-image to a pair of associative memories trained, respectively, with said set of training images of known characters and a set of training images of non-characters corresponding to sub-images captured in said window of more than one character in said document, and said determining step comprises comparing emergent behavior responses of said pair of trained associative memories.

23. The method of claim 22 wherein:

said producing step further comprises performing the following steps in each one of said pair of associative memories;

modifying said sub-image to produce a transformed sub-image which resembles one of said images with which said one associative memory has been trained more closely than said sub-image;

determining whether a difference between said sub-image and said transformed sub-image exceeds a predetermined threshold;

in response to said determining step, feeding back said transformed sub-image to said modifying step whereby said transformed sub-image is modified to produce a next transformed sub-image, whereby an iterative series of repetitions of said modifying step is performed until a repetition is performed for which said difference does not exceed said predetermined threshold; and

said determining step further comprises;

counting the number of said iterative repetitions of said modifying step and comparing said number with a threshold count.

24. the method of claim 23 wherein said modifying step comprises:

transforming said sub-image into a vector;

multiplying said vector by a matrix, whose elements each comprise a sum of corresponding elements of outer product matrices of a set of vectors, corresponding to one of said set of training images of known characters or said set of training images of non-characters.

25. An associative memory with an internal capability of classifying its output as falling into one of a plurality of different sets of training vectors, representative of respective sets of characters and non-characters, with which said associative memory has been trained comprising:

means for receiving an m×

n element input vector corresponding to an m-by-n input data array;

means for producing one of a plurality of training vectors which most closely resembles said input vector, said training vectors each comprising an information portion comprising m×

n elements corresponding to an m-by-n data array and an n element tag portion corresponding to an m+1^st row of data not included in said data array, said tag portion being operative to identify said each training vector as a member of a set of training vectors representative of characters or a set of training vectors representative of non-characters.

26. The associative memory of claim 25 further comprising means for reading the tag portion of the training vector produced by said means for producing.

27. The associative memory of claim 25 wherein said means for producing comprise:

means for converting said input vector to an (m+1)×

n element vector by appending n elements to said input vector in a location thereof corresponding to the location of the tag portions of said training vectors to produce an augmented input vector;

means for multiplying said augmented input vector received at an input thereof by a (m+1)×

n by (m+1)×

n element matrix whose elements each comprise the sum of corresponding elements of (m+1)×

n by (m+1)×

n element outer product matrices of each one of said (m+1)×

n element training vectors, whereby to produce an output vector at an output thereof;

means for feeding back the output vector produced at the output of said means for multiplying to said input of said means for multiplying to produce another output vector whenever the difference between the vector at said input and output of said means for multiplying exceeds a predetermined threshold, and for outputting the most recent output vector produced at said output of said means for multiplying otherwise.

31. The associative memory of claim 25 wherein each one of said training vectors has a unique data pattern in its tag row.

32. The associative memory of claim 31 wherein each training vector corresponds to an image of an individual character and the data pattern in the tag row thereof uniquely identifies the individual character to which the training vector corresponds.

28. An associative memory with an internal capability of classifying its output as falling into one of a plurality of classes, comprising:

means for receiving an input vector;

trained neural network means for producing one of a plurality of vectors with which the neural network has been previously trained, which one vector most closely resembles said input vector, by performing an operation which is repeatable until one of said plurality of vectors is produced; and

means responsive to the number of times said operation is repeated for determining whether said one vector falls within a predetermined class.

29. The associative memory of claim 28 wherein said neural network means comprise two neural networks each trained to produce a vector most closely resembling said input vector from a respective one of two sets of training vectors corresponding to two classes of training vectors by performing repeatable operations, and wherein said means for determining compares the number of repetitions of said operations by said two neural networks in response to the same input vector.

30. The associative memory of claim 28 wherein said repeatable operation comprises multiplying said input vector by a matrix whose elements each comprise a sum of corresponding elements of the outer product matrices of all of said training vectors.