Segregation of handwritten information from typographic information on a document

• US 9,336,437 B2
• Filed: 08/12/2014
• Issued: 05/10/2016
• Est. Priority Date: 01/27/2010
• Status: Active Grant

• Current Assignee: DST Technologies, Inc. (SS&C Technologies Holdings Incorporated)
• Original Assignee: DST Technologies, Inc. (SS&C Technologies Holdings Incorporated)
• Inventors: Ives, Paul M., Clark, Peter E., Gentry, Michael V.
• Primary Examiner(s): TUCKER, WESLEY J
• Application Number:
  US14/457,784
• Publication Number:
  US 20140348422A1
• Time in Patent Office: 637 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06V 30/412 Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables

Abstract

A system for segregating handwritten information from typographic information on a document may include a memory, an interface, and a processor. The memory stores an electronic document image of a document where the electronic document image includes pixels and each pixel has a characteristic. The processor may receive, via the interface, the electronic document image and may identify first, second and third most frequently occurring characteristics of the pixels of the electronic document image. The pixels having the first most frequently occurring characteristic represent a background of the document. The processor may determine the typographic information of the document as represented by pixels having the second most frequently occurring characteristic. The processor may determine the handwritten information of the document as represented by pixels having the third most frequently occurring characteristic. The processor may derive a first representation of the handwritten information and a second representation of the typographic information.

Claims

1. A method of segregating handwritten information from typographic information on a document, the method comprising:

receiving an electronic document image of a document, the electronic document image comprising a plurality of pixels, wherein each of the plurality of pixels comprises one of a plurality of characteristics;

identifying a first, a second, and a third most frequently occurring characteristic of the plurality of pixels, wherein the pixels of the plurality of pixels comprising the first most frequently occurring characteristic of the plurality of characteristics or the second most frequently occurring characteristic of the plurality of characteristics represent typographic information;

determining a background of the document, wherein the background is represented by the pixels of the plurality of pixels of the electronic document image which comprise the first most frequently occurring characteristic of the plurality of characteristics or the second most frequently occurring characteristic of the plurality of characteristics;

determining, by a computer processor, handwritten information of the document, wherein the handwritten information is represented by the pixels of the plurality of pixels of the electronic document image which comprise the third most frequently occurring characteristic of the plurality of characteristics; and

deriving a first representation of the handwritten information and a second representation of the typographic information.

2. The method of claim 1 wherein the characteristic of each of the plurality of pixels comprises a color of a plurality of colors, or a range of colors of the plurality of colors.

3. The method of claim 1 further comprising filtering out pixels of the plurality of pixels from the electronic document image if the color of the pixels of the plurality of pixels substantially comprises a shade of black or a shade of grey.

4. The method of claim 3 wherein a degree of the shade of black or the shade of grey is based on a shade of the second most frequently occurring color of the plurality of colors, or a shade of the third most frequently occurring color of the plurality of colors.

5. The method of claim 1 further comprising:

determining a plurality of characters corresponding to the handwritten information;

determining a document type corresponding to the typographic information; and

providing the plurality of characters and an indication of the document type.

6. The method of claim 5 further comprising determining a location of the handwritten information on the document relative to the typographic information.

7. The method of claim 6 further comprising determining a data field corresponding to the handwritten information based on the location of the handwritten information on the document relative to the typographic information, and the document type corresponding to the typographic information.

8. The method of claim 7 further comprising storing the plurality of characters in a database record corresponding to the data field.

9. The method of claim 6 wherein determining the document type corresponding to the typographic information further comprises determining the document type corresponding to the typographic information based in part on the location of the handwritten information on the document relative to the typographic information.

10. The method of claim 6 further comprising:

determining an expected location of the handwritten information on the document relative to the typographic information; and

storing an indication of whether the location of the handwritten information on the document relative to the typographic information coincides with the expected location of the handwritten information on the document relative to the typographic information.

11. The method of claim 5 wherein determining the plurality of characters corresponding to the handwritten information further comprises determining the plurality of characters corresponding to the handwritten information based in part on the document type corresponding to the typographic information.

12. The method of claim 5 further comprising:

determining a region of the document where the handwritten information is located;

determining a portion of the typographic information corresponding to the region of the document;

identifying a data field associated with the portion of the typographic information based on the document type corresponding to the typographic information; and

storing the plurality of characters corresponding to the handwritten information located in the region in a record of a database corresponding to the data field.

13. The method of claim 12 wherein determining the region of the document corresponding to the handwritten information further comprises:

(a) determining a top edge, a bottom edge, a right edge and a left edge of the document, wherein the edges are determined based on an orientation of the typographic information relative to the document;

(b) identifying an upper-left most pixel of the plurality of pixels of the handwritten information, wherein the upper-left most pixel comprises a pixel located closest to the top edge of the document and closest to the left edge of the document;

(c) determining a top bound of the region of the document based on a top line running through the upper-left most pixel, wherein the top line is parallel to the top edge of the document and the bottom edge of the document;

(d) determining a bottom pixel of the plurality of pixels of the handwritten information wherein the bottom pixel is located a height number of pixels below the upper-left most pixel;

(e) determining a bottom bound of the region of the document based on a bottom line running through the bottom pixel, wherein the bottom line is parallel to the top edge of the document and the bottom edge of the document;

(f) determining a leftmost pixel of the plurality of pixels of the handwritten information located within the top bound and the bottom bound, wherein the leftmost pixel is located at the left edge of the document, or the leftmost pixel comprises a closet pixel to the left edge of the document which is located within the top bound and the bottom bound and has no other pixels within a buffer number of pixels to the left;

(g) determining a left bound of the region of the document based on a left line running through the leftmost pixel, wherein the left line is parallel to the left edge of the document and the right edge of the document;

(h) determining a rightmost pixel of the plurality of pixels of the handwritten information located within the top bound and the bottom bound, wherein the rightmost pixel is located at the right edge of the document or the rightmost pixel comprises a closest pixel to the right edge of the document which is located within the top bound and the bottom bound and has no other pixels with the buffer number of pixels to the right;

(i) determining a right bound of the region of the document based on a right line running through the rightmost pixel, wherein the right line is parallel to the left edge of the document and the right edge of the document;

(j) determining the bottom pixel of the plurality of pixels of the handwritten information which is located within the left bound and the right bound, wherein the bottom pixel is located at the bottom edge of the document or the bottom pixel is a closest pixel to the bottom edge of the document with no other pixels within the buffer number of pixels below;

(k) determining the bottom bound of the region of the document based on a bottom line running through the bottom pixel, wherein the bottom line is parallel to the top edge of the document and the bottom edge of the document;

(l) repeating steps (d)-(k) using the bottom bound determined in step (k), the left bound determined in step (g), and the right bound determined in step (i) until the bottom bound, left bound and right bound remain constant; and

(m) determining the region of the document based on an area enclosed by the top bound, the bottom bound, the left bound and the right bound.

14. The method of claim 13 further comprising:

(n) storing an indication of the region and the plurality of characters corresponding to the handwritten information located within the region.

15. The method of claim 14 further comprising:

(o) removing the pixels corresponding to the handwritten information located within the region; and

(p) repeating steps (b)-(o) until all of the plurality of pixels of the handwritten information have been removed from the electronic document image.

16. A system for segregating handwritten information from typographic information on a document, the system comprising:

a memory operative to store an electronic document image of a document, the electronic document image comprising a plurality of pixels, wherein each of the plurality of pixels comprises one of a plurality of characteristics;

an interface coupled with the memory and operative to receive the electronic document image; and

a processor coupled with the interface and operative to receive, via the interface, the electronic document image of the document, identify a first, a second, and a third most frequently occurring characteristic of the plurality of pixels, wherein the pixels of the plurality of pixels comprising the first most frequently occurring characteristic of the plurality of characteristics of the second most frequently occurring characteristic of the plurality of characteristics represent typographic information, determine a background of the document, wherein the background is represented by the pixels of the plurality of pixels of the electronic document image which comprise the first most frequently occurring characteristic of the plurality of characteristics or the second most frequently occurring characteristic of the plurality of characteristics, determine handwritten information of the document, wherein the handwritten information is represented by the pixels of the plurality of pixels of the electronic document image which comprise the third most frequently occurring characteristic of the plurality of characteristics, and derive a first representation of the handwritten information and a second representation of the typographic information.

17. The system of claim 16 further comprising:

the processor further operative to determine a plurality of characters corresponding to the handwritten information, determine a document type corresponding to the typographic information, and provide, via the interface, the plurality of characters and an indication of the document type.

18. The system of claim 17 further comprising:

the processor further operative to determine a location of the handwritten information on the document relative to the typographic information.

19. The system of claim 18 further comprising:

the processor further operative to determine a data field corresponding to the handwritten information based on the location of the handwritten information on the document relative to the typographic information, and based on the document type corresponding to the typographic information.

20. The system of claim 19 further comprising:

the processor further operative to store the plurality of characters in a database record corresponding to the data field.

21. The system of claim 18 further comprising:

the processor further operative to determine the document type corresponding to the typographic information based in part on the location of the handwritten information on the document relative to the typographic information.

22. The system of claim 18 further comprising:

the processor further operative to determine an expected location of the handwritten information within the electronic document image relative to the typographic information, and store an indication of whether the location of the handwritten information within the electronic document relative to the typographic information coincides with the expected location of the handwritten information within the electronic document image relative to the typographic information.

23. The system of claim 17 further comprising:

the processor further operative to determine the plurality of characters corresponding to the handwritten information based in part on the document type corresponding to the typographic information.

24. The system of claim 17 further comprising:

the processor further operative to determine a region of the document where the handwritten information is located, determine a portion of the typographic information corresponding to the region of the document, identify a data field associated with the portion of the typographic information based on the document type corresponding to the typographic information, and store the plurality of characters corresponding to the handwritten information located in the region in a record of the memory corresponding to the data field.

25. The system of claim 16 wherein the characteristic of each of the plurality of pixels substantially comprises a color of a plurality of colors.

26. The system of claim 25 further comprising:

the processor further operative to filter out pixels of the plurality of pixels from the electronic document image if the color of the pixels of the plurality of pixels comprises a shade of black or a shade of grey.

27. The system of claim 26 wherein a degree of the shade of black or the shade of grey is based on a shade of the second most frequently occurring color of the plurality of colors or a shade of the third most frequently occurring color of the plurality of colors.