FORM RECOGNITION APPARATUS, METHOD, DATABASE GENERATION APPARATUS, METHOD, AND STORAGE MEDIUM

• US 20100008578A1
• Filed: 06/19/2009
• Published: 01/14/2010
• Est. Priority Date: 06/20/2008
• Status: Active Grant

• Current Assignee: Fujitsu Frontech Limited (Fujitsu Limited)
• Original Assignee: Fujitsu Frontech Limited (Fujitsu Limited)
• Inventors: Kawashima, Hajime, Hasegawa, Shohei, Eguchi, Shinichi, Yabuki, Maki, Kobara, Katsutoshi, Kanamoto, Kouichi
• Granted Patent: US 8,891,871 B2
• Time in Patent Office: Days
• US Class Current: 382/175
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1444 Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields
• CPC Class Codes: G06V 30/412 Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables

Abstract

One system to which the present invention is applied obtains the digitized form image of a form, recognizes a character string existing in the obtained form image, extracts a headline wording being a predetermined character string from the recognized character strings, determines a table structure existing in the form image, on the basis of the extracted headline wording and the arrangement of headline wordings in the form image and specifies a correspondence relationship between a headline wording and a character string other than the headline wording that is recognized, using the determination result.

Claims

1. A form recognition apparatus for recognizing a character string existing in an arbitrary table structure in a form, comprising:

an image acquisition unit capable of for obtaining a digitized form image of the form;

a character string recognition unit capable of for recognizing a character string existing in the form image obtained by the image acquisition unit;

a character string extraction unit capable of for extracting a headline wording being a predetermined character string from character strings recognized by the character string recognition unit;

a table structure determination unit capable of for determining a table structure existing in the form image, on the basis of the headline wording extracted by the character string extraction unit and arrangement of the headline wordings in the form image; and

a correspondence relationship specification unit capable of for specifying a correspondence relationship between the headline wording and a character string other than the headline wording, recognized by the character string recognition unit, using a determination result of the table structure by the table structure determination unit.

2. The form recognition apparatus according to claim 1, further comprising

a storage unit capable of for storing a database in which headline wordings that may appear in a unit table structure being a unit for entering one or more pieces of related data in the form are defined in a hierarchical structure for each unit table structure, wherein

the table structure determination unit determines the entire table structure existing in the form image, referring to the database stored in the storage unit.

3. The form recognition apparatus according to claim 2,

wherein

the hierarchical structure includes a first layer in which one or more headline wordings are defined, a second layer in which a first representative headline being a headline wording that represents a headline wording defined in the first layer and can be replaced with another headline wording and a third layer in which a second representative headline being a headline wording ordered higher than the first representative headline in the table structure, and

the table structure determination unit sets a range of the unit table structure having the second representative headline, on the basis of an extraction result of the second representative headline, by the character string extraction unit, evaluates other headline wordings extracted in the set range and specifies a headline wording existing in the unit table structure.

4. The form recognition apparatus according to claim 1,

wherein

the hierarchical structure includes a first layer in which one or more headline wordings are defined, a second layer in which a first representative headline being a headline wording that represents a headline wording defined in the first layer and can be replaced with another headline wording and a third layer in which a second representative headline being a headline wording ordered higher than the first representative headline in the table structure, further comprising

a headline addition unit capable of for setting a range of the unit table structure having the second representative headline, on the basis of an extraction result of the second representative headline, by the character string extraction unit and specifying and adding a first representative headline to be added to the unit table structure, focusing on the first representative headline replaceable with another headline wording extracted in the set range and data recognized as a character string different from the headline wording in the unit table structure.

5. The form recognition apparatus according to claim 1, further comprising:

a position specification unit capable of for specifying a position in the form image, in which a headline wording not recognized by the character string recognition unit exists, on the basis of a headline wording extracted by the character string extraction unit and the table structure determined by the table structure determination unit; and

a phrase creation unit capable of for creating a headline wording to exist in a position specified by the position specification unit.

6. The form recognition apparatus according to claim 1,

wherein

the correspondence relationship specification unit refers to accessory information of a symbol that may be attached to a character string existing as information of the headline wording, defined for each of the headline wordings and specifies the correspondence relationship.

7. A form recognition apparatus for recognizing a character string existing in an arbitrary table structure in a form, comprising:

an image acquisition unit capable of for obtaining a digitized form image of the form;

a character string recognition unit capable of for recognizing a character string existing in the form image obtained by the image acquisition unit;

a character string extraction unit capable of for extracting a headline wording being a predetermined character string from character strings recognized by the character string recognition unit;

a position specification unit capable of for specifying a position in the form image, in which a headline wording not recognized by the character string recognition unit, on the basis of a result extracted by the character string extraction unit;

a phrase creation unit capable of for creating a headline wording to existing a position specified by the position specification unit; and

a correspondence relationship specification unit capable of for specifying a correspondence relationship between the headline wording including a headline wording created by the phrase creation unit and a character string other than the headline wording, recognized by the character string recognition unit.

8. The form recognition apparatus according to claim 7, further comprising

a table structure determination unit capable of for determining a table structure existing in the form image, on the basis of a headline wording extracted by the character string extraction unit and arrangement of the headline wordings in the form image, wherein

the position specification unit specifies the position using a determination result of the table structure determination unit and

the phrase creation unit creates a headline wording to exist in the position from a determination result of the table structure determination unit.

9. The form recognition apparatus according to claim 8, further comprising

a storage unit capable of for storing a database in which headline wordings that may appear in a unit table structure being a unit for entering one or more pieces of related data in the form are defined in a hierarchical structure for each unit table structure, wherein

the table structure determination unit determines the entire table structure existing in the form image, referring to a database stored in the storage unit.

10. The form recognition apparatus according to claim 9,

wherein

the hierarchical structure includes a first layer in which one or more headline wordings are defined, a second layer in which a first representative headline being a headline wording that represents a headline wording defined in the first layer and can be replaced with another headline wording and a third layer in which a second representative headline being a headline wording ordered higher than the first representative headline in the table structure, and

the table structure determination unit sets a range of the unit table structure having the second representative headline, on the basis of an extraction result of the second representative headline, by the character string extraction unit, evaluates other headline wordings extracted in the set range and specifies a headline wording existing in the unit table structure.

11. The form recognition apparatus according to claim 9,

wherein

the hierarchical structure includes a first layer in which one or more headline wordings are defined, a second layer in which a first representative headline being a headline wording that represents a headline wording defined in the first layer and can be replaced with another headline wording and a third layer in which a second representative headline being a headline wording ordered higher than the first representative headline in the table structure,

the position specification unit extracts a range of the unit table structure having the second representative headline, on the basis of an extraction result of the second representative headline, by the character string extraction unit and specifies the position in the range, focusing on the first representative headline replaceable with another headline wording recognized in the extracted range and a character string recognized as data in the unit table structure and

the phrase creation unit creates the headline wording by specifying a first representative to be added to a position specified in the range by the position specification unit.

12. The form recognition apparatus according to claim 11, wherein

the first representative headline has a non-overlapping exclusive relationship for each of the unit table structure and

the phrase creation unit specifies a first representative headline different from a first representative headline recognized in the range, using the exclusive relationship.

13. The form recognition apparatus according to claim 12, wherein

the database defines data information of data entered into the unit table structure, and

the phrase creation unit specifies a first representative headline different from a first representative headline recognized in the range, using the exclusive relationship and the data information.

14. The form recognition apparatus according to claim 7, further comprising:

a storage unit capable of for storing a database in which headline wordings that may appear in a unit table structure being a unit for entering one or more pieces of related data into the form are defined in a hierarchical structure for each unit table structure; and

a headline addition unit capable of for setting a range of the unit table structure having the second representative headline, on the basis of an extraction result of the second representative headline, by the character string extraction unit and specifying and adding a first representative headline to be added to the unit table structure, focusing on the first representative headline replaceable with another headline wording extracted in the set range and data recognized as a character string different from the headline wording in the unit table structure.

15. The form recognition apparatus according to claim 7, further comprising

a cell creation unit capable of for creating a cell being a region including the character string, using a recognition result of the character string recognition unit, wherein

the position specification unit specifies the position for each cell created by the cell creation unit.

16. The form recognition apparatus according to claim 7, further comprising:

a cell creation unit capable of for creating a cell being a region including the character string; and

a cell correction unit capable of for correcting a plurality of cells that is for a character string entered into the form as data and continues in the same direction, of cells created by the cell creation unit.

17. The form recognition apparatus according to claim 7,

wherein

the correspondence relationship specification unit specifies the correspondence relationship, referring to accessory information of a symbol that may be attached to a character string existing as information of the headline wording, defined for each of the headline wordings.

18. A form recognition apparatus for recognizing a character string existing in an arbitrary table structure in a form, comprising:

an image acquisition unit capable of for obtaining a digitized form image of the form;

a storage unit capable of for storing a database in which headline wordings that may appear in a unit table structure being a unit for entering one or more pieces of related data into the form are defined in a hierarchical structure for each unit table structure;

a character string recognition unit capable of for recognizing a character string existing in the form image obtained by the image acquisition unit;

a character string extraction unit capable of for extracting a headline wording being a predetermined character string being a predetermined character string from character strings recognized by the character string recognition unit, referring to a database stored in the storage unit;

a headline addition unit capable of for extracting a range of the unit table structure existing in the form image, on the basis of an extraction result by the character string extraction unit and adding a headline wording corresponding to data, focusing on at least one of the headline wording recognized in the extracted range and a character string recognized as data in the unit table structure; and

a correspondence relationship specification unit capable of for specifying a correspondence relationship between the headline wording including a headline wording added by the headline addition unit and a character string other than the headline wording, recognized by the character string recognition unit.

19. The form recognition apparatus according to claim 18, wherein

the hierarchical structure includes a first layer in which one or more headline wordings are defined, a second layer in which a first representative headline being a headline wording that represents a headline wording defined in the first layer and can be replaced with another headline wording and a third layer in which a second representative headline being a headline wording ordered higher than the first representative headline in the table structure;

and

the headline addition unit extracts a range of the unit table structure having the second representative headline, on the basis of an extraction result of the second representative headline, by the character string extraction unit and adds a first representative headline corresponding to data, focusing on at least one of the first representative headline recognized in the extracted range and a character string recognized as data in the unit table structure.

20. The form recognition apparatus according to claim 18, wherein

the correspondence relationship specification unit specifies the correspondence relationship, referring to accessory information of a symbol that may be attached to a character string existing as information of the headline wording, defined for each of the headline wordings.

21. A form recognition apparatus for recognizing a character string existing in an arbitrary table structure in a form, comprising:

an image acquisition unit capable of for obtaining a digitized form image of the form;

a character string recognition unit capable of for recognizing a character string existing in the form image obtained by the image acquisition unit;

a character string extraction unit capable of for extracting a headline wording being a predetermined character string being a predetermined character string from character strings recognized by the character string recognition unit;

a table structure determination unit capable of for determining a table structure existing in the form image, on the basis of a headline wording extracted by the character string extraction unit and arrangement of the headline wordings in the form image;

a position specification unit capable of for specifying a position in the form image, in which a headline wording not recognized by the character string recognition unit exists, on the basis of a result extracted by the character string extraction unit;

a phrase creation unit capable of for creating a headline wording to exist in a position specified by the position specification unit;

a headline addition unit capable of for extracting a range of the unit table structure existing in the form image, on the basis of an extraction result by the character string extraction unit and adding a headline wording corresponding to data, focusing on at least one of the headline wording recognized in the extracted range and a character string recognized as data in the unit table structure; and

a correspondence relationship specification unit capable of for specifying a correspondence relationship between the headline wording, including a headline wording added by the headline addition unit and a character string other than the headline wording, recognized by the character string recognition unit.

22. The form recognition apparatus according to claim 21, wherein

the correspondence relationship specification unit specifies the correspondence relationship, referring to accessory information of a symbol that may be attached to a character string existing as information of the headline wording, defined for each of the headline wordings.

23. A form recognition method for recognizing a character string existing in an arbitrary table structure in a form, comprising:

obtaining a digitized form image of the form;

recognizing a character string existing in the form image obtained by the image acquisition unit;

extracting a headline wording being a predetermined character string from character strings recognized by the character string recognition process;

determining a table structure existing in the form image, on the basis of a headline wording extracted by the character string extraction process and arrangement of the headline wordings in the form image; and

specifying a correspondence relationship between the headline wording and a character string other than the headline wording, recognized by the character string recognition process, using a determination result of the table structure by the table structure determination process.

24. A form recognition method for recognizing a character string existing in an arbitrary table structure in a form, comprising:

obtaining a digitized form image of the form;

recognizing a character string existing in the form image obtained by the image acquisition process;

extracting a headline wording being a predetermined character string from character strings recognized by the character string recognition process;

specifying a position in the form image, in which a headline wording not recognized by the character string recognition process, on the basis of a result extracted by the character string extraction process;

creating a headline wording to existing a position specified by the position specification process; and

specifying a correspondence relationship between the headline wording, including a headline wording created by the phrase creation process and a character string other than the headline wording, recognized by the character string recognition process.

25. A form recognition method for recognizing a character string existing in an arbitrary table structure in a form, comprising:

obtaining a digitized form image of the form;

recognizing a character string existing in the form image obtained by the image acquisition process;

extracting a headline wording being a predetermined character string from character strings recognized by the character string recognition process, referring to a database in which headline wordings that may appear in the unit table structure, for each unit structure being a table structure being a unit for entering one or more pieces of related data in the form;

extracting a range of the unit table structure existing in the form image, on the basis of an extraction result by the character string extraction process and adding a headline wording corresponding to data, focusing on at least one of the headline wording recognized in the extracted range and a character string recognized as data in the unit table structure; and

specifying a correspondence relationship between the headline wording, including a headline wording added by the headline addition process and a character string other than the headline wording, recognized by the character string recognition process.

26. A database creation support apparatus for supporting creation of a database that can be used to recognize a character string in a form by a form recognition apparatus comprising:

a phrase input unit capable of for inputting the headline wordings; and

a hierarchical structure creation unit capable of for creating a hierarchical structure among headline wordings inputted by the phrase input unit.

27. The database creation support apparatus according to claim 26, wherein

the phrase input unit inputs two or more headline wordings that may appear in the unit table structure and a hierarchical relationship among the two or more headline wordings as an input unit for each unit table structure being a table structure being a unit for entering one or more pieces of related data in the form, and

the hierarchical structure creation unit creates three or more-layer hierarchical structure, on the basis of a headline wording common among the input unit and a hierarchical relationship indicated by each input unit.

28. The database creation support apparatus according to claim 26, further comprising

an image pick-up unit capable of for picking up and outputting a design image of the unit table structure corresponding to a hierarchical structure among headline wordings created by the hierarchical structure creation unit.

29. A database creation support method for supporting creation of a database that can be used to recognize a character string in the form by the form recognition apparatus comprising:

inputting headline wordings; and

creating a hierarchical structure among headline wordings inputted by the phrase input process.

30. A computer-accessible storage medium on which is recorded a program for enabling a computer used as a form recognition apparatus for recognizing a character string existing in an arbitrary table structure in a form to realize functions, said functions comprising:

obtaining a digitized form image of the form;

recognizing a character string existing in the form image obtained by the image acquisition function;

extracting a headline wording being a predetermined character string from character strings recognized by the character string recognition function;

determining a table structure existing in the form image, on the basis of a headline wording extracted by the character string extraction function and arrangement of the headline wordings in the form image; and

specifying a correspondence relationship between the headline wording and a character string other than the headline wording, recognized by the character string recognition function, using a determination result of the table structure by the table structure determination function.

31. A computer-accessible storage medium on which is recorded a program for enabling a computer used as a form recognition apparatus for recognizing a character string existing in an arbitrary table structure in a form to realize functions, said functions comprising:

obtaining a digitized form image of the form;

recognizing a character string existing in the form image obtained by the image acquisition function;

extracting a headline wording being a predetermined character string from character strings recognized by the character string recognition function;

specifying a position in the form image existing in a headline wording not recognized by the character string recognition function, on the basis of a result extracted by the character string extraction function; and

specifying a correspondence relationship between the headline wording and a character string other than the headline wording, recognized by the character string recognition function, using a determination result of the table structure by the table structure determination function.

32. A computer-accessible storage medium on which is recorded a program for enabling a computer used as a form recognition apparatus for recognizing a character string existing in an arbitrary table structure in a form to realize functions, said functions comprising:

obtaining a digitized form image of the form;

recognizing a character string existing in the form image obtained by the image acquisition function;

extracting a headline wording being a predetermined character string from character strings recognized by the character string recognition function;

extracting a range of the unit table structure existing in the form image, on the basis of an extraction result of the character string extraction function and adding a headline wording corresponding to the data, focusing on at least one of the headline wording recognized in the extracted range and a character string recognized as data in the unit table structure; and

specifying a correspondence relationship between the headline wording, including a headline wording added by the headline addition function and a character string other than the headline wording, recognized by the character string recognition function.

33. A computer-accessible storage medium on which is recorded a program for enabling a computer used as a database creation support apparatus that can be used to recognize a character string in a form by a form recognition apparatus to realize functions, said functions comprising:

inputting the headline wordings; and

creating a hierarchical structure among headline wordings inputted by the phrase input function.