Character recognition device, character recognition method, and character recognition program

• US 10,657,404 B2
• Filed: 07/17/2015
• Issued: 05/19/2020
• Est. Priority Date: 07/17/2015
• Status: Active Grant

• Current Assignee: Rakuten Group, Inc.
• Original Assignee: Rakuten, Inc. (Rakuten Group, Inc.)
• Inventors: Hinami, Ryota, Chiba, Naoki, Liu, Xinhao, Satoh, Shin'ichi
• Primary Examiner(s): Ky, Kevin
• Application Number:
  US15/744,131
• Publication Number:
  US 20180211127A1
• Time in Patent Office: 1,768 Days
• Field of Search: None
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/147 Determination of region of interest
• CPC Class Codes: G06V 30/18086 by performing operations within image blocks or by using histograms
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/347 Sampling; Contour coding; Stroke extraction
• CPC Class Codes: G06V 30/36 Matching; Classification

Abstract

A character recognition device includes an acquisition means configured to acquire an image containing a character region, a first recognition means configured to recognize a character from the character region by a first recognition method, a setting means configured to set reference lines along an alignment direction of the characters and passing through a specified position in each character, a second recognition means configured to recognize a character by a second recognition method, the second recognition method being a method that recognizes a character from an image within a recognition window by scanning in a recognition target region in an image while changing a size of the recognition window, and configured to set a position or a height in a vertical direction of the recognition window based on the reference lines, and an output means configured to output a word composed of characters recognized by the second recognition means.

Claims

1. A character recognition device comprising:

at least one memory operable to store computer program code;

at least one processor operable to read said program code and operate according to said program code, said program code including;

acquisition code configured to cause at least one of said at least one processor to acquire an image;

first recognition code configured to cause at least one of said at least one processor to identify a plurality of character regions in the image and recognize a character from each of the plurality of character regions of the image by a first recognition method;

setting code configured to cause at least one of said at least one processor to set, for at least some of a plurality of characters recognized by the first recognition code, at least one reference line along an alignment direction of the plurality of characters and passing through a specified position in each character;

second recognition code configured to cause at least one of said at least one processor to recognize a character by a second recognition method, the second recognition method being a method that recognizes a character from an image within a recognition window by scanning in a recognition target region in an image while changing a size of the recognition window, and configured to set a position or a height in a vertical direction of the recognition window based on at least one reference line out of the reference lines set by the setting code; and

output code configured to cause at least one of said at least one processor to output, as a recognition result, a word composed of characters recognized by the second recognition code, wherein

the setting code is further configured to cause at least one of said at least one processor to set at least two reference lines, and

the second recognition code is further configured to cause at least one of said at least one processor to set the position and the height in the vertical direction of the recognition window to a position and a height of a scanning region specified by predetermined two reference lines out of the at least one reference line set by the setting code.

2. The character recognition device according to claim 1, wherein

the setting code is further configured to cause at least one of said at least one processor to set the at least one reference line for the plurality of characters by referring to a storage means that stores information about a position of each reference line for each character.

3. The character recognition device according to claim 1, wherein

the first recognition code is further configured to cause at least one of said at least one processor to calculate a first reliability score indicating probability of a recognition result for each recognized character, and

the setting code is further configured to cause at least one of said at least one processor to set the at least one reference line for a character where the first reliability score is equal to or more than a specified value.

4. The character recognition device according to claim 3, wherein

the second recognition code is further configured to cause at least one of said at least one processor to perform character recognition in a region showing a character where the first reliability score is less than the specified value, and

the output code is further configured to cause at least one of said at least one processor to output, as a recognition result, a word composed of characters recognized by the first recognition code and the second recognition code.

5. The character recognition device according to claim 1, wherein

the second recognition code is further configured to cause at least one of said at least one processor to calculate a second reliability score indicating probability of a recognition result for each recognized character, and

the output code is further configured to cause at least one of said at least one processor to output, as a recognition result, a word made up of a character string formed based on the second reliability score of each character.

6. The character recognition device according to claim 1, wherein

the output code is further configured to cause at least one of said at least one processor to calculate an evaluation score indicating likelihood as a word composed of characters recognized by the second recognition code, and output a word as a recognition result based on the calculated evaluation score.

7. The character recognition device according to claim 1, wherein

the characters are alphabets, and

the setting code is further configured to cause at least one of said at least one processor to set, as the at least one reference line, an ascender line, a mean line, a base line and a descender line.

8. The character recognition device according to claim 7, wherein

the second recognition code is further configured to cause at least one of said at least one processor to specify, as a scanning region, an area between the ascender line and the base line, an area between the mean line and the base line, and an area between the mean line and the descender line.

9. A character recognition method in a character recognition device, executed by at least one processor in said character recognition device and comprising:

acquiring an image;

identifying a plurality of character regions in the image;

recognizing a character from each of the plurality of character regions of the image by a first recognition method;

setting, for at least some of a plurality of characters recognized in the recognizing of the character by the first recognition method, at least one reference line along an alignment direction of the plurality of characters and passing through a specified position in each character;

recognizing a character by a second recognition method, the second recognition method being a method that recognizes a character from an image within a recognition window by scanning in a recognition target region in an image while changing a size of the recognition window, and setting a position or a height in a vertical direction of the recognition window based on at least one reference line out of the reference lines set in the setting of the at least one reference line; and

outputting, as a recognition result, a word composed of characters recognized in the recognizing of the character by the second recognition method,

wherein the setting of the at least one reference line further includes setting at least two reference lines, and

wherein the recognizing of the character by the second recognition method further includes setting the position and the height in the vertical direction of the recognition window to a position and a height of a scanning region specified by predetermined two reference lines out of the at least one reference line.

10. A character recognition device comprising:

at least one memory operable to store computer program code;

at least one processor operable to read said program code and operate according to said program code, said program code including;

acquisition code configured to cause at least one of said at least one processor to acquire an image;

first recognition code configured to cause at least one of said at least one processor to identify a plurality of character regions in the image, recognize a character from each of the plurality of character regions of the image by a first recognition method, and identify a first reliability score corresponding to each of the plurality of character regions;

setting code configured to cause at least one of said at least one processor to identify a first character region and a second character region that have corresponding first reliability scores above a threshold value, and set at least two reference lines along an alignment direction of a first character of the first character region and a second character of the second character region;

second recognition code configured to cause at least one of said at least one processor to identify a third character region that has a corresponding first reliability score that is below the threshold value, recognize a third character in the third character region by a second recognition method, the second recognition method being a method that is different than the first recognition method and is based on the at least two reference lines set by the setting code; and

output code configured to cause at least one of said at least one processor to output, as a recognition result, a word composed of the first character, the second character and the third character.

11. The character recognition device according to claim 10, wherein the second recognition method is a method that recognizes a character from an image within a recognition window by scanning in a recognition target region in an image while changing a size of the recognition window, and

wherein the second recognition code is further configured to cause at least one of said at least one processor to set a position or a height in a vertical direction of the recognition window based on at least one reference line out of the reference lines set by the setting code in the character region.