Apparatus and method for recognizing a character image from an image screen

• US 20040057619A1
• Filed: 09/09/2003
• Published: 03/25/2004
• Est. Priority Date: 09/11/2002
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Seo, Jeong-Wook, Lim, Chae-Whan
• Granted Patent: US 7,773,808 B2
• Time in Patent Office: Days
• US Class Current: 382/182
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/127 with the intervention of an operator
• CPC Class Codes: G06V 30/162 Quantising the image signal

Abstract

A method and apparatus are provided for enabling a mobile terminal equipped with a camera to store document information. An image of a document picked up by the camera is displayed. In response to a document “RECOGNIZE” key input, a character image is recognized from the displayed document image and the character image is converted into character data. The character data is displayed on the first display area, and SAVE items are displayed on the second display area. A SAVE item is selected from the displayed SAVE items and character data associated with the selected SAVE item is selected, such that the selected SAVE item and the character data associated with the selected SAVE item can be displayed on the third display area. When a “CORRECT” key is input, candidate characters associated with an erroneously recognized character are displayed on the third display area. After the erroneously recognized character is corrected or replaced with a selected candidate character, a result of the correction is displayed on the third display area. When a “CONFIRM” key is input, the SAVE item and character data displayed on the third display area are stored.

Claims

1. An apparatus for recognizing a character image from a document, comprising:

an input unit for generating commands for a recognition mode, a correction mode and a storage mode;

a pre-processor for analyzing pixels of a document image in the recognition mode, classifying the document image into at least one Character Block (CB) and at least one Background Block (BB) on the basis of a result of the analysis, binarizing pixels of the BB and generating a pre-processed document image;

a character recognizer for recognizing the pre-processed document image and converting the recognized pre-processed document image into character data;

a recognition error processor for correcting or replacing erroneously recognized character data selected by the input unit with character data output by the input unit in the correction mode;

a database for storing the recognized character data in the storage mode; and

a display unit for displaying the document image and character data generated during operating of the modes.

2. The apparatus as set forth in claim 1, wherein the pre-processor comprises:

a skew correction part for classifying stripes having a preset length or above from the document image, calculating direction angles of the classified stripes, measuring a skew of an object, deciding a skew angle corresponding to the measured skew and correcting the object skew;

an Region Of Contents (ROC) extension part for classifying the document image in which the object skew is corrected into CBs and BBs, searching for positions of the CBs to extract the CBs and extending a size of an image of the extracted CBs to a size of an input document image; and

an image binarization part for comparing pixels of the CBs for the document image with a pixel threshold value, binarizing the pixels of the CBs into pixels having brightness values for character and background pixels, and binarizing pixels of the BBs into pixels having a brightness value for the background pixels.

3. The apparatus as set forth in claim 2, wherein the pre-processor further comprises:

a blurred-image detection part for classifying the input document image into the CBs and the BBs, calculating an average energy ratio for the CBs, comparing the average energy ratio with a predetermined threshold value, and determining whether the input document image is blurred according to a result of the comparison.

4. The apparatus as set forth in claim 2, wherein the pre-processor further comprises:

a noise reduction part for reducing noise of the extended image output from the ROC extension part and outputting, to the image binarization part, the image in which the noise is reduced.

5. The apparatus as set forth in claim 1, further comprising:

a camera for detecting the document and generating the document image.

6. The apparatus as set forth in claim 5, further comprising:

a speech recognizer for generating an input signal to select a SAVE item in the storage mode and generating another input signal to select and correct erroneously recognized character data in the correction mode, the speech recognizer converting an input speech signal into character data.

7. The apparatus as set forth in claim 5, wherein the character recognizer comprises:

a handwritten character recognizer for recognizing a received handwritten character image in the correction mode and converting the recognized handwritten character image into correction character data necessary for correcting the erroneously recognized character data.

8. The apparatus as set forth in claim 5, wherein the camera adjusts a focal distance and exposure time.

9. An apparatus for storing document information using a camera, comprising:

an input unit for generating commands for a pickup mode, a recognition mode, a correction mode and a storage mode;

a display unit having a first display area for displaying an input document image and character data recognized from the input document image, a second display area for displaying SAVE items, a third display area for displaying character data of a selected SAVE item, and another display area for displaying an operating mode menu;

the camera driven in the pickup mode for detecting the document image;

a pre-processor for analyzing pixels of the document image in the recognition mode, classifying the document image into at least one Character Block (CB) and at least one Background Block (BB) on the basis of a result of the analysis, binarizing pixels of the BB and generating a pre-processed document image;

a character recognizer for recognizing the pre-processed document image and converting the recognized pre-processed document image into character data;

a recognition error processor for correcting or replacing erroneously recognized character data selected by the input unit with character data output by the input unit in the correction mode; and

a database for storing the recognized character data in the storage mode.

10. The apparatus as set forth in claim 9, wherein the pre-processor comprises:

a skew correction part for classifying stripes having a preset length or above from the document image, calculating direction angles of the classified stripes, measuring a skew of an object according to a result of the calculation, deciding a skew angle corresponding to the measured skew and correcting the object skew;

an Region Of Contents (ROC) extension part for classifying the document image in which the object skew is corrected into CBs and BBs, searching for positions of the CBs to extract the CBs and extending a size of an image of the extracted CBs to a size of an input document image; and

an image binarization part for comparing pixels of the CBs for the document image with a pixel threshold value, binarizing the pixels of the CBs into pixels having brightness values for character and background pixels, and binarizing pixels of the BBs into pixels having a brightness value for the background pixels.

11. The apparatus as set forth in claim 10, wherein the pre-processor further comprises:

a blurred-image detection part for classifying the input document image into the CBs and the BBs, calculating an average energy ratio for the CBs, comparing the average energy ratio with a predetermined threshold value, and determining whether the input document image is blurred according to a result of the comparison.

12. The apparatus as set forth in claim 11, wherein the pre-processor further comprises:

a noise reduction part for reducing noise of the extended image output from the ROC extension part and outputting, to the image binarization part, the image in which the noise is reduced.

13. The apparatus as set forth in claim 12, further comprising:

a speech recognizer for generating an input signal to select a SAVE item in the storage mode and generating another input signal to select and correct erroneously recognized character data in the correction mode, the speech recognizer converting an input speech signal into character data.

14. The apparatus as set forth in claim 12, wherein the character recognizer comprises:

a handwritten character recognizer for recognizing a received handwritten character image in the correction mode and converting the recognized handwritten character image into correction character data necessary for correcting the erroneously recognized character data.

15. A method for enabling a terminal device to recognize a character image from a document image, comprising the steps of:

(a) designating an operating mode for document recognition;

(b) analyzing pixels of the document image in the document recognition mode, classifying the document image into at least one Character Block (CB) and at least one Background Block (BB) on the basis of a result of the analysis, binarizing pixels of the BB and generating a pre-processed document image;

(c) recognizing the pre-processed document image and converting the recognized pre-processed document image into character data;

(d) selecting erroneously recognized character data, and correcting or replacing the erroneously recognized character data with input character data in a correction mode; and

(e) storing the recognized character data in a storage mode.

16. The method as set forth in claim 15, wherein the step (b) comprises the steps of:

(b-1) classifying stripes having a preset length or above from the document image, calculating direction angles of the classified stripes, measuring a skew of an object according to a result of the calculation, deciding a skew angle corresponding to the measured skew and correcting the object skew;

(b-2) classifying the document image in which the object skew is corrected into CBs and BBs, searching for positions of the CBs to extract the CBs and extending a size of an image of the extracted CBs to a size of an input document image; and

(b-3) comparing pixels of the CBs for the document image with a pixel threshold value, binarizing the pixels of the CBs into pixels having brightness values for character and background pixels, and binarizing pixels of the BBs into pixels having a brightness value for the background pixels.

17. The method as set forth in claim 16, wherein the step (b) further comprises the step of:

(b-4) classifying the input document image into the CBs and the BBs, calculating an average energy ratio for the CBs, comparing the average energy ratio with a predetermined threshold value, determining whether the input document image is blurred according to a result of the comparison, and carrying out a pre-process if in the input image is not blurred.

18. The method as set forth in claim 17, wherein the step (b) further comprises the step of:

(b-5) reducing noise of the extended image output from a Region Of Contents (ROC) extension part and outputting, to the image binarization part, the image in which the noise is reduced.

19. The method as set forth in claim 18, wherein the step (d) comprises the steps of:

(d-1) displaying candidate characters corresponding to the erroneously recognized character data in the correction mode; and

(d-2) correcting or replacing the erroneously recognized character data with a candidate character selected from the displayed candidate characters.

20. The method as set forth in claim 18, wherein the step (d) comprises the steps of:

(d-3) displaying a recognition window for inputting a handwritten character in response to a request in the correction mode;

(d-4) recognizing the handwritten character when the handwritten character is input into the handwritten character recognition window; and

(d-5) correcting or replacing the erroneously recognized character data with the recognized handwritten character.

21. The method as set forth in claim 18, wherein the step (d) comprises the steps of:

(d-6) displaying candidate characters associated with the erroneously recognized character in the correction mode;

(d-7) correcting or replacing the erroneously recognized character data with a candidate character selected from the displayed candidate characters;

(d-8) displaying a handwritten character recognition window when correction character data is not contained in the displayed candidate characters;

(d-9) recognizing a handwritten character when the handwritten character is input into the handwritten character recognition window; and

(d-10) correcting or replacing the erroneously recognized character data with the recognized handwritten character.

22. The method as set forth in claim 18, wherein the step (d) comprises the steps of:

(d-11) driving a speech recognizer in the correction mode;

(d-12) allowing the speech recognizer to recognize an input speech signal and convert the recognized speech signal into the character data; and

(d-13) correcting or replacing the erroneously recognized character data with the character data provided from the speech recognizer.

23. A method for enabling a terminal device to recognize a character image from a document image, the terminal device including a display unit having a first display area for displaying a character image and character data, a second display area for displaying SAVE items, a third display area for displaying character data of a selected SAVE item, and another display area for displaying an operating mode menu, comprising the steps of:

(a) displaying the document image picked up by a camera;

(b) analyzing pixels of the document image in a document recognition mode, classifying the document image into at least one Character Block (CB) and at least one Background Block (BB) on the basis of a result of the analysis, binarizing pixels of the BB and generating a pre-processed document image;

(c) recognizing the pre-processed document image, converting the recognized pre-processed document image into character data, displaying the character data on the first display area, displaying SAVE items associated with the character data on the second display area, and displaying the pre-processed document image on the first display area;

(d) selecting a SAVE item from the displayed SAVE items, selecting character data associated with the selected SAVE item, and storing and displaying the selected character data; and

(e) repeating the steps (a) to (d) and storing selected SAVE items and character data corresponding to the selected SAVE items.

24. The method as set forth in claim 23, further comprising the step of:

(f) correcting erroneously recognized character data after the SAVE item and character data are selected,

wherein the step (f) comprises the steps of;

displaying candidate characters associated with the erroneously recognized character data on the third display area in response to an error correction request; and

correcting or replacing the erroneously recognized character data with a candidate character selected from the displayed candidate characters.

25. The method as set forth in claim 23, further comprising the step of:

(f) correcting erroneously recognized character data after the SAVE item and character data are selected,

wherein the step (f) comprises the steps of;

displaying a handwritten character recognition window on the second display area in response to an error correction request;

recognizing a handwritten character when the handwritten character is input into the handwritten character recognition window; and

correcting or replacing the erroneously recognized character data with the recognized handwritten character.

26. The method as set forth in claim 23, further comprising the step of:

(f) correcting erroneously recognized character data after the SAVE item and character data are selected,

wherein the step (f) comprises the steps of;

displaying candidate characters associated with the erroneously recognized character data on the third display area in response to an error correction request;

correcting or replacing the erroneously recognized character data with a candidate character selected from the displayed candidate characters;

displaying a handwritten character recognition window on the second display area when correction character data is not contained in the displayed candidate characters;

recognizing a handwritten character when the handwritten character is input into the recognition window; and

correcting or replacing the erroneously recognized character data with the recognized handwritten character.

27. The method as set forth in claim 23, further comprising the step of:

(f) correcting erroneously recognized character data after the SAVE item and character data are selected,

wherein the step (f) comprises the steps of;

driving a speech recognizer in response to an error correction request;

allowing the speech recognizer to recognize an input speech signal and convert the recognized speech signal into the character data; and

correcting or replacing the erroneously recognized character data with the character data provided from the speech recognizer.

28. A method for enabling a mobile terminal to recognize an image of a card bearing a person'"'"'s name and other information, the mobile terminal including a display unit having a first display area for displaying character data of the recognized card, a second display area for displaying SAVE items, a third display area for displaying character data of a selected SAVE item, and another display area for displaying an operating mode menu, comprising the steps of:

(a) displaying the card image picked up by a camera;

(b) analyzing pixels of the card image in a recognition mode, classifying the card image into at least one Character Block (CB) and at least one Background Block (BB) on the basis of a result of the analysis, binarizing pixels of the BB and generating a pre-processed card image;

(c) converting the pre-processed card image into character data, displaying the character data on the first display area, and displaying SAVE items of the card associated with the character data on the second display area;

(d) selecting a SAVE item from the displayed SAVE items, selecting character data associated with the selected SAVE item, and displaying the selected character data on the third display area;

(e) selecting erroneously recognized character data, and correcting or replacing the erroneously recognized character data with input character data in a correction mode; and

(f) storing the recognized character data in a storage mode.

29. The method as set forth in claim 28, wherein the SAVE items comprise a person'"'"'s name, a mobile phone number, a company phone number, an e-mail address, the person'"'"'s title and others.

30. The method as set forth in claim 29, wherein a process of carrying out the correction mode comprises the steps of:

displaying candidate characters associated with the erroneously recognized character data on the third display area in response to an error correction request; and

correcting or replacing the erroneously recognized character data with a candidate character selected from the displayed candidate characters.

31. The method as set forth in claim 29, wherein a process of correcting the erroneously recognized character data comprises the steps of:

displaying a handwritten character recognition window on the second display area when a correction key is input;

recognizing a handwritten character when the handwritten character is input into the handwritten character recognition window; and

correcting or replacing the erroneously recognized character data with the recognized handwritten character.

32. The method as set forth in claim 29, wherein a process of correcting the erroneously recognized character data comprises the steps of:

driving a speech recognizer when a correction key is input;

allowing the speech recognizer to recognize an input speech signal and convert the recognized speech signal into the character data; and

correcting or replacing the erroneously recognized character data with the character data provided from the speech recognizer.

33. The method as set forth in claim 29, wherein a process of correcting the erroneously recognized character data comprises the steps of:

displaying candidate characters associated with the erroneously recognized character data in the correction mode;

correcting or replacing the erroneously recognized character data with a candidate character selected from the displayed candidate characters;

displaying a handwritten character recognition window when correction character data is not contained in the displayed candidate characters;

recognizing a handwritten character when the handwritten character is input into the recognition window; and

correcting or replacing the erroneously recognized character data with the recognized handwritten character.