METHOD FOR RECOGNIZING AND TRANSLATING CHARACTERS IN CAMERA-BASED IMAGE

• US 20100008582A1
• Filed: 07/09/2009
• Published: 01/14/2010
• Est. Priority Date: 07/10/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Hwang, Seong-Taek, Oh, Sang-Wook, KIM, Sang-Ho, Kim, Hyun-Soo, Kim, Jung-Rim, Kim, Ji-Hoon, Oh, Yun-Je, Jung, Hee-Won, Lee, Dong-Chang
• Granted Patent: US 8,625,899 B2
• Time in Patent Office: Days
• US Class Current: 382/177
• CPC Class Codes: G06F 40/53 Processing of non-Latin text kana-to-kanji conversion G06F40/129; vowelisation G06F40/232
• CPC Class Codes: G06V 10/17 using hand-held instruments
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments

Abstract

A method for recognizing an image photographed by a camera and translating characters in connection with an electronic dictionary is provided. The method includes directly selecting an area to be recognized from the photographed character image and performing character recognition, translating and recognizing characters of a user'"'"'s selected word in connection with dictionary data, and displaying translation result information of user'"'"'s selected character or word in connection with dictionary data on a screen device. The recognition includes providing information on location of the selected character image area and location of the recognized character string words to the user, and then translating a character string or word in a location area selected by the user. The electronic dictionary-connected search and translation is for searching the character or word selected in connection with the electronic dictionary database, and providing translation result to the user.

Claims

1. A method for recognizing a character string or a word included in a character image photographed by a camera, and translating a recognized specific character string or word to another language by using a built-in electronic dictionary DataBase (DB), the method comprising:

recognizing the character string or word in the photographed character image, wherein a character string or words included in a character image area selected by a user is recognized and the recognized character string is post-processed; and

performing an electronic dictionary-connected search and translation, wherein a character or a word selected by the user, from among the words included in the character string which has been recognized and post-processed, is searched in connection with the electronic dictionary DB, and translation result information of the searched word is displayed on a screen device;

wherein recognizing the character string or word comprises user interfacing, in which information on location of the selected character image area and location of the recognized character string words is provided to the user, and then a character string or word in a location area selected by the user is to be translated, and

wherein performing the electronic dictionary-connected search and translation is for searching the character or word selected in the user interfacing step in connection with the electronic dictionary DB, and providing a translation result to the user through the user interfacing.

2. The method as claimed in claim 1, wherein user interfacing comprises:

(a) directly displaying, in selecting a recognition object area range for character image recognition after displaying the character image photographed by the camera on the screen device, location of a recognition area, which is being selected or has been selected, over the character image displayed on the screen device, so as to allow the user to instinctively verify a selection step and result;

(b) directly displaying relative location information of the recognized words over the character image so as to determine whether the word that the user wants to translate, is recognized, from a corresponding character recognition result of a recognized character image area; and

(c) directly displaying a word at a location selected by the user, from among a plurality of recognized words included in the recognized character image area, and corresponding translation information searched from the electronic dictionary DB, over the character image displayed on the screen device,

wherein user interfacing provides a connection function of the character image with the electronic dictionary DB.

3. The method as claimed in claim 2, wherein user interfacing further comprises:

(d) determining whether another character or word other than the user'"'"'s first selected character or word is selected, and then returning to step (c) if the user selects another character or word; and

(e) receiving, from a user, input of a determination, if the user does not select the recognized character or word, of whether to return to the step (a).

4. The method as claimed in claim 2, wherein in step (a), when the user selects an entire photographed image as a character recognition area, a recognition and translation of the character or word is performed.

5. The method as claimed in claim 2, wherein post-processing the recognized character string comprises:

filtering out special characters or numbers from the recognized character string;

constructing a word from recognized individual characters, and searching connected dictionary data for the constructed word to verify a dictionary meaning; and

providing the recognized character or word to the user via step (c).

6. A method for recognizing and translating a character string or words included in a character image obtained by camera-photographing a subject comprising a character, and providing a result to a user via a screen device, the method comprising the steps of:

(a) providing a user interface, wherein the user directly selects an area to be recognized in the photographed character image, and can in real-time confirm information of the selected area;

(b) recognizing characters and storing recognized characters included in the area selected by the user;

(c) post-processing, from among characters or character strings recognized in step (b), a character string, which exists on a location comprising a character or a word located at the user'"'"'s selected area in the character image displayed on the screen device, by filtering out special characters or numbers, and then separating the character string into selectable words by using words in an electronic dictionary DataBase (DB) and temporarily storing result information;

(d) searching for, from words temporarily stored in step (c), a word located in the user'"'"'s first selected area in connection with the electronic dictionary DB, and verifying a result; and

(e) providing a translation result of the word verified in step (d) to the user.

7. The method as claimed in claim 6, wherein step (a) is performed by using a drag of a touch screen, or a pointing device such as a mouse.

8. The method as claimed in claim 6, wherein step (b) includes recognizing the characters included in the user'"'"'s selected area, and then storing a recognition result as individual characters and location information of the individual characters, words comprising the individual characters and location information of the words, lines comprising the words and location information of the lines, blocks comprising a series of lines and location information of the blocks, and a series of blocks and location information of the series of blocks.

9. The method as claimed in claim 6, wherein step (d) includes providing location information of the user'"'"'s selected word with respect to the character image to the user, which is overlapped on the screen device providing the character image in a pre-recognition state, with a highlighting marker wrapping a corresponding word, and

wherein when the user selects only a part of the entire character image, location information of a word or characters included in location information of only an area selected by the user is provided.

10. The method as claimed in claim 6, wherein in step (d) if the user does not accurately select location of a word to be actually translated, a distances between the user'"'"'s selected location and locations of recognized words are calculated, and a word located closest to the user'"'"'s selected location is translated in connection with the electronic dictionary.

11. The method as claimed in claim 6, wherein when a language to be translated has no space between words, step (c) comprises:

loading data of all character strings stored after recognizing characters of the character image;

filtering out numbers or special characters included in the loaded character strings;

comparing the filtered character strings with the electronic dictionary DB, and separating searched words; and

searching for words from among the separated words that are directly selected by the user via the screen device, in connection with the electronic dictionary DB.

12. The method as claimed in claim 6, wherein in step (e), the translation result is provided in a semi-transparent type allowing other image information to be shown to the user, and if the result overlaps a previous search result, the search result is provided to the user via the screen device in such a manner that the result does not overlap the previous search result.

13. The method as claimed in claim 6, further comprising:

(f) operating the electronic dictionary DB after step (e);

(g) determining whether to return to the screen device providing the translation result from the state where the electronic dictionary DB is operating in step (f);

(h) determining whether another word to be translated is selected;

(i) determining whether another area to be recognized in the character image is selected, if another word to be translated is not selected in step (h); and

(j) determining whether to recognize another character image if another unrecognized area in the character image is not selected in step (i),

wherein in step (f), a means with which the user directly operates the electronic dictionary DB is provided to the user via the screen device, and then when the operating means is selected, an operating state of the electronic dictionary DB is displayed on the screen device.

14. The method as claimed in claim 6, wherein step (d) comprises:

sequentially, after post-processing the character string at the user'"'"'s selected location, searching for a first character to a last character of the character string in connection with an entry word or keyword of the electronic dictionary DB built in a portable terminal, and then obtaining candidate words to be selectable as a final electronic dictionary DB search word and separately storing the candidate words; and

selecting, from among a plurality of the stored candidate words, a candidate word comprising location information corresponding to the user'"'"'s first selected location, as the final electronic dictionary DB search word.

15. The method as claimed in claim 14, wherein in a character string of languages having no space between individual words, constructing an electronic dictionary DB search word for use in a search comprises:

selecting a maximum length of a candidate character string to be firstly used for searching the electronic dictionary DB, while taking frequencies of lengths of words of a corresponding language into consideration; and

updating, when the candidate character string of the electronic dictionary DB search word does not exist in the electronic dictionary DB, the candidate character string of the electronic dictionary DB search word by reducing a length of the candidate character string and correspondingly reducing a number of characters included in the included character string.

16. The method as claimed in claim 11, wherein the language is one of Chinese and Japanese.

17. The method as claimed in claim 15, wherein the language is one of Chinese and Japanese.