Photography recognition translation

• US 9,519,641 B2
• Filed: 10/15/2012
• Issued: 12/13/2016
• Est. Priority Date: 09/18/2012
• Status: Expired due to Fees

• Current Assignee: ABBYY Production LLC (ABBYY Software)
• Original Assignee: ABBYY Development LLC
• Inventors: Solntseva, Ekaterina, Tarachyov, Konstantin
• Primary Examiner(s): BAKER, MATTHEW H
• Application Number:
  US13/651,866
• Publication Number:
  US 20140081619A1
• Time in Patent Office: 1,520 Days
• Field of Search: 704 2- 7
• US Class Current: 1/1
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

Methods are described for efficient and substantially instant recognition and translation of text in photographs. A user is able to select an area of interest for subsequent processing. Optical character recognition (OCR) may be performed on the wider area than that selected for determining the subject domain of the text. Translation to one or more target languages is performed. Manual corrections may be made at various stages of processing. Variations of translation are presented and made available for substitution of a word or expression in the target language. Translated text is made available for further uses or for immediate access.

Claims

1. A method for translating text in an electronic image, the method comprising:

acquiring, by an electronic device, access to the electronic image;

receiving, by the electronic device, user input providing an indication of an area of interest in the electronic image;

recognizing characters in the received area of interest and characters an extended area beyond the area of interest;

determining a subject domain associated with the recognized characters in the area of interest based at least in part on the recognized characters in the extended area;

translating the recognized characters in the identified area of interest from a source language into a target language at least in part based on the determined subject domain; and

displaying the translated characters in the target language on a display of the electronic device.

2. The method of claim 1, wherein the recognized characters in the area of interest represent a word, word combination, text or hieroglyph.

3. The method of claim 1, comprising:

after recognizing characters in the area of interest, displaying the recognized characters on the display of the electronic device; and

before translating the recognized characters into the target language, receiving an indication of translating the recognized characters.

4. The method of claim 1, further comprising translating a sentence or a paragraph overlapping with the area of interest.

5. The method of claim 1, further comprising translating all text in the electronic image.

6. The method of claim 1, further comprising acquiring the electronic image through a camera of the electronic device.

7. The method of claim 6, wherein acquiring access to the electronic image by the electronic device is through an interaction with a control of the camera of the electronic device.

8. The method of claim 1, further comprising:

after acquiring access to the electronic image, displaying at least a portion of the electronic image on the display of the electronic device, wherein receiving the indication of the area of interest includes receiving the indication through the displayed electronic image.

9. The method of claim 1, further comprising:

after recognizing the characters in the electronic image and prior to translating the recognized characters, displaying a result of optical character recognition on the display of the electronic device; and

prior to translating the recognized characters, receiving an indication corresponding to a correction to one or more of the recognized characters.

10. The method of claim 9, further comprising:

after receiving the indication corresponding to the correction to one or more of the recognized characters, making the correction to the one or more of the recognized characters, and displaying the corrected recognized characters.

11. The method of claim 9, wherein the correction of the recognized characters-is done by automatic insertion of a correct variant in the recognized characters by the electronic device, or by manual input of the correct variant from a keyboard.

12. The method of claim 1, wherein the extended area comprises a sentence, a paragraph, or a section of a document overlapping with the area of interest.

13. The method of claim 1, wherein the subject domain is further based on a history of translations or a history of corrections of translation of other images accessed by the electronic device.

14. The method of claim 1, wherein the subject domain for translation is further based on data content resident on the electronic device.

15. The method of claim 1, wherein the subject domain for translation is further based on geolocation data.

16. The method of claim 1, further comprising determining a language for recognizing the characters in the electronic image based on geolocation data.

17. The method of claim 16, wherein the determination of the language for recognizing the characters further comprises:

establishing coordinates of location of the electronic device by a navigation module;

searching and acquiring from a database a country or region based on the coordinates of location;

searching and determining from the database a list of one or more languages that are used in the country or region; and

making the list of one or more languages available for use in recognizing the characters in the electronic image.

18. The method of claim 1, wherein the source language is determined based on geolocation data.

19. The method of claim 1, wherein translating the recognized characters is performed by dictionary translation, or machine translation, or a human-rendered translation.

20. The method of claim 1, further comprising:

after displaying the translated characters, correcting a portion of the translated characters in response to detecting an indication of a change for at least one character from the portion of the translated characters.

21. The method of claim 1, wherein the extended area is selected by a user of the electronic device.

22. The method of claim 1, wherein the extended area is a picture in the electronic image and the determining of the subject domain is at least in part based on said picture.

23. The method of claim 1, wherein the displaying the translated characters in the target language on the display of the electronic device further comprises replacing the recognized characters with the translated characters and preserving a background of the area of interest.

34. The method of claim 1 further comprising displaying alternative variants of translation of the recognized characters in the recognized area of interest.

24. An electronic device comprising:

a display;

a processor;

a memory in electronic communication with the processor, the memory configured with instructions for performing a method by the processor, the method comprising;

receiving, by the electronic device, user input providing an indication of an area of interest in an electronic image;

recognizing characters in the received area of interest and characters an extended area beyond the area of interest;

determining a subject domain associated with the recognized characters in the area of interest based at least in part on the recognized characters in the extended area;

translating the recognized characters in the identified area of interest from a source language into a target language at least in part based on the determined subject domain; and

displaying the translated characters in the target language on the display of the electronic device.

25. The electronic device of claim 24, wherein the method further comprises:

after recognizing characters in the area of interest, displaying the recognized characters on the display of the electronic device; and

before translating the recognized characters into the target language, receiving an indication of translating the recognized characters.

26. The electronic device of claim 24, wherein the method further comprising translating a sentence or a paragraph overlapping with the area of interest.

27. The electronic device of claim 26, wherein the method further comprising translating all text in the electronic image.

28. The electronic device of claim 24, wherein the electronic device further includes a camera, and wherein the method further comprises capturing the electronic image through the camera.

29. The electronic device of claim 24, wherein the method further comprises:

before receiving by the electronic device the indication of the area of interest in the electronic image, displaying at least a portion of the electronic image on the display, wherein receiving the indication of the area of interest includes receiving the indication through the displayed image.

30. The electronic device of claim 24, wherein the method further comprises:

after recognizing the characters in the electronic image and prior to translating the recognized characters, displaying a result of optical character recognition on the display of electronic device; and

prior to translating the recognized characters, displaying an indication of a possible error associated with one of the recognized characters, and receiving by the electronic device an indication corresponding to a correction to the recognized character of the possible error.

31. The electronic device of claim 30, wherein displaying the indication of the possible error includes displaying one or more variants of recognition to correct the possible error, and wherein the method further comprises, in response to receiving the indication corresponding to the correction, insertion of a correct variant in the recognized characters corresponding to the indication.

32. The electronic device of claim 24, wherein the subject domain is further based on a history of translations or a history of corrections of translation of other images accessed by the electronic device.

35. The electronic device of claim 24 further comprising displaying alternative variants of translation of the recognized characters in the recognized area of interest.

33. A non-transitory computer-readable medium storing instructions which, when executed by an electronic device, cause the electronic device to:

acquire access to an electronic image;

receive user input providing an indication of an area of interest in the electronic image;

recognize characters in the received area of interest and characters in an extended area beyond the area of interest;

determine a subject domain associated with the recognized characters in the area of interest based at least in part on the recognized characters in the extended area;

translate the recognized characters in the identified area of interest from a source language into a target language based on the determined subject domain; and

display the translated characters in the target language.

36. The non-transitory computer-readable medium of claim 33 further comprising displaying alternative variants of translation of the recognized characters in the recognized area of interest.