System and Method for Automatic Natural Language Translation of Embedded Text Regions in Images During Information Transfer

• US 20080300859A1
• Filed: 08/15/2008
• Published: 12/04/2008
• Est. Priority Date: 06/05/2003
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Chen, Yen-Fu, Dunsmoir, John W.
• Granted Patent: US 8,031,943 B2
• Time in Patent Office: Days
• US Class Current: 704/7
• CPC Class Codes: G06F 2209/542 Intercept
• CPC Class Codes: G06F 9/543 User-generated data transfer, e.g. clipboards, dynamic data exchange [DDE], object linking and embedding [OLE]

Abstract

Image or graphic elements such as pictures, icons and clip art, are intercepted during transfer from a computer source to a computer destination, such as during a cut-and-paste operation. Embedded text within the intercepted element is optically recognized, and encoded text is extracted. It is determined if the language of the text matches a desired language of the destination. If not, a translation of the optically-recognized text data is performed, and the translated text data is rendered into image data. The image data is then substituted into the intercepted element to overlay or replace the original embedded text, thereby “translating” the image or graphic element to a desired destination language. The modified image or graphic element is then delivered to the destination, such as a document, program, file or data structure.

Claims

1. A system comprising:

a clipboard buffer portion of a computing platform configured to receive a copy of a user-selected image elements from within a source computer file upon operation of a user cut-and-paste command;

an isolator and interceptor configured to receive a user-supplied isolation rule defining one or more types of image elements to be isolated, examine said copied image element in said clipboard buffer, responsive to said image element meeting said isolation rule, transfer said image element from said clipboard buffer to an isolation store, and disable pasting of said image element into a destination computer file;

a recognizer configured to recognize images of text within said image element stored in said isolation store;

a source language identifier configured to determine a natural language in which said recognized text is expressed;

a preference comparitor configured to determine if said natural language of said recognized text matches a user-specified destination natural language;

a translator configured to perform natural language translation of said recognized text into said user-specified destination natural language;

a text image renderer configured to produce computer-readable translated text image data for said translated text;

an image modifier configured to replace text within said image element stored in said isolation store with said translated text image data; and

a copier configured to complete pasting of said image element having said replaced text to one or more user-designated destination points within said destination computer file.

2. The system as set forth in claim 1 wherein said recognizer includes an Optical Character Recognizer configured to recognize regions of embedded text, and to generate text data.

3. The system as set forth in claim 1 wherein said recognizer comprises a Text String Extractor configured to extract strings of encoded text.

4. The system as set forth in claim 1 wherein said source language identifier is configured to search for each recognized character in a system alphabet corresponding to said destination natural language.

5. The system as set forth in claim 1 wherein said source language identifier is configured to search for each recognized word in a system dictionary corresponding to said destination natural language.

6. The system as set forth in claim 1 wherein said translator is configured to perform operations comprising:

selecting an appropriate translation resource corresponding to said identified source natural language and said destination natural language, and

invoking said selected translation resource to perform translation of the recognized text for said source embedded text region.

7. The system as set forth in claim 6 wherein said source language identifier is configured to search for each recognized character in one or more system alphabets, each system alphabet corresponding to a source natural language.

8. The system as set forth in claim 6 wherein said source language identifier is configured to search for each recognized word in one or more system dictionaries, each system dictionary corresponding to a source natural language.

9. A method comprising:

receiving by a clipboard buffer portion of a computing platform configured a copy of a user-selected image elements from within a source computer file upon operation of a user cut-and-paste command;

isolating and intercepting said user-selected image elements by receiving a user-supplied isolation rule defining one or more types of image elements to be isolated, examining said copied image element in said clipboard buffer, responsive to said image element meeting said isolation rule, transferring said image element from said clipboard buffer to an isolation store, and disabling pasting of said image element into a destination computer file;

automatically recognizing images of text within said image element stored in said isolation store;

identifying a source language by determining a natural language in which said recognized text is expressed;

comparing a user preference by determining if said natural language of said recognized text matches a user-specified destination natural language;

performing automatic natural language translation of said recognized text into said user-specified destination natural language;

rendering text image by producing computer-readable translated text image data for said translated text;

replacing said text within said image element stored in said isolation store with said translated text image data; and

completing pasting of said image element having said replaced text to one or more user-designated destination points within said destination computer file.

10. The method as set forth in claim 9 wherein said automatic recognizing includes performing Optical Character Recognition on one or more regions of embedded text to generate text data.

11. The method as set forth in claim 9 wherein said recognition comprises extracting strings of encoded text.

12. The method as set forth in claim 9 wherein said identifying of a source language comprises searching for each recognized character in a system alphabet corresponding to said destination natural language.

13. The method as set forth in claim 9 wherein said identifying of a source language comprises searching for each recognized word in a system dictionary corresponding to said destination natural language.

14. The method as set forth in claim 9 wherein said automatic translation comprises:

selecting an appropriate translation resource corresponding to said identified source natural language and said destination natural language; and

invoking said selected translation resource to perform translation of the recognized text for said source embedded text region.

15. A computer readable medium comprising:

a computer memory suitable for encoding software programs, and one or more software programs stored therein and configured to cause a processor to perform operations comprising;

receiving by a clipboard buffer portion of a computing platform configured a copy of a user-selected image elements from within a source computer file upon operation of a user cut-and-paste command;

isolating and intercepting said user-selected image elements by receiving a user-supplied isolation rule defining one or more types of image elements to be isolated, examining said copied image element in said clipboard buffer, responsive to said image element meeting said isolation rule, transferring said image element from said clipboard buffer to an isolation store, and disabling pasting of said image element into a destination computer file;

automatically recognizing images of text within said image element stored in said isolation store;

identifying a source language by determining a natural language in which said recognized text is expressed;

comparing a user preference by determining if said natural language of said recognized text matches a user-specified destination natural language;

performing automatic natural language translation of said recognized text into said user-specified destination natural language;

rendering text image by producing computer-readable translated text image data for said translated text;

replacing said text within said image element stored in said isolation store with said translated text image data; and

completing pasting of said image element having said replaced text to one or more user-designated destination points within said destination computer file.

16. The medium as set forth in claim 15 wherein said automatic recognizing includes performing Optical Character Recognition on one or more regions of embedded text to generate text data.

17. The medium as set forth in claim 15 wherein said recognition comprises extracting strings of encoded text.

18. The medium as set forth in claim 15 wherein said identifying of a source language comprises searching for each recognized character in a system alphabet corresponding to said destination natural language.

19. The medium as set forth in claim 15 wherein said identifying of a source language comprises searching for each recognized word in a system dictionary corresponding to said destination natural language.

20. The medium as set forth in claim 15 wherein said automatic translation comprises:

selecting an appropriate translation resource corresponding to said identified source natural language and said destination natural language; and

invoking said selected translation resource to perform translation of the recognized text for said source embedded text region.