Automatic natural language translation of embedded text regions in images during information transfer

• US 8,031,943 B2
• Filed: 08/15/2008
• Issued: 10/04/2011
• Est. Priority Date: 06/05/2003
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Chen, Yen-Fu, Dunsmoir, John W.
• Primary Examiner(s): Strege; John B
• Application Number:
  US12/192,391
• Publication Number:
  US 20080300859A1
• Time in Patent Office: 1,145 Days
• Field of Search: 382/182, 715/259, 715/265, 715/271, 704/2, 704/3, 704/9
• US Class Current: 382/182
• CPC Class Codes: G06F 2209/542 Intercept
• CPC Class Codes: G06F 9/543 User-generated data transfer, e.g. clipboards, dynamic data exchange [DDE], object linking and embedding [OLE]

Abstract

Text regions of natural language embedded in images are automatically translated via a clipboard buffer during a cut-and-paste command according to an isolation rule defining one or more types of image elements to be isolated, by examining the image elements, responsive to meeting the isolation rule, transferring the image element from the clipboard to an isolation store, disabling pasting of the image element; recognizing text within the isolated image; determining a natural language in which the recognized text is expressed; performing natural language translation of the recognized text into a user-specified destination natural language; producing image data for the translated text; overlaying the image data onto the embedded text in the isolated image element; and pasting the modified image element from the isolation store to a user-designated destination points within a destination computer file.

Claims

1. A system for automatic natural language translation of embedded text regions in images during information transfer, the system comprising:

a computing platform having hardware for performing a logical process;

a clipboard buffer portion of the computing platform configured to receive a copy of one or more user-selected image elements from within a source computer file responsive to user operation of a cut-and-paste command;

an isolator and interceptor portion of the computing platform configured to;

receive a user-supplied isolation rule defining one or more types of image elements to be isolated, examine the one or more image elements in the clipboard buffer against the isolation rule,

responsive to the one or more image elements meeting the isolation rule, transfer the one or more image elements from the clipboard buffer portion to an isolation store, and

disable pasting of the one or more image elements into a destination computer file;

a recognizer portion of the computing platform configured to recognize text from within the one or more image elements in the isolation store;

a source language identifier portion of the computing platform configured to determine a natural language in which the recognized text is expressed;

a preference comparitor portion of a computing platform configured to determine if the natural language of the recognized text matches a user-specified destination natural language;

a translator portion of a computing platform configured to perform natural language translation of the recognized text into the user-specified destination natural language;

a text image renderer portion of a computing platform configured to produce image data for translated text;

an image modifier portion of the computing platform configured to overlay text within the one or more image elements stored in the isolation store with the image data; and

a copier portion of the computing platform configured to complete pasting of the one or more image elements from the isolation store to one or more user-designated destination points within the destination computer file.

2. The system as set forth in claim 1 wherein the recognizer portion includes an Optical Character Recognizer configured to recognize regions of embedded text, and to generate text data.

3. The system as set forth in claim 1 wherein the recognizer portion comprises a Text String Extractor configured to extract strings of encoded text.

4. The system as set forth in claim 1 wherein the source language identifier portion is configured to search for each recognized character in a system alphabet corresponding to the user-specified destination natural language.

5. The system as set forth in claim 1 wherein the source language identifier portion is configured to search for each recognized word in a system dictionary corresponding to the user-specified destination natural language.

6. The system as set forth in claim 2 wherein the translator portion is configured to perform operations comprising:

selecting an appropriate translation resource corresponding to the identified source natural

language and the user-specified destination natural language, and invoking the selected translation resource.

7. The system as set forth in claim 6 wherein the source language identifier portion is configured to search for characters in one or more system alphabets, each system alphabet corresponding to a source natural language.

8. The system as set forth in claim 6 wherein the source language identifier portion is configured to search for words in one or more system dictionaries, each system dictionary corresponding to a source natural language.

9. A method for automatic natural language translation of embedded text regions in images during information transfer, the method comprising:

a computer receiving in a clipboard buffer a

copy of one or more user-selected image elements from within a source computer file responsive to user operation of a cut-and-paste command;

the computer receiving a user-supplied isolation rule defining one or more types of image elements to be isolated;

the computer examining the one or more image elements in the clipboard buffer against the isolation rule;

the computer, responsive to the one or more image elements meeting the isolation rule, transferring the one or more image elements from the clipboard buffer to an isolation store, and disabling pasting of the one or more image elements into a destination computer file;

the computer automatically recognizing text within the one or more image elements in the isolation store;

the computer identifying a source language by determining a natural language in which the recognized text is expressed;

the computer comparing a user preference by determining if the natural language of the recognized text matches a user-specified destination natural language;

the computer performing automatic natural language translation of the recognized text into the user-specified destination natural language;

the computer rendering text image data for the translated text;

the computer overlaying the text within the one or more image elements stored in the isolation store with the rendered image data; and

the computer pasting of the one or more image elements from the isolation store to one or more user-designated destination points within the destination computer file.

10. The method as set forth in claim 9 wherein the automatic recognizing comprises performing Optical Character Recognition on one or more regions of embedded text to generate text data.

11. The method as set forth in claim 9 wherein the Optical Character Recognition said comprises extracting strings of encoded text.

12. The method as set forth in claim 9 wherein the identifying of the source language comprises searching for each recognized character in a system alphabet corresponding to the user-specified destination natural language.

13. The method as set forth in claim 9 wherein the identifying of the source language comprises searching for each recognized word in a system dictionary corresponding to the destination natural language.

14. The method as set forth in claim 9 wherein the automatic translation comprises the steps of:

selecting an appropriate translation resource corresponding to the identified source natural language and the destination natural language; and

invoking the selected translation resource to perform translation.

15. A computer program product for automatic natural language translation of embedded text regions in images during information transfer comprising:

one or more computer-readable, tangible storage devices;

program instructions, stored on at least one of the one or more storage devices, to receive by a clipboard buffer portion of a computing platform configured a copy of one or more user-selected image elements from within a source computer file responsive to user operation of a cut-and-paste command;

program instructions, stored on at least one of the one or more storage devices, to receive a user-supplied isolation rule defining one or more types of image elements to be isolated;

program instructions, stored on at least one of the one or more storage devices, to examine the one or more image elements said in the clipboard buffer against the isolation rule;

program instructions, stored on at least one of the one or more storage devices, to, responsive to one or more image elements meeting the isolation rule, transferring the one or more isolation elements from the clipboard buffer to an isolation store, and to disable pasting of the image element into a destination computer file;

program instructions, stored on at least one of the one or more storage devices, to automatically recognize text within the one or more image elements in the isolation store;

program instructions, stored on at least one of the one or more storage devices, to identify a source language by determining a natural language in which the recognized text is expressed;

program instructions, stored on at least one of the one or more storage devices, to compare a user preference by determining if the natural language of the recognized text matches a user-specified destination natural language;

program instructions, stored on at least one of the one or more storage devices, to perform automatic natural language translation of the recognized text into the user-specified destination natural language;

program instructions, stored on at least one of the one or more storage devices, to render image data for the translated text;

program instructions, stored on at least one of the one or more storage devices, to overlay text within the one or more image elements stored in the isolation store with the rendered image data; and

program instructions, stored on at least one of the one or more storage devices, to paste the one or more image elements from the image store to one or more user-designated destination points within the destination computer file.

16. The computer program product as set forth in claim 15 wherein the program instructions to recognize text comprises program instructions for Optical Character Recognition on one or more regions of embedded text to generate text data.

17. The computer program product as set forth in claim 15 wherein the program instructions to recognize text comprises program instructions to extract strings of encoded text.

18. The computer program product as set forth in claim 15 wherein the program instructions to identify the source language comprises program instructions to search for a character in a system alphabet corresponding to the destination natural language.

19. The computer program product as set forth in claim 15 wherein the program instructions to identify the source language comprises program instructions to search for a word in a system dictionary corresponding to the destination natural language.

20. The computer program product as set forth in claim 15 wherein the program instructions for automatic translation comprises:

program instructions to select an appropriate translation resource corresponding to the identified source natural language and the destination natural language; and

program instructions to invoke the selected translation resource to perform translation.