LANGUAGE PROCESSING RESOURCES FOR AUTOMATED MOBILE LANGUAGE TRANSLATION

• US 20140172407A1
• Filed: 12/14/2012
• Published: 06/19/2014
• Est. Priority Date: 12/14/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Quirk, Christopher, Aue, Anthony, Schaffalitzky, Frederik, Eden, Ibrahim, Galley, Michel
• Granted Patent: US 9,183,197 B2
• Time in Patent Office: Days
• US Class Current: 704/2
• CPC Class Codes: G06F 40/42 Data-driven translation

Abstract

Automated language translation often involves language translation resources of significant size (e.g., 50-gigabyte phrase tables) and significant computational power exceeding the capabilities of many mobile devices. Remotely accessible servers capable of near-realtime, automated translation may be inaccessible or prohibitively costly while traveling abroad. Presented herein are adaptations of language translation techniques for offline mobile devices involving reducing the size and raising the efficiency of the language modeling resources. A word index may be provided that stores respective string representations of the words of a language, and maps respective words to a location (e.g., address or offset) of respective word representations within the word index. Language translation resources (e.g., phrase tables) may then specify logical relationships using the word index addresses of the involved words, rather than the string equivalents. This technique significantly condenses the language resources and provides faster, bidirectional access to the word representations of the language.

Claims

1. A method of representing a language comprising at least two words and at least one translation of a word sequence on a device having a processor and a memory, the method comprising:

executing on the processor instructions configured to;

store in the memory a word index comprising, for respective words of the language;

the word stored at an index location in the word index, and

a word mapping that identifies the index location of the word in the word index; and

store in the memory a translation mapping identifying, for a word index sequence comprising at least one index location, the translation of the words respectively located at the index locations of the word index.

2. The method of claim 1, the translation mapping comprising:

a phrase table mapping respective word sequences to a word sequence in the language; and

a target language model mapping respective word sequences to a translation in a language.

3. The method of claim 2, the instructions further configured to, for respective word sequences:

identify a mapping probability of the word sequence to the translation; and

store the mapping probability of the word sequence in the target language model.

4. The method of claim 3:

the instructions further configured to store in the target language model a mapping probability table identifying, for respective mapping probability integers, a mapping probability floating point; and

identifying the mapping probability further comprising;

using the mapping probability table, translating, for the translation of the word sequence, the mapping probability floating point into a mapping probability integer; and

including the mapping probability integer of the translation in the target language model.

5. The method of claim 4, the instructions further configured to:

cluster the mapping probability floating points into mapping probability clusters; and

for respective clusters, select the mapping probability integer for the word sequences mapped into the target language model.

6. The method of claim 1, the translation language selected from a language translation set comprising:

a translation between a first natural language and a second natural language;

a translation between a first dialect of a language and a second dialect of the language;

a translation between a first language modality of a natural language and a second language modality of the natural language;

a translation between a colloquial version of a natural language and a standardized version of the natural language; and

a translation between a technical language and a natural language.

7. The method of claim 1:

the device comprising a word mapping function configured to, for respective words, identify a word mapping value; and

storing the word mapping of the word index comprising;

for respective words, using the word mapping function, compute the word mapping value; and

store the word mapping as an association of the word mapping function and the word mapping location.

8. The method of claim 1:

the language comprising at least two language domains respectively comprising a subset of the words of the language;

storing the word index comprising;

for respective language domains, storing on the device a word index for the words of the language domain of the language; and

storing the translation mapping comprising;

for respective language domains, storing on the device translation mapping for the words of the language domain of the language.

9. The method of claim 1, the index location comprising, at the index location for the word:

a word size of a string representation of the word stored at the index location; and

following the word size, the string representation.

10. A method of translating a word sequence from a source language to a target language on a device having a processor and a memory comprising a source word index for the source language, a target word index for the target language, and a translation mapping, the method comprising:

executing on the processor instructions configured to;

for respective words of the word sequence, identify the source index location of the word in the source word index;

using the translation mapping, identify a translation of the source index locations of the words of the word sequence, the translation comprising at least one target index location in the target word index;

for respective target index locations, retrieve a translated word in the target language at the target index location in the target word index; and

present the translated words in the target language.

11. The method of claim 10:

the translation mapping comprising;

a phrase table mapping respective word sequences in the source language to at least one candidate translation in the target language, and

a translation model identifying, for respective candidate translations, a mapping probability of the candidate translation in the target language; and

identifying the translation comprising;

using the phrase table, mapping the words of the word sequence into at least one candidate translations; and

using the translation model, selecting among the at least one candidate translation a translation in the target language.

12. The method of claim 11:

the translation model identifying, for respective word sequences, a mapping probability of the word sequence to the translation; and

selecting among the at least one candidate translation comprising;

selecting the candidate translation having a highest mapping probability among the candidate translations.

13. The method of claim 12:

the instructions further configured to store in the translation model a mapping probability table identifying, for respective translations, a mapping probability integer; and

selecting the candidate translation comprising;

reading from the translation model a mapping probability integer;

using the mapping probability table, translating, for the translation, the mapping probability integer into a mapping probability floating point; and

identifying the mapping probability of the translation in the translation model using the mapping probability floating point.

14. The method of claim 10:

the device comprising a word mapping function configured to, for respective words in the source language, identify a word mapping value;

the source word index comprising a word mapping index identifying, for respective word mapping values, the source index location of the word in the source word index; and

identifying the source index location of the word in the source word index comprising;

using the word mapping function, identifying the word mapping value of the word; and

using the source word index, identifying the source index location of the word in the source word index.

15. The method of claim 10:

the target word index storing at the target index location a word header comprising;

a word size of a string representation of the translated word stored at the target index location, and

following the word size, the string representation; and

retrieving the translated word from the target word index comprising;

reading from the word header, at the target index location, the word size; and

following the word size, reading the string representation of the word size for the translated word.

16. The method of claim 10, comprising:

generating a word index cache storing at least one chunk of a word index; and

the instructions further configured to;

upon accessing an index location in the word index;

determine whether the index location is within a chunk stored in the word index cache;

upon determining that the index location is within a chunk stored in the word index cache, access the index location in the chunk in the word index cache; and

upon determining that the index location is not within a chunk stored in the word index cache;

read a chunk of the word index including the index location;

access the index location in the chunk; and

store the chunk in the word index cache.

17. The method of claim 10, comprising:

generating a translation mapping cache storing at least one chunk of the translation mapping; and

the instructions further configured to;

upon accessing a word sequence in the translation index;

determine whether the word sequence is within a chunk stored in the translation mapping cache;

upon determining that the word sequence is within a chunk stored in the translation mapping cache, access the word sequence in the chunk in the translation mapping cache; and

upon determining that the word sequence is not within a chunk stored in the translation mapping cache;

read a chunk of the translation mapping including the word sequence;

access the word sequence in the translation mapping; and

store the chunk in the translation mapping cache.

18. The method of claim 10:

the device having access to a language store comprising, for respective selected languages, a language pack comprising a word index and a translation mapping for the selected language; and

the instructions further configured to, upon receiving a selection of a selected language;

request the selected language from the language store; and

upon receiving the language pack from the language store, store the language pack for the selected language in the memory.

19. The method of claim 18:

the language store comprising, for at least one selected language, at least two language packs respectively representing a language domain of the selected language; and

the instructions further configured to, upon receiving a selection of a target language domain for a selected language having a source language domain stored in the memory of the device;

request the target language domain for the selected language from the language store; and

upon receiving the language pack for the target language domain from the language store, store the language pack for the target language domain of the selected language in the memory.

20. A nonvolatile computer-readable storage device comprising instructions that, when executed on a processor of a device having a memory, cause the device to translate word sequences in a source language to translations in a target language by:

storing in the memory a word index comprising, for respective words of the language;

the word stored at an index location in the word index, and

a word mapping that identifies the index location of the word in the word index;

store in the memory a translation mapping identifying, for a word index sequence comprising at least two word indices, the translation of the words of the word sequence mapping to the respective index locations of the words of the word index sequence; and

upon receiving a word sequence in the source language to be translated into the target language;

for respective words of the word sequence, identifying the source index location of the word in the source word index;

using the translation mapping, identifying a translation of the source index locations of the words of the word sequence, the translation comprising at least one target index location in the target word index;

for respective target index locations, retrieving a translated word in the target language at the target index location in the target word index; and

presenting the translated words in the target language.