STATISTICAL MACHINE TRANSLATION APPARATUS AND METHOD

• US 20100088085A1
• Filed: 04/09/2009
• Published: 04/08/2010
• Est. Priority Date: 10/02/2008
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Jeon, Jae-Hun, Lee, Jae-Won
• Application Number:
  US12/420,922
• Publication Number:
  US 20100088085A1
• Time in Patent Office: Days
• US Class Current: 704/7
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models

Abstract

A statistical machine translation apparatus and method reflecting linguistic information are provided. In the process of generating a translation model based on statistical information on source language sentences and target language sentences during word alignment, the translation model is generated using word alignment results that are amended based on a bilingual dictionary. Further, instead of using the source language sentence and the target language sentence (i.e., their bilingual corpora) as materials to generate the translation model, it is determined whether or not the morphemes are meaningful content words in the source and target language sentences. Based on the determination, pre-processing is performed on the source language sentence and the target language sentence.

Claims

1. A statistical machine translation apparatus, comprising:

a source language pre-processor configured to analyze morphemes of an input source language sentence and generating a resulting source language sentence, in which tags representing characteristics per morpheme are attached to the morphemes;

a target language pre-processor configured to analyze morphemes of an input target language sentence and generating a resulting target language sentence, in which tags representing characteristics per morpheme are attached to the morphemes;

a bilingual dictionary configured to store pairs of source and target language words having the same meaning; and

a translation model generator configured to generate a translation model for the source and target language sentences, using the bilingual dictionary.

2. The apparatus of claim 1, wherein, in response to word alignment for generating the translation model being performed, the translation model generator is further configured to:

generate common alignment information extracted from both forward direction alignment information, in which the source language words and their corresponding target language words are aligned, and backward direction alignment information, in which the target language words and their corresponding source language words are aligned; and

amend the common alignment information based on the bilingual dictionary.

3. The apparatus of claim 2, wherein the translation model generator is configured to amend the common alignment information to conform the pairs of source language words and target language words included in the common alignment information to those in the bilingual dictionary.

4. The apparatus of claim 2, wherein, in response to the source language word and its corresponding target language word included in the common alignment information not matching each other, the translation model generator is configured to search for a target word for the source language word in the bilingual dictionary, determines the searched target word as the target language word, and amend the common alignment information.

5. The apparatus of claim 1, wherein:

the source language pre-processor is configured to transfer the source language morpheme or tag to the translation model generator in response to the source language morpheme being determined as a content word that is a meaningful morpheme, using the tags attached per morpheme of the resulting source language sentence; and

the target language pre-processor is configured to transfer the target language morpheme or tag to the translation model generator in response to the target language morpheme being determined as a content word that is a meaningful morpheme, using the tags attached per morpheme of the target language sentence.

6. The apparatus of claim 5, wherein:

the source language pre-processor is configured to transfer a source language morpheme to the translation model generator in response to the source language morpheme being determined as a content word among the source language morphemes and transfer a tag of a source language morpheme to the translation model generator in response to determining the source language morpheme is not a content word; and

the target language pre-processor is configured to transfer a target language morpheme to the translation model generator in response to the target language morpheme being determined as a content word among the target language morphemes and transfer a tag of a target language morpheme to the translation model generator in response to determining the target language morpheme is not a content word.

7. The apparatus of claim 6, wherein the translation model generator is configured to generate the translation model using the source language morpheme that is determined as a content word, the target language morpheme that is determined as a content word, the tag of the source language morpheme that is determined not to be a content word, or the tag of the target language morpheme that is determined not to be a content word.

8. The apparatus of claim 1, further comprising:

a decoding pre-processor configured to analyze morphemes of an input source language sentence, and generating source language words to which tags representing characteristics per morpheme are attached;

a decoder configured to translate the source language words to which the tags are attached into a target language sentence using the translation model; and

a name entity dictionary that includes categorized information on name entities,

wherein, in response to there being a source language word that is determined to have no target word in the source language sentence, the decoder is configured to search for a target word for the source language word using the name entity dictionary and translate the source language word into the target word using the searched results.

9. The apparatus of claim 8, wherein the decoder is configured to perform context analysis on the source language sentence including the source language word that is determined to have no target word and determine a category within which the source language word that is determined to have no target word falls.

10. The apparatus of claim 8, wherein the decoder is configured to use a target language corresponding to pronunciation of the source language as a target word for the source language word that is determined to have no target word in the name entity dictionary.

11. A machine translation method, comprising:

pre-processing by a source language pre-processor the source language sentence by analyzing morphemes of an input source language sentence, and generating a resulting source language sentence, in which tags representing characteristics per morpheme are attached to the morphemes;

pre-processing by a target language pre-processor the target language sentence by analyzing morphemes of an input target language sentence, and generating a resulting target language sentence, in which tags representing characteristics per morpheme are attached to the morphemes; and

generating by a translation model generator a translation model of the source and target language sentences, using a bilingual dictionary storing pairs of source and target language words having the same meaning.

12. The method of claim 11, wherein performing word alignment for generating the translation model while generating the translation model by the translation model generator comprises:

generating forward direction alignment information, in which source language words and their corresponding target language words are aligned;

generating backward direction alignment information, in which the target language words and their corresponding source language words are aligned;

generating common alignment information extracted from both the forward direction alignment information and the backward direction alignment information; and

amending the generated common alignment information based on the bilingual dictionary.

13. The method of claim 12, wherein the common alignment information is amended by the translation model generator to conform the pairs of source and target language words included in the common alignment information to those in the bilingual dictionary.

14. The method of claim 12, wherein, in response to the source language word and its corresponding target language word included in the common alignment information not matching each other while amending the common alignment information, a target word for the source language word in the bilingual dictionary is determined by the translation model generator as the target language word, so that the common alignment information is amended by the translation model generator.

15. The method of claim 11, wherein:

pre-processing the source language word by the source language pre-processor includes determining whether each source language morpheme is a content word that is a meaningful morpheme, using the tag attached per morpheme of each resulting source language sentence, and leaving the source language morpheme or the tag in response to determining the morpheme is a content word; and

pre-processing the target language word by the target language pre-processor includes determining whether each target language morpheme is a content word that is a meaningful morpheme, using the tag attached per morpheme of each resulting target language sentence, and leaving the target language morpheme or the tag in response to determining the morpheme is a content word.

16. The method of claim 15, wherein, among the source or target language morphemes, in response to determining by the source or target language pre-processor a source language morpheme or a target language morpheme is a content word, the source language morpheme or the target language morpheme is left, and in response to determining by the source or target language pre-processor a source language morpheme or a target language morpheme is not a content word, a tag of the source or target language morpheme that is not determined as a content word is left.

17. The method of claim 16, wherein the translation model is generated by the translation model generator using the left source language morpheme, the left target language morpheme, the left tag of the source language morpheme, or the left tag of the target language morpheme.

18. The method of claim 11, further comprising:

performing decoding pre-processing by a decoding pre-processor in which morphemes of an input source language sentence are analyzed to generate source language words; and

performing decoding by a decoder that translates the source language words to which the tags are attached into a target language sentence using the translation model,

wherein the performing the decoding includes;

in response to determining the input source language sentence including the source language word has no target word, searching by a searcher for a target word for the source language word using a name entity dictionary, the name entity dictionary including categorized information on name entities; and

translating by a translator the source language word into the target word using the searched results.

19. The method of claim 18, wherein performing decoding by the decoder includes performing context analysis on the source language sentence including the source language word that is determined to have no target word and determining a category within which the source language word that is determined to have no target word falls.

20. The method of claim 18, wherein, in performing decoding by the decoder, a target language corresponding to pronunciation of the source language word that is determined to have no target word in the name entity dictionary is used as the target word.