TRAINING MARKOV RANDOM FIELD-BASED TRANSLATION MODELS USING GRADIENT ASCENT

• US 20140365201A1
• Filed: 02/18/2014
• Published: 12/11/2014
• Est. Priority Date: 06/09/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Gao, Jianfeng, He, Xiaodong
• Granted Patent: US 10,025,778 B2
• Time in Patent Office: Days
• US Class Current: 704/2
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models

Abstract

Various technologies described herein pertain to training and utilizing a general, statistical framework for modeling translation via Markov random fields (MRFs). An MRF-based translation model can be employed in a statistical machine translation (SMT) system. The MRF-based translation model allows for arbitrary features extracted from a phrase pair to be incorporated as evidence. The parameters of the model are estimated using a large-scale discriminative training approach based on stochastic gradient ascent and an N-best list based expected Bilingual Evaluation Understudy (BLEU) as an objective function.

Claims

1. A system that translates an input string in a source language to an output string in a target language, comprising:

a statistical machine translation (SMT) system that receives the input string in the source language and generates the output string in the target language based upon the input string in the source language, wherein the SMT system comprises;

a Markov random field (MRF)-based phrase translation model; and

a decoder component that evaluates scores of phrase translation pair hypotheses between the source language and the target language utilizing the MRF-based phrase translation model based upon a source phrase included in the input string in the source language, wherein the decoder component generates the output string in the target language as a function of the scores of the phrase translation pair hypotheses.

2. The system of claim 1, wherein:

the SMT system further comprises a feature extraction component that extracts features of the phrase translation pair hypotheses for the source phrase, wherein the phrase translation pair hypotheses for the source phrase comprise the source phrase included in the input string in the source language and candidate target phrases in the target language; and

the decoder component evaluates the scores of the phrase translation pair hypotheses between the source language and the target language utilizing the MRF-based phrase translation model based upon the features of the phrase translation pair hypotheses for the source phrase.

3. The system of claim 2, wherein the features of the phrase translation pair hypotheses for the source phrase comprise phrase-pair features.

4. The system of claim 2, wherein the features of the phrase translation pair hypotheses for the source phrase comprise word-pair features.

5. The system of claim 2, wherein the features of the phrase translation pair hypotheses for the source phrase comprise phrase-pair features and word-pair features.

6. The system of claim 2, wherein the features of the phrase translation pair hypotheses for the source phrase comprise phrase-pair features, word-pair features, and triplet features.

7. The system of claim 1, wherein the SMT system further comprises:

at least one disparate model;

wherein the decoder component generates the output string in the target language utilizing the at least one disparate model; and

wherein the decoder component uses a weighted log-linear combination of the MRF-based phrase translation model and the at least one disparate model.

8. The system of claim 7, wherein the at least one disparate model comprises one or more of a phrase translation model, a word translation model, a lexicalized reordering model, a word count model, a phrase count model, or an n-gram language model.

9. The system of claim 1, further comprising an online adaptation component that receives feedback pertaining to the output string in the target language generated by the SMT system and updates the MRF-based phrase translation model responsive to the feedback.

10. The system of claim 9, wherein the feedback comprises a modified translation of the input string relative to the output string, the modified translation being in the target language, and wherein the online adaptation component utilizes the modified translation as a positive example and the output string as a negative example to update the MRF-based phrase translation model.

11. The system of claim 1, further comprising a training component that learns parameters of the MRF-based phrase translation model from training data using a large-scale discriminative training algorithm based on stochastic gradient ascent and an objective function for parameter optimization.

12. The system of claim 11, wherein the training component further comprises:

a candidate identification component that generates respective N-best lists of translation hypotheses for source sentences in the training data;

a label component that computes respective objective function scores for the translation hypotheses;

a score evaluation component that computes respective translation scores for the translation hypotheses using current parameters of the MRF-based phrase translation model; and

an optimization component that updates the parameters of the MRF-based phrase translation model utilizing stochastic gradient ascent based on the objective function scores and the translation scores for the translation hypotheses.

13. A method of training a Markov random field (MRF)-based phrase translation model for a statistical machine translation (SMT) system, comprising:

generating respective N-best lists of translation hypotheses for source sentences in training data;

computing respective objective function scores for the translation hypotheses;

computing respective translation scores for the translation hypotheses using current parameters of the MRF-based phrase translation model for the SMT system; and

updating the parameters of the MRF-based phrase translation model utilizing stochastic gradient ascent based on the objective function scores and the translation scores for the translation hypotheses.

14. The method of claim 13, wherein the objective function scores are expected Bilingual Evaluation Understudy (BLEU) scores.

15. The method of claim 13, further comprising updating the parameters of the MRF-based phrase translation model responsive to received feedback.

16. The method of claim 13, further comprising optimizing respective weights for the MRF-based phrase translation model and at least one disparate model for the SMT system.

17. The method of claim 13, further comprising:

receiving, at the SMT system, an input string in a source language;

evaluating scores of phrase translation pair hypotheses between the source language and the target language utilizing the MRF-based phrase translation model based upon a source phrase included in the input string in the source language; and

generating an output string in the target language as a function of the scores of the phrase translation pair hypotheses.

18. The method of claim 13, wherein the MRF-based phrase translation model comprises a linear combination of phrase-pair features, word-pair features, and triplet features.

19. A method of translating an input string in a source language to an output string in a target language, comprising:

extracting features of phrase translation pair hypotheses for a source phrase included in the input string in the source language, wherein the phrase translation pair hypotheses for the source phrase comprise the source phrase included in the input string in the source language and candidate target phrases in the target language, and wherein the features of the phrase translation pair hypotheses for the source phrase comprise phrase-pair features, word-pair features, and triplet features;

evaluating scores of phrase translation pair hypotheses between the source language and the target language based upon the features of the phrase translation pair hypotheses for the source phrase included in the input string in the source language; and

generating the output string in the target language as a function of the scores of the phrase translation pair hypotheses.

20. The method of claim 19, further comprising evaluating the scores of the phrase translation pair hypotheses between the source language and the target language based upon the features of the phrase translation pair hypotheses for the source phrase included in the input string in the source language utilizing a Markov random field (MRF)-based phrase translation model of a statistical machine translation (SMT) system.