Word alignment score computing apparatus, word alignment apparatus, and computer program

• US 10,049,105 B2
• Filed: 02/12/2015
• Issued: 08/14/2018
• Est. Priority Date: 03/07/2014
• Status: Active Grant

• Current Assignee: National Institute of Information and Communications Technology
• Original Assignee: National Institute of Information and Communications Technology
• Inventors: Tamura, Akihiro, Watanabe, Taro, Sumita, Eiichiro
• Primary Examiner(s): Riley, Marcus T
• Application Number:
  US15/118,703
• Publication Number:
  US 20170068665A1
• Time in Patent Office: 1,279 Days
• Field of Search: None
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06F 40/45 Example-based machine translation; Alignment
• CPC Class Codes: G10L 25/30 using neural networks

Abstract

[Object] An object is to provide an apparatus for attaining highly precise word alignment.

[Solution] The apparatus includes: selecting means receiving a bilingual sentence pair and a word alignment for the bilingual sentence pair, for successively selecting words f_j of a sentence in a first language in a prescribed order; and a recurrent neural network (RNN) 100, computing, for all words of the sentence in the first language, a score 102 representing a probability that a word pair consisting of the word f_j and a word e_{a_{j}} aligned with the word f_j by a word alignment a_j in a second language of the bilingual sentence pair is a correct word pair, and based on this score, for computing a score of the word alignment a_j. When computing a score of word pair (f_j, e_{a_{j}}), RNN 100 computes a score 102 of the word pair (f_j, e_{a_{j}}) based on all word alignments a₁^j-1 selected by the selecting means prior to the word f_j of the word pair (f_j, e_{a_{j}}), of the word alignments a_j, by means of a recurrent connection 118.

Claims

1. A word alignment score computing apparatus for computing a score of word alignment of a bilingual sentence pair of first and second languages, comprising a processor configured to:

responsive to said bilingual sentence pair and a word alignment for the bilingual sentence pair, successively select words of a sentence in said first language of said bilingual sentence pair in a prescribed order; and

compute, for every word of the sentence in said first language of said bilingual sentence pair, a score representing a probability of the selected word and a word in said second language aligned with said word by said word alignment forming a correct word pair, and based on this score, for computing a score of said word alignment;

wherein;

in computing a score of a certain word pair, said processor computes the score of the certain word pair based on all alignments of the selected words preceding that word in said first language which forms the certain word pair;

utilize a recurrent neural network having a first input receiving a word selected by said processor and a second input receiving a word in said second language aligned with said word by said word alignment, and

apply the selected word and the word aligned with the word by said word alignment to said first and second inputs, respectively;

wherein said recurrent neural network includes;

an input layer having said first and second inputs, and computing and outputting word embedding vectors from words respectively applied to said first and second inputs,

a hidden layer connected to receive outputs of said input layer for generating, by a predetermined non-linear operation, a vector representing a relation between two outputs from said input layer, and

an output layer computing and outputting said score based on the output of said hidden layer; and

wherein the output of said hidden lever is applied as an input to said hidden layer when a next word pair is given to said word alignment score computing apparatus.

2. A word alignment apparatus for estimating word alignment of bilingual sentence pair of first and second languages, comprising:

the word alignment score computing apparatus according to claim 1;

word alignment candidate generating means for generating a plurality of word alignment candidates for said bilingual sentence pair;

computing means for computing, for each of said plurality of word alignment candidates generated by said word alignment candidate generating means, a word alignment score for said bilingual sentence pair using said word alignment score computing apparatus; and

word alignment determining means for determining and outputting as the word alignment for said bilingual sentence pair, that one of the plurality of word alignment candidates computed by said computing means for said plurality of word alignment candidates which corresponds to the highest score.

3. A computer program stored on a non-transitory computer readable medium causing, when executed by the processor, said processor to function as described in claim 1.

10. The word alignment apparatus according to claim 1, wherein the processor is further configured to:

generate a plurality of word alignment candidates for said bilingual sentence pair;

compute, for each of said plurality of generated word alignment candidates, a word alignment score for said bilingual sentence pair using said word alignment score computing apparatus; and

determine and output as the word alignment for said bilingual sentence pair, that one of the computed word alignment candidates for said plurality of word alignment candidates which corresponds to a highest score.

14. A computer program stored on a non-transitory computer readable medium causing, when executed by the processor, said processor to function as described in claim 2.

4. A word alignment score computing apparatus for computing a score of word alignment of a bilingual sentence pair of first and second languages, comprising a processor configured to:

responsive to said bilingual sentence pair and a word alignment for the bilingual sentence pair, successively select words of a sentence in said first language of said bilingual sentence pair in a prescribed order; and

compute, for every word of the sentence in said first language of said bilingual sentence pair, a score representing a probability of the selected word and a word in said second language aligned with said word by said word alignment forming a correct word pair, and based on this score, for computing a score of said word alignment;

wherein;

in computing a score of a certain word pair, said processor computes the score of the certain word pair based on all alignments of the selected words preceding that word in said first language which forms the certain word pair;

successively select words of the sentence in said first language of said bilingual sentence pair starting from a beginning of the sentence in said first language;

utilize a recurrent neural network having a first input receiving the selected word and a second input receiving a word in said second language aligned with said word by said word alignment, and

apply the selected word and the word aligned with the word by said word alignment to said first and second inputs, respectively;

wherein said recurrent neural network includes;

an input layer having said first and second inputs, and computing and outputting word embedding vectors from words respectively applied to said first and second inputs,

a hidden layer connected to receive outputs of said input layer, for generating, by a predetermined non-linear operation, a vector representing a relation between two outputs from said input layer, and

an output layer computing and outputting said score based on the output of said hidden layer; and

wherein the output of said hidden layer is applied as an input to said hidden layer when a next word pair is given to said word alignment score computing apparatus.

7. The word alignment apparatus according to claim 4, wherein the processor is further configured to:

generate a plurality of word alignment candidates for said bilingual sentence pair;

compute, for each of said plurality of generated word alignment candidates, a word alignment score for said bilingual sentence pair using said word alignment score computing apparatus; and

determine and output as the word alignment for said bilingual sentence pair, that one of the computed word alignment candidates for said plurality of word alignment candidates which corresponds to a highest score.

11. A computer program stored on a non-transitory computer readable medium causing, when executed by the processor, said processor to function as described in claim 4.

5. A word alignment score computing apparatus for computing a score of word alignment of a bilingual sentence pair of first and second languages, comprising a processor configured to:

responsive to said bilingual sentence pair and a word alignment for the bilingual sentence pair, successively select words of a sentence in said first language of said bilingual sentence pair in a prescribed order; and

compute, for every word of the sentence in said first language of said bilingual sentence pair, a score representing a probability of the selected word and a word in said second language aligned with said word by said word alignment forming a correct word pair, and based on this score, for computing a score of said word alignment;

wherein;

in computing a score of a certain word pair, said processor computes the score of the certain word pair based on all alignments of selected words preceding that word in said first language which forms the certain word pair;

compute a score representing a probability of a word pair consisting of the selected word and a word in a sentence in said second language of said bilingual sentence pair aligned with said word by said word alignment being a correct word pair;

compute, based on scores of all words of the sentence in said first language of said bilingual sentence pair, the score of said word alignment;

utilize a recurrent neural network having a first input receiving the selected word and a second input receiving a word in said second language aligned with said word by said word alignment, and

apply the selected word and the word aligned with the word by said word alignment to said first and second inputs, respectively;

wherein said recurrent neural network includes;

an input layer having said first and second inputs, and computing and outputting word embedding vectors from words respectively applied to said first and second inputs,

a hidden layer connected to receive outputs of said input layer, for generating, by a predetermined non-linear operation, a vector representing a relation between two outputs from said input layer, and

an output layer computing and outputting said score based on the output of said hidden layer; and

wherein the output of said hidden layer is applied as an input to said hidden layer when a next word pair is given to said word alignment score computing apparatus.

8. The word alignment apparatus according to claim 5, wherein the processor is further configured:

generate a plurality of word alignment candidates for said bilingual sentence pair;

compute, for each of said plurality of generated word alignment candidates, a word alignment score for said bilingual sentence pair using said word alignment score computing apparatus; and

determine and output as the word alignment for said bilingual sentence pair, that one of the computed word alignment candidates for said plurality of word alignment candidates which corresponds to a highest score.

12. A computer program stored on a non-transitory computer readable medium causing, when executed by the processor, said processor to function as described in claim 5.

6. A word alignment score computing apparatus for computing a score of word alignment of a bilingual sentence pair of first and second languages comprising a processor configured to,

responsive to said bilingual sentence pair and a word alignment for the bilingual sentence pair, successively select words of a sentence in said first language of said bilingual sentence pair in a prescribed order;

and

compute, for every word of the sentence in said first language of said bilingual sentence pair, a score representing a probability or the selected word and a word in said second language aligned with said word by said word alignment forming a correct word pair, and based on this score, for computing a score of said word alignment;

wherein;

in computing a score of a certain word pair, said processor computes the score of the certain word pair based on all alignments of the selected words preceding that word in said first language which forms the certain word pair;

compute a score representing a probability of a word pair consisting of the selected word and a word in a sentence in said second language of said bilingual sentence pair aligned with said word by said word alignment being a correct word pair;

compute, based on scores of all words of the sentence in said first language of said bilingual sentence pair computed by said processor, the score of said word alignment;

utilize a recurrent neural network having a first input receiving the selected word and a second input receiving a word in said second language aligned with said word by said word alignment, and

apply the selected word and the word aligned with the word by said word alignment to said first and second inputs, respectively;

compute the score of said word alignment by multiplying the scores of all words of the sentence in said first language of said bilingual sentence pair computed by said processor;

wherein said recurrent neural network includes;

an input layer having said first and second inputs, and computing and outputting word embedding vectors from words respectively applied to said first and second inputs,

a hidden layer connected to receive outputs of said input layer, for generating, by a predetermined non-linear operation, a vector representing a relation between two outputs from said input layer, and

an output layer computing and outputting said score based on the output of said hidden layer; and

wherein the output of said hidden layer is applied as an input to said hidden layer when a next word pair is given to said word alignment score computing apparatus.

9. The word alignment apparatus according to claim 6, wherein the processor is further configured to:

generate a plurality of word alignment candidates for said bilingual sentence pair;

compute, for each of said plurality of generated word alignment candidates, a word alignment score for said bilingual sentence pair using said word alignment score computing apparatus; and

determine and output as the word alignment for said bilingual sentence pair, that one of the computed word alignment candidates for said plurality of word alignment candidates which corresponds to a highest score.

13. A computer program stored on a non-transitory computer readable medium causing, when executed by the processor, said processor to function as described in claim 6.