CROSS-LINGUAL DISCRIMINATIVE LEARNING OF SEQUENCE MODELS WITH POSTERIOR REGULARIZATION

• US 20150169549A1
• Filed: 12/13/2013
• Published: 06/18/2015
• Est. Priority Date: 12/13/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Das, Dipanjan, Ganchev, Kuzman
• Granted Patent: US 9,779,087 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 40/237 Lexical tools
• CPC Class Codes: G06F 40/45 Example-based machine translation; Alignment
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation

Abstract

A computer-implemented method can include obtaining (i) an aligned bi-text for a source language and a target language, and (ii) a supervised sequence model for the source language. The method can include labeling a source side of the aligned bi-text using the supervised sequence model and projecting labels from the labeled source side to a target side of the aligned bi-text to obtain a labeled target side of the aligned bi-text. The method can include filtering the labeled target side based on a task of a natural language processing (NLP) system configured to utilize a sequence model for the target language to obtain a filtered target side of the aligned bi-text. The method can also include training the sequence model for the target language using posterior regularization with soft constraints on the filtered target side to obtain a trained sequence model for the target language.

Claims

1. A computer-implemented method, comprising:

obtaining, at a computing device having one or more processors, (i) an aligned bi-text for a source language and a target language, and (ii) a supervised sequence model for the source language;

labeling, at the computing device, a source side of the aligned bi-text using the supervised sequence model to obtain a labeled source side of the aligned bi-text;

projecting, at the computing device, labels from the labeled source side to a target side of the aligned bi-text to obtain a labeled target side of the aligned bi-text;

filtering, at the computing device, the labeled target side based on a task of a natural language processing (NLP) system configured to utilize a sequence model for the target language to obtain a filtered target side of the aligned bi-text; and

training, at the computing device, the sequence model for the target language using posterior regularization with soft constraints on the filtered target side to obtain a trained sequence model for the target language.

2. The computer-implemented method of claim 1, further comprising filtering, at the computing device, the aligned bi-text to remove sentence pairs based on the task of the NLP system.

3. The computer-implemented method of claim 2, wherein the task of the NLP system is part-of-speech (POS) tagging for the target language.

4. The computer-implemented method of claim 2, wherein the task of the NLP system is named entity segmenting.

5. The computer-implemented method of claim 4, wherein filtering the labeled target side includes utilizing a dictionary for the target language to remove labels projected from the labeled source side.

6. The computer-implemented method of claim 4, wherein filtering the aligned bi-text includes removing at least one of (i) sentence pairs where greater than a threshold of source language tokens are unaligned, (ii) sentence pairs where an entity is unaligned, (iii) sentence pairs having an entity longer than three tokens, and (iv) sentence pairs having no entities.

7. The computer-implemented method of claim 1, wherein training the sequence model using posterior regularization with soft constraints includes optimizing an objective function using stochastic projected gradients for parameters for the sequence model and optimal dual variables, wherein the objective function is defined as:

8. The computer-implemented method of claim 1, wherein the source language is a resource-rich language having an abundance of labeled training data, and wherein the target language is a resource-poor language having approximately no labeled training data.

9. The computer-implemented method of claim 1, further comprising:

receiving, at the computing device, a question in the target language;

analyzing, at the computing device, the question using the trained sequence model to obtain an answer to the question; and

outputting, from the computing device, the answer.

10. A computing device comprising one or more processors configured to perform operations comprising:

obtaining (i) an aligned bi-text for a source language and a target language, and (ii) a supervised sequence model for the source language;

labeling a source side of the aligned bi-text using the supervised sequence model to obtain a labeled source side of the aligned bi-text;

projecting labels from the labeled source side to a target side of the aligned bi-text to obtain a labeled target side of the aligned bi-text;

filtering the labeled target side based on a task of a natural language processing (NLP) system configured to utilize a sequence model for the target language to obtain a filtered target side of the aligned bi-text; and

training the sequence model for the target language using posterior regularization with soft constraints on the filtered target side to obtain a trained sequence model for the target language.

11. The computing device of claim 10, wherein the operations further comprise filtering the aligned bi-text to remove sentence pairs based on the task of the NLP system.

12. The computing device claim 11, wherein the task of the NLP system is part-of-speech (POS) tagging for the target language.

13. The computing device of claim 11, wherein the task of the NLP system is named entity segmenting.

14. The computing device of claim 13, wherein filtering the labeled target side includes utilizing a dictionary for the target language to remove labels projected from the labeled source side.

15. The computing device of claim 13, wherein filtering the aligned bi-text includes removing at least one of (i) sentence pairs where greater than a threshold of source language tokens are unaligned, (ii) sentence pairs where an entity is unaligned, (iii) sentence pairs having an entity longer than three tokens, and (iv) sentence pairs having no entities.

16. The computing device of claim 10, wherein training the sequence model using posterior regularization with soft constraints includes optimizing an objective function using stochastic projected gradients for parameters for the sequence model and optimal dual variables, wherein the objective function is defined as:

17. The computing device of claim 10, wherein the source language is a resource-rich language having an abundance of labeled training data, and wherein the target language is a resource-poor language having approximately no labeled training data.

18. The computing device of claim 10, wherein the operations further comprise:

receiving a question in the target language;

analyzing the question using the trained sequence model to obtain an answer to the question; and

outputting the answer.

19. A non-transitory, computer-readable medium having instructions stored thereon that, when executed by one or more processors of a computing device, cause the computing device to perform operations comprising:

obtaining (i) an aligned bi-text for a source language and a target language, and (ii) a supervised sequence model for the source language;

labeling a source side of the aligned bi-text using the supervised sequence model to obtain a labeled source side of the aligned bi-text;

projecting labels from the labeled source side to a target side of the aligned bi-text to obtain a labeled target side of the aligned bi-text;

filtering the labeled target side based on a task of a natural language processing (NLP) system configured to utilize a sequence model for the target language to obtain a filtered target side of the aligned bi-text; and

training the sequence model for the target language using posterior regularization with soft constraints on the filtered target side to obtain a trained sequence model for the target language.

20. The computer-readable medium of claim 19, wherein training the sequence model using posterior regularization with soft constraints includes optimizing an objective function using stochastic projected gradients for parameters for the sequence model and optimal dual variables, wherein the objective function is defined as: