Multi-domain machine translation system with training data clustering and dynamic domain adaptation

• US 10,437,933 B1
• Filed: 08/16/2016
• Issued: 10/08/2019
• Est. Priority Date: 08/16/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Clifton, Ann, Denkowski, Michael, Lavie, Alon
• Primary Examiner(s): Spooner, Lamont M
• Application Number:
  US15/238,101
• Time in Patent Office: 1,148 Days
• Field of Search: 704 2- 8
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06F 40/45 Example-based machine translation; Alignment
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation

Abstract

A machine translation system capable of clustering training data and performing dynamic domain adaptation is disclosed. An unsupervised domain clustering process is utilized to identify domains in general training data that can include in-domain training data and out-of-domain training data. Segments in the general training data are then assigned to the domains in order to create domain-specific training data. The domain-specific training data is then utilized to create domain-specific language models, domain-specific translation models, and domain-specific model weights for the domains. An input segment to be translated can be assigned to a domain at translation time. The domain-specific model weights for the assigned domain can be utilized to translate the input segment.

Claims

1. An apparatus, comprising:

one or more-processors; and

one or more non-transitory computer-readable storage media having instructions stored thereupon which are executable by the one or more processors and which, when executed, cause the apparatus to;

identify a plurality of domains in general training data comprising client training data and background training data, the client training data and the background training data being expressed in a source language and a target language;

assign segments in the general training data to one of the plurality of domains to create domain-specific training data for the plurality of domains;

generate a domain-specific translation model for the plurality of domains using the domain-specific training data;

generate a domain-specific language model for the plurality of domains using the domain-specific training data;

extract domain-specific tuning data from the domain-specific training data;

generate candidate translations using the domain-specific tuning data, the domain-specific language models, and the domain-specific translation models;

determine feature scores corresponding to individual ones of the domain-specific language models and the domain-specific translation models based at least in part on the candidate translations;

learn, based at least in part on the domain-specific tuning data and the feature scores, domain-specific model weights associated with the feature scores;

generate a translation package comprising the domain-specific language models, the domain-specific translation models, and the domain-specific model weights;

receive a request to translate an input segment in the source language into the target language;

identify individual domain-specific model weights to be utilized to translate the input segment;

identify one or more phrases associated with the input segment; and

translate the input segment into the target language by selecting, based at least in part on the individual domain-specific model weights, one or more candidate translations of the candidate translations corresponding to the one or more phrases of the input segment.

2. The apparatus of claim 1, wherein the plurality of domains in the general training data are identified using an unsupervised domain clustering process.

3. The apparatus of claim 2, wherein the unsupervised domain clustering process comprises k-means clustering.

4. The apparatus of claim 1, wherein to identify the individual domain-specific model weights to be utilized to translate the input segment comprises:

receiving, with the input segment, an indication of a domain associated with the input segment; and

determining the individual domain-specific model weights to be utilized to translate the input segment based on the indication.

5. The apparatus of claim 1, wherein to identify the individual domain-specific model weights to be utilized to translate the input segment comprises:

associating the input segment with one or more of the plurality of domains; and

determining the individual domain-specific model weights to be utilized to translate the input segment based upon the one or more of the plurality of domains with which the input segment is associated.

6. The apparatus of claim 1, wherein the non-transitory computer-readable storage media has further instructions stored thereupon to cause the translation package to be deployed to one or more host computers in a service provider network for use in translating input segments in the source language to the target language.

7. The apparatus of claim 1, wherein the non-transitory computer-readable storage media has further instructions stored thereupon to:

tokenize the input segment by removing at least one of a contraction or punctuation from the input segment; and

de-tokenize the translated input segment by adding at least one of a contraction or punctuation to the input segment.

8. A computer-implemented method for statistical machine translation, the method comprising:

assigning segments in general training data to one of a plurality of domains to create domain-specific training data for the plurality of domains;

generating domain-specific translation models for the plurality of domains using the domain-specific training data;

generating domain-specific language models for the plurality of domains using the domain-specific training data;

extracting domain-specific tuning data from the domain-specific training data;

generating candidate translations using the domain-specific tuning data, the domain-specific language models, and the domain-specific translation models;

determining feature scores corresponding to individual ones of the domain-specific language models and the domain-specific translation models based at least in part on the candidate translations;

learning, based at least in part on the domain-specific tuning data and the feature scores, domain-specific model weights for the plurality of domains;

utilizing the domain-specific language models, the domain-specific translation models, and the domain-specific model weights to translate input segments from a source language to a target language;

receiving a request to translate a particular input segment in the source language into the target language;

identifying individual domain-specific model weights to be utilized to translate the particular input segment;

identifying one or more phrases associated with the particular input segment; and

translating the particular input segment into the target language by selecting, based at least in part on the individual domain-specific model weights, one or more candidate translations of the candidate translations corresponding to the one or more phrases of the particular input segment.

9. The computer-implemented method of claim 8, wherein assigning segments in general training data to one of a plurality of domains to create domain-specific training data for the plurality of domains comprises:

identifying a plurality of domains in the general training data; and

assigning the segments in the general training data to one of the plurality of domains to create the domain-specific training data.

10. The computer-implemented method of claim 9, wherein the plurality of domains in the general training data are identified using an unsupervised domain clustering process.

11. The computer-implemented method of claim 10, wherein the unsupervised domain clustering process comprises k-means clustering.

12. The computer-implemented method of claim 8, wherein identifying the individual domain-specific model weights to be utilized to translate the particular input segment comprises:

receiving, with the particular input segment, an indication of a domain associated with the particular input segment; and

determining the individual domain-specific model weights to be utilized to translate the particular input segment based on the indication.

13. The computer-implemented method of claim 8, wherein identifying the individual domain-specific model weights to be utilized to translate the particular input segment comprises:

associating the particular input segment with one or more of the plurality of domains responsive to receiving the request to translate the particular input segment; and

determining the individual domain-specific model weights to be utilized to translate the particular input segment based upon the one or more of the plurality of domains with which the particular input segment is associated.

14. The computer-implemented method for statistical machine translation of claim 8, further comprising:

tokenizing the input segment by removing at least one of a contraction or punctuation from the input segment; and

de-tokenizing the translated input segment by adding at least one of a contraction or punctuation to the input segment.

15. A non-transitory computer-readable storage media having instructions stored thereupon that are executable by one or more processors and which, when executed, cause the one or more processors to:

receive a request to translate an input segment in a source language into a target language;

responsive to receiving the request, identify a domain for the input segment;

identify individual domain-specific model weights for the identified domain, wherein the domain-specific model weights are learned based at least in part on domain-specific tuning data and feature scores corresponding to individual ones of domain-specific language models and domain-specific translation models;

identify one or more phrases associated with the input segment; and

translate the input segment from the source language into the target language by selecting, based at least in part on the individual domain-specific model weights, one or more candidate translations corresponding to the one or more phrases of the input segment.

16. The non-transitory computer-readable storage media of claim 15, wherein identifying the individual domain-specific model weights to be utilized to translate the input segment comprises:

receiving, with the input segment, an indication of the domain associated with the input segment; and

determining the individual domain-specific model weights to be utilized to translate the input segment based on the indication.

17. The non-transitory computer-readable storage media of claim 15, wherein identifying the individual domain-specific model weights to be utilized to translate the input segment comprises:

assigning the input segment to one of a plurality of domains responsive to receiving the request to translate the input segment; and

determining the individual domain-specific model weights to be utilized to translate the input segment based upon the one of the plurality of domains to which the input segment is assigned.

18. The non-transitory computer-readable storage media of claim 15, having further instructions stored thereupon to:

assign segments in general training data to one of a plurality of domains to create domain-specific training data for the plurality of domains;

generate the domain-specific translation models for the plurality of domains using the domain-specific training data; and

generate the domain-specific language models for the plurality of domains using the domain-specific training data.

19. The non-transitory computer-readable storage media of claim 18, wherein the plurality of domains in the general training data are identified using an unsupervised domain clustering process.

20. The non-transitory computer-readable storage media of claim 19, wherein the unsupervised domain clustering process comprises k-means clustering.