Encoding and adaptive, scalable accessing of distributed models

• US 9,619,465 B2
• Filed: 05/23/2014
• Issued: 04/11/2017
• Est. Priority Date: 02/17/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Och, Franz Josef, Dean, Jeffrey, Brants, Thorsten, Franz, Alexander Mark, Ponte, Jay, Xu, Peng, Teh, Sha-Mayn, Chin, Jeffrey, Thayer, Ignacio E., Carver, Anton, Rosart, Daniel, Hawkins, John S., Driesen, Karel
• Primary Examiner(s): Spooner, Lamont
• Application Number:
  US14/285,693
• Publication Number:
  US 20140257787A1
• Time in Patent Office: 1,054 Days
• Field of Search: 704 1- 10, 707706-708, 715264
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06F 40/47 Machine-assisted translation, e.g. using translation memory
• CPC Class Codes: G06F 40/49 using very large corpora, e.g. the web
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation

Abstract

Systems, methods, and apparatus for accessing distributed models in automated machine processing, including using large language models in machine translation, speech recognition and other applications.

Claims

1. A system comprising:

a translation server operable to perform machine translation obtaining translation model data from a translation model for translation between a source language and a target language and language model data from a language model for the target language, the translation server further operable to translate text in the source language into the target language using the obtained translation model data and language model data,

the translation server comprising;

a request queue operable to store requests for language model data to be obtained for translating a segment in the source language, and

a segment translation server cache operable to store language model data obtained by the requests by the translation server,

wherein the translation server is further operable to;

process the translation of the segment using language model data from a second language model for the target language to produce an initial translation of the segment before the requests for the language data in the language model in the request queue are sent out,

update the requests for the language model data of the language model in the request queue based on the initial translation,

send out the updated requests in the request queue to obtain language model data from the language model for processing the initial translation, and

after the updated requests are served and the data for the updated requests are stored in the segment translation server cache, process the initial translation with the data for the updated requests to produce a final translation.

2. The system of claim 1, wherein:

the segment translation server cache is operable to delete the obtained language model data after the segment is translated.

3. The system of claim 1, wherein:

the segment translation server cache is operable to delete the obtained language model data periodically.

4. The system of claim 1, wherein

the translation server is operable to process translation of the segment before all of the requests in the request queue are served, and

the translation server is further operable to finalize translation of the segment using the language model data in the segment translation server cache obtained by the requests.

5. The system of claim 1, wherein:

the segment translation server cache is further operable to store history information of translation of an assigned segment.

6. The system of claim 1, wherein the translation model is divided into a plurality of translation model partitions, each translation model partition being less than the entire translation model and being stored on a different translation model server of a plurality of translation model servers, and the respective translation model partitions together constituting the entire translation model.

7. The system of claim 1, wherein language model is divided into a plurality of language model partitions, each language model partition being less than the entire language model and the respective language model partitions together constituting the entire language model.

8. A system comprising:

a translation server operable to perform machine translation obtaining translation model data from a translation model for translation between a source language and a target language and language model data from a language model for the target language, the translation server further operable to translate text in the source language into the target language using the obtained translation model data and language model data,

the translation server comprising;

a request queue operable to store requests for language model data to be obtained for translating a segment in the source language,

a segment translation server cache operable to store language model data obtained by the requests by the translation server; and

a second segment translation server cache storing a selected portion of the language model,

wherein the translation server is operable to;

process the translation of the segment using language model data from a second language model for the target language to produce an initial translation of the segment before the requests for the language data in the language model in the request queue are sent out,

update the requests for the language model data of the language model in the request queue based on the initial translation,

send out the updated requests in the request queue to obtain language model data from the language model for processing the initial translation, and

after the updated requests are served and the data for the updated requests are stored in the segment translation server cache, process the initial translation with the data for the updated requests to produce a final translation,

after completing translation of the segment, delete data in the segment translation server cache and retain the selected portion of the language model in the second segment translation server cache.

9. The system of claim 8, wherein:

the translation server is operable to look up the second segment translation server cache for a piece of language model data needed for translating the segment before generating a request for the piece of language model data, and

when the piece of language model data is present in the second segment translation server cache, the translation server is operable to use the piece of language model data for translation without generating the request for the piece of language model data.

10. The system of claim 8, wherein the translation model is divided into a plurality of translation model partitions, each translation model partition being less than the entire translation model and being stored on a different translation model server of a plurality of translation model servers, and the respective translation model partitions together constituting the entire translation model.

11. The system of claim 8, wherein language model is divided into a plurality of language model partitions, each language model partition being less than the entire language model and the respective language model partitions together constituting the entire language model.

12. The system of claim 8, wherein the translation server is further operable to:

obtain the translation model data from the translation model based on the segment;

translate the segment into a set of possible translations based on the translation model data;

obtain the language model data from the language model based on the set of possible translations, the language model data matching at least one token in at least one possible translation of the set of possible translations; and

determine a translation of the segment based on the obtained language model data and the set of possible translations.

13. A system comprising:

a translation server operable to perform machine translation obtaining translation model data from a translation model for translation between a source language and a target language and language model data from a language model for the target language, the translation server further operable to translate text in the source language into the target language using the obtained translation model data and language model data,

the translation server comprising;

a request queue operable to store requests for language model data to be obtained for translating a segment in the source language,

a segment translation server cache operable to store language model data obtained by the requests by the translation server; and

a second segment translation server cache storing a selected portion of the language model,

wherein the translation server is operable to;

periodically delete data in the segment translation server cache,

process the translation of the segment using language model data from a second language model for the target language to produce an initial translation of the segment before the requests for the language data in the language model in the request queue are sent out,

update the requests for the language model data of the language model in the request queue based on the initial translation,

send out the updated requests in the request queue to obtain language model data from the language model for processing the initial translation, and

after the updated requests are served and the data for the updated requests are stored in the segment translation server cache, process the initial translation with the data for the updated requests to produce a final translation.

14. The system of claim 13, wherein the translation model is divided into a plurality of translation model partitions, each translation model partition being less than the entire translation model and being stored on a different translation model server of a plurality of translation model servers, and the respective translation model partitions together constituting the entire translation model.

15. The system of claim 13, wherein language model is divided into a plurality of language model partitions, each language model partition being less than the entire language model and the respective language model partitions together constituting the entire language model.

16. The system of claim 13, wherein:

the translation server is operable to look up the second segment translation server cache for a piece of language model data needed for translating the segment before generating a request for the piece of language model data; and

when the piece of language model data is present in the second segment translation server cache, the translation server is operable to use the piece of language model data for translation without generating the request for the piece of language model data.

17. The system of claim 13, wherein the translation server is further operable to:

obtain the translation model data from the translation model based on the segment;

translate the segment into a set of possible translations based on the translation model data;

obtain the language model data from the language model based on the set of possible translations, the language model data matching at least one token in at least one possible translation of the set of possible translations; and

determine a translation of the segment based on the obtained language model data and the set of possible translations.

18. The system of claim 13, wherein the translation server periodically deletes data in the segment translation server cache comprises deleting old data when the segment translation server cache is running out of space.