Encoding and Adaptive, Scalable Accessing of Distributed Models

• US 20080262828A1
• Filed: 02/16/2007
• Published: 10/23/2008
• Est. Priority Date: 02/17/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Brants, Thorsten, Dean, Jeffrey, Xu, Peng, Rosart, Daniel, Chin, Jeffrey, Teh, Sha-Mayn, Driesen, Karel, Ponte, Jay, Och, Franz Josef, Franz, Alexander Mark, Thayer, Ignacio E., Hawkins, John S., Carver, Anton
• Granted Patent: US 8,296,123 B2
• Time in Patent Office: Days
• US Class Current: 704/3
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06F 40/47 Machine-assisted translation, e.g. using translation memory
• CPC Class Codes: G06F 40/49 using very large corpora, e.g. the web
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation

Abstract

Systems, methods, and apparatus for accessing distributed models in automated machine processing, including using large language models in machine translation, speech recognition and other applications.

Claims

1. A system comprising:

a plurality of machine translation resource servers, each machine translation resource server storing and operable to serve a partition of a collection of machine translation resource data for translation from a source language to a target language, the respective partitions together constituting the collection of machine translation resource data and each respective partition being less than the collection of machine translation resource data; and

at least one translation server operable to receive source text in the source language to be translated into the target language, the translation server further operable to obtain machine translation resource data from the plurality of machine translation resource servers and to use the obtained machine translation resource data to translate the source text into the target language.

2. The system of claim 1, wherein:

the plurality of machine translation resource servers comprise;

a plurality of language model servers storing and operable to serve a partition of a language model for the target language, the respective partitions together constituting the entire language model, and

a translation model server storing and operable to serve to the translation server a translation model for translation between the source language and the target language; and

wherein the translation server is operable to obtain translation model data from the translation model server and language model data from the plurality of language model servers, and is further operable to translate the source text into the target language based on the obtained translation model data and language model data.

3. The system of claim 2, further comprising:

one or more replica servers for the translation model server.

4. The system of claim 2, further comprising:

one or more replica servers for each of the language model servers.

5. The system of claim 4, wherein:

the translation server comprises;

a plurality of segment translation servers each operable to communicate with the translation model server, the language model servers and replica servers, each segment translation server operable to translate one segment of the source text into the target language,

a translation front end to receive the source text and to divide the source text into a plurality of segments in the source language, and

a load balancing module in communication with the translation front end to receive the segments of the source text and operable to distribute the segments to the segments to the segment translation servers for translation based on work load at the segment translation servers, the load balancing module further operable to direct translated segments in the target language from the segment translation servers to the translation front end.

6. The system of claim 2, wherein:

the translation server comprises;

a plurality of segment translation servers each operable to communicate with the translation model server, and the language model servers, each segment translation server operable to translate one segment of the source text into the target language,

a translation front end to receive the source text and to divide the source text into a plurality of segments in the source language, and

a load balancing module in communication with the translation front end to receive the segments of the source text and operable to distribute the segments to the segments to the segment translation servers for translation based on work load at the segment translation servers, the load balancing module further operable to direct translated segments in the target language from the segment translation servers to the translation front end.

7. The system of claim 2, wherein:

the translation model comprises mapping information between the source language and the target language and scoring information associated with each mapping.

8. The system of claim 7, wherein:

the mapping information comprises a relation between (1) one or more tokens in the source language and (2) one or more tokens in the target language.

9. The system of claim 7, wherein:

the scoring information comprises statistical data for each mapping between the source language and the target language.

10. The system of claim 9, wherein:

the statistical data comprises a probability of occurrence of each mapping between the source language and the target language.

11. The system of claim 7, wherein:

the translation server is operable to use the scoring information to select a desired translation for the source text from possible translations constructed from the obtained translation model data and language model data.

12. The system of claim 2, wherein:

the language model comprises n-grams in the target language and statistical data for the n-grams, each n-gram being a sequence of n tokens in the target language, wherein

n is a positive integer.

13. The system of claim 12, wherein:

the language model comprises n grams for n is greater than 3.

14. The system of claim 12, wherein:

each respective partition of the plurality of language model servers comprises all n-grams in the language model satisfying a partitioning criterion.

15. The system of claim 14, wherein:

each respective partition of the plurality of language model servers comprises all n-grams in the language model having a common token at a predetermined position in each n-gram.

16. The system of claim 15, wherein:

each respective partition of the plurality of language model servers comprises all n-grams in the language model having a plurality of common tokens.

17. The system of claim 16, wherein:

the plurality of common tokens are at predetermined positions in each n-gram.

18. The system of claim 16, wherein:

the plurality of common tokens are the last two tokens in a sequence of n tokens.

19. The system of claim 12, wherein:

the statistical data indicates a respective frequency of occurrence of each of the respective n-grams in a corpus of target language text.

20. The system of claim 12, wherein:

each n-gram is represented by a sequence if identification numbers and each identification number is assigned to uniquely represent a particular token in the target language.

21. The system of claim 1, wherein:

the translation server comprises;

at least one translation front end operable to divide the source text into a plurality of segments in the source language,

a plurality of segment translation servers, each segment translation server operable to obtain at least a portion of the obtained machine translation resource data and to translate a segment in the source language into the target language, and

a load balancing module operable to assign the segments to one or more of the segment translation servers for translation based on translation load at the segment translation servers, the load balancing module operable to direct translated segments to the translation front end.

22. The system of claim 21, further comprising:

a translation cache storing translations of, at least, tokens and segments each comprising a combination of tokens, from the source language to the target language, and

wherein either the translation front end or the load balancing module, is operable to look up the translation cache for a translation of a segment before directing the segment to segment translation servers, and the load balancing module is further operable to select each segment, that does not have a corresponding translation in the translation cache, to be sent to the segment translation servers for translation.

23. The system of claim 21, wherein:

each segment translation server comprises a segment translation server cache operable to store at least part of the obtained machine translation resource data.

24. The system of claim 23, wherein:

the segment translation server cache is operable to store the at least part of the obtained machine translation resource data during a period when an assigned segment is being translated by each segment translation server.

25. The system of claim 24, wherein:

the segment translation server cache is operable to delete the at least part of the obtained machine translation resource data after the assigned segment is translated.

26. The system of claim 24, wherein:

the segment translation server cache is operable to delete the at least part of the obtained machine translation resource data periodically.

27. The system of claim 23, wherein:

each segment translation server is further operable to check presence of a piece of machine translation resource data to be obtained from the plurality of the machine translation resource servers, and to obtain data from the plurality of the machine translation resource servers that is not stored in the segment translation server cache.

28. The system of claim 23, wherein:

the segment translation server cache is operable to further store history information of translation of an assigned segment.

29. The system of claim 23, wherein:

each segment translation server further comprises a request queue which is operable to store requests of different parts of language model data to be obtained from the plurality of language model servers, and

each segment translation server is operable to sequentially send the requests in the request queue to one or more of the plurality of language model servers.

30. The system of claim 29, wherein:

each segment translation server is further operable to process translation of an assigned segment using other machine translation resource data different from the language model data to be obtained by the requests in the request queue before all of the requests in the request queue are served, and

each segment translation server is further operable to finalize translation of the assigned segment processed by using the other machine translation resource data using the language model data obtained from the requests.

31. The system of claim 30, wherein:

the plurality of machine translation resource servers further comprise at least one machine translation resource server which serves the other machine translation resource data.

32. The system of claim 31, wherein:

the machine translation resource server which serves the other machine translation resource data stores a second, different language model for the target language.

33. The system of claim 23, wherein:

each segment translation server further comprises a second segment translation server cache which stores a selected portion of the machine translation resource data, and

each segment translation server is operable to obtain data from the plurality of machine translation resource servers that is not part of the stored selected portion in the second segment translation server cache.

34. The system of claim 1, further comprising:

a translation cache, the translation cache storing translations of tokens and combinations of tokens from the source language to the target language, and

the translations comprise machine translations and manual human translations.

35. The system of claim 34, wherein:

the translation cache comprises translation quality information.

36. The system of claim 2, wherein:

the translation server comprises a statistical machine translation server.

37. The system of claim 36, wherein:

the translation server further comprises a machine translation server operating a machine translation mechanism that is different from statistical machine translation.

38. The system of claim 37, wherein:

the machine translation server is a rule-based machine translation server.

39. The system of claim 1, further comprising:

a communication network with which the machine translation resource servers and the translation server are in communication, the communication network operable to direct the source text from a client computer in the communication network to the translation server for translation.

40. The system of claim 39, wherein:

the communication network is operable to direct translation of the source text in the target language from the translation server to the client computer.

41. The system as in claim 1, wherein:

the machine translation resources servers comprise;

a plurality of language model servers respectively storing and operable to serve different partitions of a language model for the target language, the respective partitions together constituting the entire language model and each respective partition being less than the whole of the language model;

one or more replica language model servers for each of the plurality of language model servers;

a plurality of translation model servers respectively storing and operable to serve different partitions of a translation model for translation between the target language and a human source language, the respective partitions together constituting the entire translation model and each respective partition being less than the whole of the translation model; and

one or more replica translation model servers for each of the plurality of translation model servers;

the translation server comprises;

a plurality of translation front ends each operable to divide source text, in the source language to be translated into the target language, into a plurality of segments in the source language;

a plurality of segment translation servers each operable to perform machine translation obtaining translation model data from the plurality of translation model servers and the replica translation model servers and obtaining language model data from the plurality of language model servers and the replica language model servers; and

a load balancing module operable to assign the segments to one or more of the plurality of segment translation servers for translation based on translation load at the plurality of segment translation servers.

42. The system of claim 41, further comprising:

a translation cache storing translations of, at least, tokens and segments each comprising a combination of tokens, from the source language to the target language.

43. The system of claim 42, wherein:

each translation front end is operable to look up the translation cache for a suitable translation for a segment to be translated and, when the suitable translation is present, to retrieve the suitable translation without directing the segment to the load balancing module for translation by one of the plurality of the segment translation servers.

44. The system of claim 42, wherein:

the load balancing module is operable to look up the translation cache for a suitable translation for a segment to be translated from a translation front end and, when the suitable translation is present, to retrieve the suitable translation from the translation cache and direct the retrieved translation to the translation front end, without directing the segment to one of the plurality of the segment translation servers for translation.

45. The system of claim 41, wherein:

each segment translation server comprises a segment translation server cache operable to store language model data obtained from the plurality of language model servers and the replica language model servers for translating a segment.

46. The system of claim 45, wherein:

each segment translation server is further operable to check presence of a piece of language model data to be obtained from the plurality of the language model servers and the replica language model servers, and when the piece of language model data is present, to use the piece of language model data for translation without obtaining the piece of language model data from the plurality of the language model servers and the replica language model servers, and

wherein each segment translation server is further operable to obtain the piece of language model data from the plurality of the language model servers and the replica language model servers when the piece of language model data is not in the segment translation server cache.

47. The system of claim 45, wherein:

each segment translation server further comprises a request queue which is operable to store requests for language model data to be obtained from the plurality of language model servers, and

each segment translation server is operable to sequentially send the requests in the request queue to one or more of the plurality of language model servers and the replica language model servers and store language model data obtained by the requests in the segment translation server cache.

48. The system of claim 45, wherein:

each segment translation server is further operable to process translation of a segment before all of the requests in the request queue are served, and

each segment translation server is further operable to finalize translation of the segment using the language model data obtained by the requests.

49. The system of claim 48, wherein:

each segment translation server is further operable to fill the segment translation server cache with language model data from another source before all of the requests in the request queue are served and processes the translation using the language model data from the other source, and

after all of the requests in the request queue are served, each segment translation server is operable to store language model data obtained by the requests in the segment translation server cache and further process the translation with the language model data obtained by the requests.

50. The system of claim 41, wherein:

each segment translation server comprises;

a first segment translation server cache operable to store language model data obtained from the plurality of language model servers and the replica language model servers for translating a segment, and

a second segment translation server cache storing a selected portion of the language model in the plurality of language model servers and the replica language model servers.

51. The system of claim 50, wherein:

each segment translation server is operable to, after completing translation of the segment, delete data in the first segment translation server cache and retain the selected portion of the language model in the second segment translation server cache.

52. The system of claim 41, further comprising:

a plurality of second translation model servers respectively storing and operable to serve different partitions of a second translation model for translation between the target language and a second human source language, the respective partitions together constituting the entire second translation model and each respective partition being less than the whole of the second translation model;

one or more second replica translation model servers for each of the plurality of second translation model servers;

a plurality of second translation front ends each operable to divide a second source text, in the second language to be translated into the target language, into a plurality of segments in the second source language;

a plurality of second segment translation servers each operable to perform machine translation obtaining second translation model data from the plurality of second translation model servers and the second replica translation model servers and obtaining language model data from the plurality of language model servers and the replica language model servers; and

a second load balancing module operable to assign the segments to one or more of the plurality of second segment translation servers for translation, based on translation load at the plurality of second segment translation servers.

53. The system of claim 41, wherein:

each segment translation server is operable to monitor work load at a respective language model server and corresponding one or more replica language model servers for a respective partition, and

in response to monitored work load, each segment translation server is further operable to select a server from the respective language model server and corresponding one or more replica language model servers to request for data in the respective partition of the language model.

54. A system comprising:

a translation server operable to perform machine translation obtaining translation model data from a translation model for translation between a source language and a target language and language model data from a language model for the target language, the translation server further operable to translate text in the source language into the target language using the obtained translation model data and language model data,

the translation server comprising;

a request queue operable to store requests for language model data to be obtained for translating a segment in the source language, and

a segment translation server cache operable to store language model data obtained by the requests by the translation server.

55. The system of claim 54, wherein:

the segment translation server cache is operable to delete the obtained language model data after the segment is translated.

56. The system of claim 54, wherein:

the segment translation server cache is operable to delete the obtained language model data periodically.

57. The system of claim 54, wherein

the translation server is operable to process translation of the segment before all of the requests in the request queue are served, and

the translation server is further operable to finalize translation of the segment using the language model data in the segment translation server cache obtained by the requests.

58. The system of claim 54, wherein

the translation server is operable to process the translation of the segment using language model data from a second language model for the target language while the requests for the language data in the language model in the request queue are being served and language model data obtained by the requests is being stored in the segment translation server cache, and

after all of the requests in the request queue are served, the translation server is further operable to process the translation, which is generated by using the language model data from the second language model, with the language model data obtained by the requests.

59. The system of claim 54, wherein:

the translation server is operable to process the translation of the segment using language model data from a second language model for the target language to produce an initial translation of the segment before the requests for the language data in the language model in the request queue are sent out,

the translation server is operable to update the requests for the language model data of the language model in the request queue based on the initial translation,

the translation server is further operable to send out the updated requests in the request queue to obtain language model data from the language model for processing the initial translation, and

after the updated requests are served and the data for the updated requests are stored in the segment translation server queue, the translation server is further operable to process the initial translation with the data for the updated requests to produce a final translation.

60. The system of claim 54, wherein:

the segment translation server cache is further operable to store history information of translation of an assigned segment.

61. The system of claim 54, wherein:

the translation server further comprises;

a second segment translation server cache storing a selected portion of the language model,

wherein the translation server is operable to, after completing translation of the segment, delete data in the segment translation server cache and retain the selected portion of the language model in the second segment translation server cache.

62. The system of claim 54, wherein:

the translation server further comprises;

a second segment translation server cache storing a selected portion of the language model,

wherein the translation server is operable to periodically delete data in the segment translation server cache.

63. The system of claim 61, wherein:

the translation server is operable to look up the second segment translation server cache for a piece of language model data needed for translating the segment before generating a request for the piece of language model data, and

when the piece of language model data is present in the second segment translation server cache, the translation server is operable to use the piece of language model data for translation without generating the request for the piece of language model data.

64. The system of claim 54, further comprising:

a communication network with which the translation server is in communication to receive the text in the source language from a client computer in the communication network.

65. A method, comprising:

dividing a collection of machine language translation resource data for translation from a source language to a target language into a plurality of partitions each being less than the collection of machine language translation resource data;

storing the plurality of partitions on different computer servers, respectively; and

operating a machine translation server to access and use the collection of machine language translation resource data on the different computer servers to perform translation from the source language into the target language.

66. The method of claim 65, wherein:

the collection of machine language translation resource data comprises a language model for the target language which further includes n-grams with n greater than 3.

67. The method of claim 65, wherein:

the collection of machine language translation resource data comprises a translation model between the target language and the source language.

68. The method of claim 65, further comprising:

providing different translation servers to respectively translate different segments of a client document to be translated from the source language into the target language; and

operating each translation server to obtain selected data from the collection of machine language translation resource data for translating each respective segment.

69. The method of claim 68, further comprising:

configuring at least two different translation servers to operate at different translation speeds and different translation quality levels; and

selecting a translation server for translating a segment based on a speed and a translation quality level for the segment.

70. The method of claim 68, further comprising:

operating each translation server to generate requests for the selected data in a request queue; and

sequentially sending the requests to the computer servers to obtain the selected data.

71. The method of claim 70, further comprising:

operating each translation server to wait for all the requests in the request queue to be served before translating a respective segment.

72. The method of claim 70, further comprising:

operating each translation server to perform an initial translation of a respective segment when the requests in the request queue are being served; and

after all the requests in the request queue are served, updating the initial translation to produce a final translation using the selected data.

73. The method of claim 72, further comprising:

using another different collection of machine language translation resource data to perform the initial translation of the respective segment.

74. The method of claim 68, further comprising:

operating each translation server to generate requests for the selected data in a request queue;

without sending the requests to the computer servers and replica computer servers, operating each translation server to apply a second, different collection of machine language translation resource data to process translation of the respective segment;

using information from processing with the second, different collection of machine language translation resource data to update the requests for the selected data in the request queue;

sending the updated requests to the computer servers and replica computer servers to obtain data; and

using data requested by the updated requests to complete translation of the respective segment.

75. A method for machine translation of text from a source language into a target language using a translation model for translation between the source language and the target language and a language model for the target language, comprising:

partitioning the translation model into a plurality of partitions of different data, wherein each translation model partition is less than the translation model;

storing the plurality of translation model partitions on a plurality of different translation model servers;

partitioning the language model into a plurality of language model partitions of different data, wherein each language model partition is less than the language model;

storing the plurality of language model partitions on a plurality of different language model servers;

monitoring work load of a plurality of translation severs each operable to translate text in the source language into the target language using the translation model and the language model;

distributing segments of a text to be translated from the source language into the target language to one or more selected translation servers from the plurality of translation servers based on the work load;

operating each selected translation server to access the translation model severs and the language model servers to fetch desired translation model data and language model data for each respective segment to be translated; and

compiling translated segments from the selected translation servers to produce a translated text.

76. The method of claim 75, further comprising:

controlling an amount of processing computation at each of the selected translation servers to adjust a speed in completing the translation of the text.

77. The method of claim 76, further comprising:

controlling the selected translation servers to produce an initial translation of the text of a low translation quality; and

after completion of the initial translation, controlling the selected translation servers to continue processing the translation to produce a second translation of the text of a high translation quality.

78. The method of claim 77, further comprising:

delivering the initial translation of the text to a client requesting the translation at a first time; and

delivering the second translation of the text to the client to replace the initial translation at a second time.

79. The method of claim 77, further comprising:

selecting a first part of the text to be translated by the selected translation servers at the high translation quality; and

selecting a second part of the text to be translated by the selected translation servers at the low translation quality.

80. A computer implemented method, comprising:

receiving a client document in a source language to be translated into a target language;

dividing the client document into segments to translate each segment;

accessing at least one of a plurality of different language model servers, which collectively store a language model for the target language, to retrieve selected language model data related to translation one of the segments, each language model server storing and operable to serve a partition of the language model; and

translating the segment into the target language using the retrieved selected language model data.

81. The method of claim 80, further comprising:

during accessing a first language model server for at least one portion of the selected language model data related to translation of the segment under translation, simultaneously accessing a first replica server of the first language model server, which replicates data on the first language model server, to retrieve the at least one portion of the selected language model data; and

using data from either one of the first language model server and the first replica server, whichever arrives first, in translating the segment.

82. The method of claim 80, further comprising:

setting a timeout period for accessing each language model server;

after the timeout period expires for accessing a particular language model server prior to receiving requested data from the particular language model server, obtaining the data requested from the particular language model from a different data source.

83. The method of claim 82, wherein:

the different data source includes the data from a different language model.

84. The method of claim 80, further comprising:

setting a timeout period for accessing each language model server; and

after the timeout period expires for accessing a particular language model server prior to receiving requested data from the particular language model server, initiating access to a replica server of the particular language model server, which replicates data on the particular language model server, to obtain the data requested from the particular language model.

85. The method of claim 80, further comprising:

accessing a translation model server which stores a translation model for translation between the source language and the target language, to retrieve selected translation model data related to translation one of the segments; and

translating the segment into the target language using the retrieved selected language model data and translation model data.

86. The method of claim 80, further comprising:

prior to translating each segment using data from the language model, performing an initial translation of the segment using data from a different translation resource; and

using data from the language model to update the initial translation to produce a final translation of the segment.

87. The method of claim 86, wherein:

the different translation resource is a second language model for the target language that is smaller than the language model.

88. The method of claim 80, further comprising:

prior to accessing at least one of a plurality of different language model servers for retrieving data for translating one segment, searching a local cache which stores translations of predetermined segments to determine whether an existing translation is available for the segment; and

using the existing translation for the segment if the existing translation is available, and if not, accessing the at least one of a plurality of different language model servers for retrieving data for translating the segment.

89. A method comprising:

is using a machine translation system to receive text in a source language from a client and to translate the text into a target language,

wherein the translating in the machine translation system comprises;

selecting a portion of the text to translate at a low translation quality to produce an initial translated portion while translating the selected portion at a high translation quality;

delivering the initial translated portion to the client while continuing translating the selected portion at the high translation quality; and

after the selected portion is translated into a second translated portion at the high translation quality, delivering the second translated portion at the high translation quality to the client to automatically replace the initial translated portion.

90. The method of claim 89, wherein

the machine translation system comprises a first translation server producing a translation at the low translation quality and a second translation server producing a translation at the high translation quality and operating slower the first translation server, and

the translating in the machine translation system comprises:

using the first translation server to produce the initial translated portion and the second translation server to produce the second translated portion.

91. The method of claim 89, wherein:

the translating in the machine translation system comprises;

using an existing translation for the selected portion stored in a cache of the machine translation system as the initial translated portion;

accessing at least one of a plurality of different language model servers, which collectively store a language model for the target language, to retrieve selected language model data related to translating the selected portion at the high translation quality, wherein each language model server stores and is operable to serve a partition of the language model; and

after receiving the selected language model data, translating the selected portion into the second translated portion using the selected language model data.