Translation system adapted for query translation via a reranking framework

• US 8,713,037 B2
• Filed: 06/30/2011
• Issued: 04/29/2014
• Est. Priority Date: 06/30/2011
• Status: Expired due to Fees

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Nikoulina, Vassilina, Lagos, Nikolaos
• Primary Examiner(s): Perveen, Rehana
• Assistant Examiner(s): Hershley, Mark E
• Application Number:
  US13/173,582
• Publication Number:
  US 20130006954A1
• Time in Patent Office: 1,034 Days
• Field of Search: 707/760
• US Class Current: 707/760
• CPC Class Codes: G06F 16/3337 Translation of the query language, e.g. Chinese to English
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06F 40/45 Example-based machine translation; Alignment
• CPC Class Codes: G06F 40/51 Translation evaluation

Abstract

A system and method suited to translation of queries are disclosed. The method includes receiving an input query in a source language and outputting a target query, based on a translation of the input query into a target language, different from the source language. The translation is performed with a machine translation system which has been trained on representations of features of translated queries that have been generated by translation of an original query, in the source language, into the target language and a measure of information retrieval performance of each the translated queries, for each of a set of original queries.

Claims

1. A translation method comprising:

receiving an input query in a source language; and

outputting a target query, the target query being identified from a set of candidate target queries, each target query being based on a translation of the input query into a target language, different from the source language, with a machine translation system which includes a reranking model for ranking the candidate target queries, which has been trained by a method which includes;

for each of a plurality of training queries in the source language, translating the training query in the source language into the target language to generate translated queries which are each a translation of the respective training query;

for each of the translated queries;

computing a feature representation of the translated query;

retrieving a set of annotated documents from a document collection in response to the translated query, the documents in the retrieved set of annotated documents including annotations that are based on responsiveness of each of the documents to each of the training queries, and

computing a precision score for the translated query based on relevance scores of the retrieved documents in the set of annotated documents, each of the relevance scores being based on the annotations of the documents in the retrieved set of annotated documents; and

learning feature weights for the reranking model based on the precision scores and feature representations of the translated queries.

2. The method of claim 1, wherein the machine translation system is implemented with a computer processor.

3. The method of claim 1, wherein the outputting comprises outputting the target query to a search engine and retrieving information based on the target query.

4. The method of claim 1, wherein the outputting of the target query comprises, for each of a set of candidate queries generated by the machine translation system as a translation of the input query in the target language, generating a features vector based on features extracted from the candidate query.

5. The method of claim 1, wherein the information retrieval performance of each translated query is determined with respect to at least one of the training query and a reference translation thereof in the target language.

6. The method of claim 1, wherein the method includes training the machine translation system.

7. The method of claim 1, wherein the learning of the feature weights is performed with a margin infused relaxed algorithm.

8. The method of claim 1, wherein at least one of the features is a feature which is not used in generating the translation of the input query.

9. The method of claim 1, wherein the outputting comprises:

translating the input query with a decoder of the machine translation system to generate a set of candidate queries in the target language;

extracting features for each of the candidate queries in the set; and

ranking the candidate queries based on a representation of the extracted features for each candidate query and feature weights for the features in the reranking model.

10. The method of claim 9, wherein the ranking comprises computing a function of the extracted features and the learned feature weights for the features.

11. The method of claim 10, wherein the computing of the function comprises computing a translation score for each of the candidate queries as a weighted linear combination of its extracted features.

12. The method of claim 11, wherein the method includes outputting, as the target query, a candidate query which satisfies:

{circumflex over (t)}=argmax<sub>tiε

GEN(q)</sub>Λ

·

F(t_i) 

 

(Eqn.

     1)

where t_iε

(GEN(q)) represents a candidate query generated from a source query q, Λ

represents a set of feature weights learned in training, one weight for each of the features in F(t_i) and ·

represents the dot product.

13. The method of claim 9, wherein at least one of the extracted features is based on parts of speech for the candidate queries.

14. The method of claim 13, wherein at least one of the part of speech features is based on a part of speech for an element of the candidate query and a corresponding part of speech of an element of the input query with which the element of the candidate query is aligned in the translation, wherein each of the elements comprises at least one word.

15. The method of claim 14, wherein the at least one of the part of speech features is also based on a frequency of the element in the candidate query as a translation of element of the input query in a training corpus of bi-sentences, each bi-sentence including a sentence in the source language and a sentence in the target language.

16. The method of claim 9, wherein the outputting of the target query comprises outputting a most highly ranked candidate query as the target query or outputting a combination of more highly ranked candidate queries as the target query.

17. The method of claim 9, wherein the translating of the input query with the machine translation system comprises:

retrieving a set of biphrases, each biphrase comprising at least one word of the input query in the source language and at least one corresponding word in the target language; and

with a scoring function, computing a set of the retrieved biphrases to cover the input query, for each of the set of candidate queries, each candidate query comprising the corresponding words in the target language forming the set of retrieved biphrases.

18. A computer program product comprising a non-transitory computer-readable recording medium which stores instructions for performing the method of claim 1.

19. A translation system comprising non-transitory memory which stores instructions for performing the method of claim 1 and a processor in communication with the memory for executing the instructions.

20. A query translation system comprising:

a decoder which receives a source query in a source language and outputs a set of candidate queries in a target language, each of the candidate queries being a translation of the same source query;

a reranking module which outputs a target query based on at least one of the candidate queries, the reranking module extracting features of each of the candidate queries and computing a function in which the extracted features are weighted by feature weights, the feature weights having been learned on features of each of a set of translated queries generated by translation of each of a set of training queries into the target language and a precision score computed for each of the translated queries, the computing of the precision for each of the translated queries including retrieving a set of annotated documents from a document collection in response to the translated query, the documents in the retrieved set of annotated documents including annotations that are based on responsiveness of each of the documents to each of the training queries, and computing the precision score for the translated query based on relevance scores of the retrieved documents in the set of annotated documents, each of the relevance scores being based on the annotations of the documents in the retrieved set of annotated documents; and

a processor which implements the reranking module.

21. The query translation system of claim 20, wherein the decoder comprises a phrase-based statistical machine translation system, each of the candidate queries being a translation of the same source query generated by retrieving biphrases from a biphrase library, each retrieved biphrase comprising at least one word of the source query in the source language and at least one corresponding word in the target language, the decoder computing a set of the retrieved biphrases to cover the input query for each of the candidate queries, each candidate query comprising the corresponding words in the target language forming the respective set of retrieved biphrases.

22. A method for training a translation system for translation of queries, comprising:

for each of a plurality of training queries in a source language;

translating the training query to generate a set of translated queries in a target language;

for each translated query in the set of translated queries;

extracting values of features for each of a finite set of features;

retrieving a set of annotated documents from a document collection in response to the translated query, the documents in the retrieved set of annotated documents including annotations that are based on responsiveness of each of the documents to each of the training queries; and

computing a precision score for the translated query based on relevance scores of the retrieved documents in the set of annotated documents, each of the relevance scores being based on the annotations of the documents in the retrieved set of annotated documents;

learning feature weights for each of the features based on the extracted values of the features and the respective precision score of each translated query; and

storing the feature weights for use in translating a new query, different from each of the training queries, from the source language to the target language, whereby candidate translations of the new query are ranked based on their extracted values of features and the stored feature weights.

23. A computer program product comprising a non-transitory computer-readable recording medium which stores instructions for performing the method of claim 22.

24. A computer system comprising non-transitory memory which stores instructions for performing the method of claim 22 and a processor in communication with the memory for executing the instructions.