QUERY TRANSLATION THROUGH DICTIONARY ADAPTATION

• US 20100070521A1
• Filed: 09/18/2008
• Published: 03/18/2010
• Est. Priority Date: 09/18/2008
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Clinchant, Stephane, Renders, Jean-Michel
• Granted Patent: US 8,775,154 B2
• Time in Patent Office: Days
• US Class Current: 707/760
• CPC Class Codes: G06F 16/3337 Translation of the query language, e.g. Chinese to English

Abstract

Cross-lingual information retrieval is disclosed, comprising: translating a received query from a source natural language into a target natural language; performing a first information retrieval operation on a corpus of documents in the target natural language using the translated query to retrieve a set of pseudo-feedback documents in the target natural language; re-translating the received query from the source natural language into the target natural language using a translation model derived from the set of pseudo-feedback documents in the target natural language; and performing a second information retrieval operation on the corpus of documents in the target natural language using the re-translated query to retrieve an updated set of documents in the target natural language

Claims

1. A method of performing cross-lingual information retrieval, the method comprising:

translating a query in a source natural language into a target natural language different from the source natural language to generate a starting query in the target natural language;

performing a first information retrieval operation on a corpus of documents in the target natural language using the starting query in the target natural language to retrieve a set of pseudo-feedback documents in the target natural language;

generating an updated query in the target natural language based on the set of pseudo-feedback documents in the target natural language; and

performing a second information retrieval operation on the corpus of documents in the target natural language using the updated query in the target natural language to retrieve an updated set of documents in the target natural language.

2. The method as set forth in claim 1, wherein the translating employs a dictionary defining candidate translation terms corresponding to query terms of the query in the source natural language, and the generating an updated query in the target natural language comprises:

computing query term translation probabilities relating source language query terms with candidate target language translation terms, the computing being based on query context information provided by the set of pseudo-feedback documents in the target natural language; and

generating the updated query in the target natural language based on the computed query term translation probabilities.

3. The method as set forth in claim 1, wherein the generating an updated query in the target natural language comprises:

deriving source term-target term pair translation probabilities from the pseudo-feedback documents; and

re-translating the query in the source natural language into the target natural language using the derived source term-target term pair translation probabilities.

4. The method as set forth in claim 3, wherein the deriving of the source term-target term pair translation probabilities comprises:

adjusting the source term-target term pair translation probabilities to maximize a likelihood of the pseudo-feedback documents given the source query.

5. The method as set forth in claim 4, wherein the adjusting employs an expectation maximization (EM) algorithm.

6. The method as set forth in claim 1, further comprising:

outputting the updated set of documents in the target natural language in a user-perceptible output format.

7. The method as set forth in claim 1, further comprising:

translating the updated set of documents in the target natural language into the source natural language; and

outputting the translated updated set of documents in a user-perceptible output format.

8. The method as set forth in claim 1, wherein:

the query in the source natural language is represented as a starting query language model in the source natural language; and

the generating an updated query generates the updated query represented as an updated query language model in the target natural language.

9. The method as set forth in claim 8, wherein the generating an updated query represented as an updated query language model in the target natural language comprises computing the updated query language model to be equivalent to:

$p<br><br>\left(w_t|q_s\right)=\underset{w_s}{\sum <br><br>}<br><br>{\mathrm{\theta <br><br>}}_{\mathrm{st}}^{\mathrm{adapted}}<br><br>P<br><br>\left(w_s|q_s\right)$

where P(w_s|q_s) is the starting query language model in the source natural language, θ

_st ^adapted represents a translation model generated by maximizing a likelihood of the set of pseudo-feedback documents in the target natural language given the query in the source natural language, and p(w₁|q_s) is equal or equivalent to the updated query language model.

10. The method as set forth in claim 8, wherein the generating an updated query represented as an updated query language model in the target natural language comprises:

deriving source term-target term pair translation probabilities from the pseudo-feedback documents; and

transforming the starting query language model in the source natural language based on the source term-target term pair translation probabilities to generate the updated query language model in the target natural language.

11. The method as set forth in claim 10, wherein the transforming does not utilize the starting query language model in the target natural language.

12. A cross-lingual information retrieval system comprising:

a monolingual information retrieval system configured to retrieve documents from a corpus of documents in a target natural language based on a received query in the target natural language; and

a processor configured to add cross-lingual information retrieval capability to the monolingual information retrieval system, the processor performing a process including;

translating a query from a source natural language into the target natural language and inputting the translated query to the monolingual information retrieval system to retrieve a set of pseudo-feedback documents in the target natural language,

generating a source language-to-target language translation model based on the set of pseudo-feedback documents, and

re-translating the query from the source natural language into the target natural language using the translation model and inputting the re-translated query to the monolingual information retrieval system to retrieve an updated set of documents in the target natural language.

13. The cross-lingual information retrieval system as set forth in claim 12, wherein the generating comprises:

adjusting source term-to-target term translation probability parameters to maximize a likelihood of the set of pseudo-feedback documents given the query.

14. The cross-lingual information retrieval system as set forth in claim 12, wherein the processor receives the query as or converts the query to a source language query model indicative of likelihood that source language vocabulary terms belong to the source language query, and the re-translating represents the re-translated query as a target language query model indicative of likelihood that target language vocabulary terms belong to the query.

15. The cross-lingual information retrieval system as set forth in claim 12, wherein the processor configured to add cross-lingual information retrieval capability to the monolingual information retrieval system is also configured to embody the monolingual information retrieval system.

16. The cross-lingual information retrieval system as set forth in claim 12, further comprising:

an output device configured to represent the updated set of documents in a human-perceptible representation.

17. The cross-lingual information retrieval system as set forth in claim 12, further comprising:

a machine translation engine configured to translate the updated set of documents from the target natural language to the source natural language; and

an output device configured to represent the machine-translated updated set of documents in a human-perceptible representation.

18. A storage medium storing instructions executable to perform a cross-lingual information retrieval method comprising:

translating a received query from a source natural language into a target natural language;

performing a first information retrieval operation on a corpus of documents in the target natural language using the translated query to retrieve a set of pseudo-feedback documents in the target natural language;

re-translating the received query from the source natural language into the target natural language using a translation model derived from the set of pseudo-feedback documents in the target natural language; and

performing a second information retrieval operation on the corpus of documents in the target natural language using the re-translated query to retrieve an updated set of documents in the target natural language.

19. The storage medium as set forth in claim 18, wherein the re-translating comprises:

maximizing a likelihood of the set of pseudo-feedback documents given the received query respective to a set of source language term-to-target language term translation pair probabilities defining the translation model.

20. The storage medium as set forth in claim 18, wherein the performed information retrieval method further comprises:

second re-translating the received query from the source natural language into the target natural language using an updated translation model derived from the updated set of documents in the target natural language; and

performing a third information retrieval operation on the corpus of documents in the target natural language using the second re-translated query to retrieve a further updated set of documents in the target natural language.