Systems, methods, software, and interfaces for multilingual information retrieval

• US 20060173886A1
• Filed: 01/04/2006
• Published: 08/03/2006
• Est. Priority Date: 01/04/2005
• Status: Active Grant

• Current Assignee: Thomson Reuters Enterprise Centre GmbH (The Woodbridge Co. Ltd.)
• Original Assignee: Thomson Reuters Global Resources Unlimited Company (The Woodbridge Co. Ltd.)
• Inventors: Lund, Elizabeth S., Moulinier, Isabelle
• Granted Patent: US 9,418,139 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/3337 Translation of the query language, e.g. Chinese to English
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

The present inventors have a devised one or more novel methods, systems, and interfaces for facilitating multi-lingual searches. One exemplary method entails creating multiple language-specific indices for a collection of documents, with each index including stemmed and non-stemmed versions of terms from the documents. Users submit queries that are associated with a set of one or more target languages. Query processing entails translating original and stemmed versions of each term in a query into each of the target languages, using one or more techniques that each yield a set of potentially equivalent query terms. Each set of potentially equivalent query terms is then processed against the corresponding language-specific index, using a conventional monolingual search technique, such as a Boolean or natural language query, to identify documents from the collection. The resultant documents are presented to the user in language groupings or by computed relevance.

Claims

1. A method comprising:

defining a set of one or more language-specific indices for a collection of documents, with each index including stemmed and non-stemmed versions of terms contained in the documents;

receiving a query from a user, with the query associated with a set of one or more target languages;

parsing the query into one or more terms, with each term associated with a corresponding language identifier and a stemmed version of the term;

translating the original and stemmed versions of each term into each of the target languages to define respective sets of one or more equivalent query terms; and

identifying a set of documents from the collection of documents for each of the target languages, with each set identified based on the equivalent query terms for the corresponding target language.

2. The method of claim 1, wherein each term in each index is associated with a document identifier for uniquely identifying one of the documents in the collection, a positional indicator for indicating a position of the term in the one of the documents, and a language indicator for indicating language of the term.

3. The method of claim 1, wherein before receiving the query from a user, the user defines the query by interacting with a graphical user interface having a query submission screen having a control region for entering terms of the query;

a control region for selecting one or more of the target language; and

a control region for submitting the query.

4. The method of claim 1, wherein the query is associated with a set of one or more target languages selected by the user.

5. The method of claim 1, wherein translating the original and stemmed versions of each term into two or more equivalent query terms in one of the target languages, includes identifying equivalent query terms using at least two of the following:

a database of pre-translated documents, an electronic language dictionary, an automated translator, and a pivot language.

6. The method of claim 1:

wherein identifying the set of documents from the collection for each of the target languages, includes;

determining a relevance score for documents based on the equivalent query terms; and

identifying documents having a relevance score exceeding a predetermined threshold; and

wherein the method further comprises presenting a graphical user interface listing the identified set of documents in groups based on corresponding language and/or in rank order of relevance.

7. A system comprising:

a collection of documents;

a set of one or more language-specific indices for the collection of documents, with each index including stemmed and non-stemmed versions of terms contained in the documents; and

a server for interacting with the collection of documents and the set of language-specific indices, with the server configured;

to receive a query from a user, with the query associated with a set of one or more target languages;

to parse the query into one or more terms, with each term associated with a corresponding language identifier and a stemmed version of the term;

to translate the original and stemmed versions of each term into each of the target languages and thus define respective sets of one or more equivalent query terms; and

to identify a set of documents from the collection of documents for each of the target languages, with each set identified based on the equivalent query terms for the corresponding target language.

8. The system of claim 7, wherein each term in each index is associated with a document identifier for uniquely identifying one of the documents in the collection, a positional indicator for indicating a position of the term in the one of the documents, and a language indicator for indicating language of the term.

9. The system of claim 7, wherein the server is further configured to define a graphical user interface for enabling a user to submit a query, the interface having a query submission screen having a control region for entering terms of the query;

a control region for selecting one or more of the target language; and

a control region for submitting the query.

10. The system of claim 7, wherein to translate the original and stemmed versions of each term into two or more equivalent query terms in one of the target languages, the server is further configured to identify equivalent query terms using at least two of the following:

a database of pre-translated documents, an electronic language dictionary, an automated translator, and a pivot language.

11. The system of claim 7, wherein the server includes a processor and a memory, and the memory includes coded instructions for causing the processor:

to receive the query from a user, to parse the query into one or more terms, to translate the original and stemmed versions of each term into each of the target languages and thus define respective sets of one or more equivalent query terms, and to identify a set of documents from the collection of documents for each of the target languages.

12. A server for interacting with a collection of documents and a set of language-specific indices, with the server configured:

to receive a query from a user, with the query associated with a set of one or more target languages;

to parse the query into one or more terms, with each term associated with a corresponding language identifier and a stemmed version of the term;

to translate the original and stemmed versions of each term into each of the target languages and thus define respective sets of one or more equivalent query terms; and

to identify a set of documents from the collection of documents for each of the target languages, with each set identified based on the equivalent query terms for the corresponding target language.

13. The server of claim 12, wherein each term in each index is associated with a document identifier for uniquely identifying one of the documents in the collection, a positional indicator for indicating a position of the term in the one of the documents, and a language indicator for indicating language of the term.

14. The server of claim 12, wherein the server is further configured to define a graphical user interface for enabling a user to submit a query, the interface having a query submission screen having a control region for entering terms of the query;

a control region for selecting one or more of the target language; and

a control region for submitting the query.

15. The server of claim 12, wherein to translate the original and stemmed versions of each term into two or more equivalent query terms in one of the target languages, the server is further configured to identify equivalent query terms using at least two of the following:

a database of pre-translated documents, an electronic language dictionary, an automated translator, and a pivot language.

16. The server of claim 12, wherein the server includes a processor and a memory, and the memory includes coded instructions for causing the processor:

to receive the query from a user, to parse the query into one or more query terms, to translate original and stemmed versions of each query term into each of the target languages and thus define respective sets of one or more equivalent query terms, and to identify a set of documents from the collection of documents for each of the target languages.

17. A machine-readable medium for causing a server to interact with a collection of documents and a set of language-specific indices, with the medium comprising instructions for causing the server:

to receive a query from a user, with the query associated with a set of one or more target languages;

to parse the query into one or more terms, with each term associated with a corresponding language identifier and a stemmed version of the term;

to translate the original and stemmed versions of each term into each of the target languages and thus define respective sets of one or more equivalent query terms; and

to identify a set of documents from the collection of documents for each of the target languages, with each set identified based on the equivalent query terms for the corresponding target language.

18. The medium of claim 17, wherein each term in each index is associated with a document identifier for uniquely identifying one of the documents in the collection, a positional indicator for indicating a position of the term in the one of the documents, and a language indicator for indicating language of the term.

19. The medium of claim 17, further including instructions to define a graphical user interface for enabling a user to submit a query, the interface having a query submission screen having a control region for entering terms of the query;

a control region for selecting one or more of the target language; and

a control region for submitting the query.

20. The medium of claim 17, wherein the instructions for causing the server to translate the original and stemmed versions of each term into two or more equivalent query terms in one of the target languages, include instructions to identify equivalent query terms using at least two of the following:

a database of pre-translated documents, an electronic language dictionary, an automated translator, and a pivot language.