Method and system for searching a multi-lingual database

• US 20030149686A1
• Filed: 02/01/2002
• Published: 08/07/2003
• Est. Priority Date: 02/01/2002
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kozakov, Lev, Kim, Moon Ju, Drissi, Youssef, Leon Rodriguez, Juan
• Granted Patent: US 6,952,691 B2
• Time in Patent Office: Days
• US Class Current: 707/3
• CPC Class Codes: G06F 16/319 Inverted lists
• CPC Class Codes: G06F 16/3337 Translation of the query language, e.g. Chinese to English
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99937 Sorting
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99939 Privileged access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

A system and method of searching a database in which documents of different languages are included. The system includes a synonym or keyword dictionary which is bidirectional and allows for translation of keywords between a first language and other languages. The translated words keywords for the document are stored in an inverted index which is then used for searching, either in a selected language, a second language or in all languages, as determined by the user. This use of multiple searching and a translated synonym dictionary avoids the need for translation of the entire document and avoids inaccuracies which may result from translations.

Claims

1. A method of creating and searching a database of multi-lingual documents, the method comprising the steps of:

using a keyword dictionary to extract keywords;

translating the keywords into the languages supported by a bi-directional synonym listing;

creating a keyword listing in each supported language;

using the keywords in each supported language to create more than one inverted index of the documents including the keywords, one inverted index being in a first language and another inverted index being in a second language;

retrieving documents in a search mode comprising the steps of translating the search terms using the same bi-directional synonym listing, generating a translated query, submitting parallel inquiries to first and second different inverted indices, receiving first and second results of the search using the first and second inverted indices and selecting one of the first and second results for use.

2. A method including the steps of claim 1 wherein the step of using the keywords to create an inverted index includes the step of using the Unicode system to manage multiple languages.

3. A method of searching a database including the steps of claim 1 wherein the step of generating an inquiry includes identifying the language in which it is presented.

4. A method of searching a database including documents written in more than one language, the steps of the method comprising:

identifying a search in one language;

converting the search into another language;

searching the database in both the one language and the another language and generating a first list of results associated with the one language and a second list of results from the another language; and

providing the first and second lists of search results as output.

5. The method of claim 4 wherein the step of converting the search into another language includes the step of using a bidirectional keyword dictionary.

6. The method of claim 5 wherein the method further includes the step of. selecting one of the first list of results and the second list of results as the output of the search.

7. The method of claim 4 wherein the step of converting a search term into another language includes the step of transposing it to the Unicode system.

8. A system for searching a database based on an input search, the system comprising:

a synonym list for keywords which includes corresponding words in at least a first and a second language;

an inverted index for the database including keywords and synonyms for keywords based on the synonym list;

an indicator of the language associated with the input search along with a selection of whether documents are desired in the natural language or in all languages;

a search engine responsive to the natural language indication and the selection of documents, said search engine using the inverted index to determine a first set of results which include the documents which correspond to the input search in the input language and a second set of results which correspond to the documents which include documents which are not in the input language.

9. A system of the type described in claim 8 wherein the synonym list is a two-way synonym list which is usable in converting a search time from a first language to a second language and from a second language to a first language.

10. A searching system of the type described in claim 8 wherein the converting of search terms uses the Unicode system for managing multiple languages.

15. A database searching program stored on a medium comprising:

a first module which includes a bi-directional synonym listing of terms;

a second module which receives text and selects one or more keywords;

a third module which receives the one or more keywords and translates the one or more keywords into another language;

a fourth module which searches a database using the keywords in one language and the translations of keywords using the translations of the third module, said fourth module creating a first set of results using a first language and a second set of results which includes documents in a language other than the first language; and

a fifth module which receives an indication from the user as to whether results in the first language or results including documents in a second language are desired

16. A program including the elements of claim 15 wherein the third module of the program which translates words into another language uses the Unicode system for managing multiple languages.

17. A program including the elements of claim 15 where the program further includes a module for identifying the language of the user and uses the language of the user in the third module which translates the keywords.