Searching a multi-lingual database

• US 8,027,994 B2
• Filed: 08/21/2008
• Issued: 09/27/2011
• Est. Priority Date: 02/01/2002
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Leon Rodriguez, Juan, Kozakov, Lev, Kim, Moon Ju, Drissi, Youssef
• Primary Examiner(s): Woo; Isaac M
• Application Number:
  US12/195,862
• Publication Number:
  US 20080306923A1
• Time in Patent Office: 1,132 Days
• Field of Search: 707600-831
• US Class Current: 707/791
• CPC Class Codes: G06F 16/319 Inverted lists
• CPC Class Codes: G06F 16/3337 Translation of the query language, e.g. Chinese to English
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99937 Sorting
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99939 Privileged access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

A system and method of searching a database in which documents of different languages are included. The system includes a synonym or keyword dictionary which is bi-directional and allows for translation of keywords between a first language and other languages. The translated words keywords for the document are stored in an inverted index which is then used for searching, either in a selected language, a second language or in all languages, as determined by the user. This use of multiple searching and a translated synonym dictionary avoids the need for translation of the entire document and avoids inaccuracies which may result from translations.

Claims

1. A computerized method of searching documents written in a plurality of languages, the method comprising:

receiving a first query that includes at least one keyword in a first language;

generating a second query by translating the at least one keyword into a second language;

applying the first query against documents including at least one document written in the first language and at least one document written in the second language with the at least one keyword in the first language;

applying the second query against documents written in the second language;

generating a first set of results based on the first query, wherein the first set of results includes each document written in the first language that matches the first query; and

generating a second set of results based on the first and second queries, wherein the second set of results includes each document written in the second language that matches at least one of the first query or the second query.

2. The method of claim 1, wherein the generating a second query uses a keyword dictionary that supports translation between the plurality of languages.

3. The method of claim 2, wherein the keyword dictionary includes a synonym listing that translates a keyword in any of the plurality of languages into a corresponding keyword in another of the plurality of languages.

4. The method of claim 1, wherein the generating a second query translates the at least one keyword using the Unicode system.

5. The method of claim 1, further comprising providing at least one of the first and second sets of results for display to a user.

6. The method of claim 1, wherein the applying the first query includes:

applying the first query against an inverted index in the first language; and

applying the first query against an inverted index in the second language.

7. The method of claim 1, further comprising:

identifying keywords from each of the plurality of documents;

translating each identified keyword into each of the plurality of languages; and

creating an index in each of the plurality of languages, wherein the applying the first query uses at least one of the plurality of indexes.

8. The method of claim 7, further comprising creating a single inverted index from the translated keywords.

9. The method of claim 7, wherein the translating uses a keyword dictionary that supports translation between the plurality of languages.

10. The method of claim 1, further comprising determining a native language of a user.

11. A computer system comprising:

a system for searching documents written in a plurality of languages, the system comprising at least one computer, wherein the searching is implemented using a method including;

receiving a first query that includes at least one keyword in a first language;

generating a second query by translating the at least one keyword into a second language;

applying the first query against documents including at least one document written in the first language and at least one document written in the second language with the at least one keyword in the first language;

applying the second query against documents written in the second language;

generating a first set of results based on the first query, wherein the first set of results includes each document written in the first language that matches the first query; and

generating a second set of results that includes each document written in the second language based on the first and second queries, wherein the second set of results matches at least one of the first query or the second query.

12. The system of claim 11, wherein the generating a second query uses a keyword dictionary that supports translation between the plurality of languages.

13. The system of claim 11, the method further including providing at least one of the first and second sets of results for display to a user.

14. The system of claim 11, the method further including determining a native language of a user.

15. The system of claim 11, the method further including:

identifying keywords from each of the plurality of documents;

translating each identified keyword into each of the plurality of languages; and

creating an index in each of the plurality of languages, wherein the system for applying the first query uses at least one of the plurality of indexes.

16. A document searching program stored on a computer-useable medium, which causes a computer system to perform a method when executed on the computer system, wherein the documents are written in a plurality of languages, the method comprising:

receiving a first query that includes at least one keyword in a first language;

generating a second query by translating the at least one keyword into a second language;

applying the first query against documents including at least one document written in the first language and at least one document written in the second language with the at least one keyword in the first language;

applying the second query against documents written in the second language;

generating a first set of results based on the first query, wherein the first set of results includes each document written in the first language that matches the first query; and

generating a second set of results that includes each document written in the second language based on the first and second queries, wherein the second set of results matches at least one of the first query or the second query.

17. The program of claim 16, wherein the generating a second query uses a keyword dictionary that supports translation between the plurality of languages.

18. The program of claim 16, the method further comprising a providing at least one of the first and second sets of results for display to a user.

19. The program of claim 16, the method further comprising a determining a native language of a user.

20. The program of claim 16, the method further comprising:

identifying keywords from each of the plurality of documents;

translating each identified keyword into each of the plurality of languages; and

creating an index in each of the plurality of languages, wherein the applying the first query uses at least one of the plurality of indexes.