System and method for cross-language knowledge searching

• US 7,672,831 B2
• Filed: 10/24/2005
• Issued: 03/02/2010
• Est. Priority Date: 10/24/2005
• Status: Active Grant

• Current Assignee: Allium US Holding LLC
• Original Assignee: Invention Machine Corporation (S&P Global Inc.)
• Inventors: Zhyhalko, Heorhi, Sovpel, Igor, Todhunter, James
• Primary Examiner(s): Dorvil; Richemond
• Assistant Examiner(s): He; Jialong
• Application Number:
  US11/258,248
• Publication Number:
  US 20070094006A1
• Time in Patent Office: 1,590 Days
• Field of Search: 704/4, 704 7- 9, 704/10, 704/2, 707/4, 707/3
• US Class Current: 704/4
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation

Abstract

A system and method for cross-language knowledge searching. The system has a Semantic Analyzer, a natural language user request/document search pattern/semantic index Generator, a user request search pattern Translator and a Knowledge Base Searcher. The system also provides automatic semantic analysis and semantic indexing of natural language user requests/documents on knowledge recognition and cross-language relevant to user request knowledge extraction/searching. System functionality is ensured by Linguistic Knowledge Base as well as by a number of unique bilingual dictionaries of concepts/objects and actions.

Claims

1. A computer-based method for cross-language knowledge searching, the method implemented by at least one computer processor accessing at least one knowledge base comprising sources in a first language and sources in a second language and a bilingual dictionary stored in at least one storage device, the method comprising:

building the bilingual dictionary using parallel corpora, including;

for each sentence in a first source in the first language, generating a first source semantic index in the first language;

for each sentence in a second source in the second language, generating a second source semantic index in the second language, where the second source is a translation of the first source and each first source semantic index and corresponding second source semantic index form parallel semantic indexes having parallel eSAO component pairs; and

recognizing semantic components in an input expression in the first language;

generating a first semantic index in the first language from the semantic components, wherein the first semantic index includes first lexical units, at least one first lexical unit comprising a word with a part of speech (POS) tag;

translating the first semantic index into a second semantic index in the second language using a bilingual dictionary of actions and concepts, including translating the first lexical units into second lexical units in the second language, and translating a first word from the first semantic index into corresponding words in the second language and tagging each of the corresponding words with a POS tag of the first word; and

retrieving information relevant to the input expression from a knowledge base, which includes semantically indexed information in the second language, when the first and second semantic indexes match a subset of semantic indexes of the knowledge base associated with the information.

2. The method according to claim 1, wherein the method further comprises:

performing a search of the knowledge base of sources in the first language based on the first semantic index to access the information relevant to the input expression.

3. The method according to claim 2, wherein building the knowledge base comprises:

performing semantic analysis on a plurality of sources in the first and the second language; and

generating semantic indices for the plurality of sources based on the semantic analysis.

4. The method according to claim 1, wherein generating the first semantic index includes performing a semantic analysis identifying expanded subject-action-object (eSAO) units in the input expression.

5. The method according to claim 4, wherein generating the first semantic index comprises identifying one or more eSAO components in each identified eSAO unit and the first semantic index comprises the one or more eSAO components.

6. The method according to claim 5, wherein translating the first semantic index to the second semantic index comprises translating each eSAO component in the first semantic index from the first language into at least one corresponding eSAO component in the second language.

7. The method according to claim 6, wherein translating each eSAO component from the first language comprises accessing one or more bilingual dictionaries of actions and/or concepts, wherein the bilingual dictionaries match an eSAO component in the first language to one or more eSAO components in the second language.

8. The method according to claim 7, wherein the bilingual dictionaries comprise bilingual dictionaries of actions and bilingual dictionaries of concepts/objects.

9. The method according to claim 1, wherein the input expression comprises a user request.

10. The method according to claim 1, further comprising:

translating the first semantic index to a plurality of semantic indices for a plurality of languages different than the first language; and

performing a search of knowledge bases of sources in the plurality of languages based on the plurality of semantic indices to access information relevant to the expression.

11. The method according to claim 1, wherein retrieving information relevant to the input expression comprises providing one or more documents in the first and the second language relevant to the input expression or providing one or more output expressions relevant to the input expression.

12. The method of claim 1, further including:

extracting parallel actions and concepts from parallel semantic indexes and storing the parallel actions and concepts in the bilingual dictionary.

13. The method of claim 12, further comprising:

automatically editing the bilingual dictionary, including deleting repeating pairs of lexical units from the parallel eSAO component pairs; and

storing the edited bilingual dictionary in a linguistic knowledge base.

14. The method according to claim 12, wherein one corpus from one parallel corpora comprises sources comprising sentence-by-sentence translations in the first language of the sources of the other corpus from the same parallel corpora in the second language.

15. A computer-readable medium having computer-executable instructions for performing a method for cross-language knowledge searching when executed by at least one processor, the method comprising:

accessing at least one knowledge base comprising sources in a source language and sources in a target language and a bilingual dictionary stored in at least one storage device;

building the bilingual dictionary using parallel corpora, including;

for each sentence in a first source in the source language, generating a first source semantic index in the source language;

for each sentence in a second source in the target language, generating a second source semantic index in the target language, where the second source is a translation of the first source and each first source semantic index and corresponding second source semantic index form parallel semantic indexes having parallel eSAO component pairs;

recognizing semantic components in a user request received in a source language;

generating a first semantic index in the source language from the semantic components, the first semantic index including first lexical units, at least one lexical unit comprising a word with a part of speech (POS) tag;

translating the first semantic index into a second semantic index in a target language using a bilingual dictionary of actions and concepts, including translating the first lexical units into second lexical units in the target language, and translating a first word from the first semantic index into corresponding words in the target language and tagging each of the corresponding words with a POS tag of the first word; and

retrieving information relevant to the user request from the knowledge base that includes semantically indexed information in the target language, when the first and second semantic indexes match a subset of semantic indexes of the knowledge base associated with the information.

16. A computerized cross-language knowledge search system, comprising:

a bilingual dictionary builder that uses parallel corpora, comprising;

a first semantic analyzer configured to generate, for each sentence in a first document in a first language, a first document semantic index including eSAO components in the first language;

a second semantic analyzer configured to generate, for each sentence in a second document in a second language, a second document semantic index including eSAO components in the second language, where the second document is a translation of the first document and each first document semantic index and corresponding second document semantic index form parallel semantic indexes having parallel eSAO component pairs;

the first semantic analyzer also configured to recognize semantic components in a user request received in the first language,

a request pattern index generator configured to generate a first semantic index in the first language from the semantic components of the user request, the first semantic index including first lexical units, at least one lexical unit comprising a word with a part of speech (POS) tag;

a request pattern translator that accesses a bilingual dictionary of actions and concepts to translate the first semantic index into a second semantic index in the second language, including translating the first lexical units into second lexical units in the second language, and translating a first word from the first semantic index into corresponding words in the second language and tagging each of the corresponding words with a POS tag of the first word; and

a knowledge base searcher configured to retrieve information from a knowledge base that includes semantically indexed information in the second language, when the first and second semantic indexes match a subset of semantic indexes of the knowledge base associated with the information.

17. The system according to claim 16, wherein the knowledge base searcher is also configured to retrieve information from one or more knowledge bases of sources in the first language based on the first language expression semantic index.

18. The system according to claim 17, wherein the system further comprises:

one or more source semantic analyzers configured to identify semantic components in the sources in the first and in one or more other languages, wherein each source semantic analyzer corresponds to the first and to one of the one or more other languages; and

one or more source pattern index generators configured to the identified semantic components in the sources into source semantic indices in the first and in one or more of the other languages of the sources, wherein the one or more knowledge bases comprise the source semantic indices.

19. The system according to claim 18, wherein the one or more source semantic analyzers are coupled to a linguistic knowledge base.

20. The system according to claim 16, wherein the semantic components comprise expanded subject-action-object (eSAO) components.

21. The system according to claim 20, further comprising:

a linguistic knowledge base that comprises a plurality of eSAO component pairs, wherein one part of an eSAO component pair comprises one or more eSAO components in one language and the other part of the eSAO component pair comprises one or more eSAO components in another language.

22. The system according to claim 21, wherein the plurality of eSAO component pairs are organized into one or more bilingual dictionaries.

23. The system according to claim 22, wherein the one or more bilingual dictionaries comprise bilingual dictionaries of actions and bilingual dictionaries of concepts/objects.

24. The system according to claim 21, further comprising:

a document pattern translator that locates each eSAO component in the first language expression semantic index in one part of an eSAO component pair and produces the other language expression semantic index based on the other part of the eSAO component pair.

25. The system according to claim 16, wherein the input expression comprises a user request.

26. The system of claim 16, wherein the

dictionary builder is also configured to extract parallel actions and concepts from parallel semantic indexes and store the parallel actions and concepts in the bilingual dictionary.

27. The system of claim 26, further comprising:

a dictionary editor configured to edit the bilingual dictionary, including deleting repeating pairs of lexical units from the parallel eSAO component pairs, and store the edited bilingual dictionary in a linguistic knowledge base.

28. The system according to claim 26, wherein one corpus from the parallel corpora comprises sources comprising sentence-by-sentence translations in the first language of the sources of the other corpus from the same parallel corpora in the second language.