METHOD AND SYSTEM FOR SEMANTIC SEARCHING

• US 20110257963A1
• Filed: 06/30/2011
• Published: 10/20/2011
• Est. Priority Date: 10/10/2006
• Status: Active Grant

• Current Assignee: ABBYY Development LLC
• Original Assignee: ABBYY InfoPoisk LLC
• Inventors: Rylov, Alexander, Zuev, Konstantin
• Granted Patent: US 9,098,489 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

A method comprising a preliminary automated analysis of at least one corpus of natural language text is disclosed. For each sentence of a corpus, the method includes performing a syntactic analysis using linguistic descriptions to generate at least one syntactic structure for the sentence, building a semantic structure for the sentence, associating each generated syntactic and semantic structure with the sentence, and saving each structure. For each corpus text that was preliminary analyzed, performing an indexing operation to index lexical meanings and values of linguistic parameters of each syntactic structure and each semantic structure associated with sentences in the corpus text. A semantic search includes at least one automatic preliminary analyzed corpus of sentences comprising searched values of linguistic, syntactic and semantic parameters. Due to a deep semantic analysis of a corpus, the search may be executed in various languages, in resources of various languages, and in the text of corpora of various languages regardless of the language of the query.

Claims

1. A computer-implemented method for facilitating a semantic search, the method comprising:

identifying a corpora of natural language texts including a plurality of sentences;

performing a syntactic analysis on each sentence using a linguistic description associated with a language of the sentence;

generating, based at least in part on the syntactic analysis, at least one syntactic structure for each sentence;

generating a semantic structure for each sentence of the corpora of natural language texts;

associating each generated syntactic structure and the semantic structure with a respective sentence;

creating an index for each meaning of a linguistic parameter of each of the generated syntactic structures;

creating an index for each meaning of a parameter of each of the semantic structures; and

storing each generated syntactic structure and the semantic structure for each sentence of the corpora of natural language text.

2. The computer-implemented method of claim 1 wherein the performing the syntactic analysis includes resolving an ambiguity of lexical unit meanings within at least one of the sentences of the corpora of natural language texts.

3. The computer-implemented method of claim 1 wherein each sentence includes one or more language-specific properties, wherein the building the semantic structure includes encoding each language-specific property with a language-independent parameter.

4. The computer-implemented method of claim 3 wherein the method further comprises:

indexing at least one meaning of each language-independent parameter.

5. The computer-implemented method of claim 4 wherein the method further comprises:

after storing each generated syntactic structure and the semantic structure for each sentence of the corpora of natural language texts, performing a search and sending a semantic search result based at least in part upon the indexing the at least one meaning of each language-independent parameter or upon a language-specific property.

6. The computer-implemented method of claim 5 wherein the search may be performed in a corpora of natural language texts of different languages.

7. The computer-implemented method of claim 6 wherein the semantic search result includes sentences in different languages.

8. The computer-implemented method of claim 5 wherein the semantic search result is sent in a source language.

9. The computer-implemented method of claim 1 wherein the method further comprises indexing combinations of two or more meanings of syntactic parameters.

10. The computer-implemented method of claim 1 wherein the method further comprises indexing combinations of two or more meanings of semantic parameters.

11. The computer-implemented method of claim 1 wherein the method further comprises:

after storing each generated syntactic structure and the semantic structure for each sentence of the corpora of natural language text, performing a search query and sending a syntactic search result based at least in part upon the indexes of meanings of linguistic parameters and the indexes of meanings of the parameters associated with the syntactic structures.

12. The computer-implemented method of claim 1 wherein the method further comprises:

after storing each generated syntactic structure and the semantic structure for each sentence of the corpora of natural language text, performing a search query and sending a semantic search result based at least in part upon the indexes of meanings of the parameters associated with the semantic structures.

13. The computer-implemented method of claim 1, where each linguistic description includes at least a semantic description, a syntactic description, a lexical description, and a morphological description.

14. The computer-implemented method of claim 13 wherein the method of further comprises generating an overall index from:

a value of a semantic parameter associated with the semantic description,

a value of the syntactic parameter associated with the syntactic description,

a value of a lexical parameter associated with the lexical description, and

a value of a morphological parameter associated with the morphological description.

15. The computer-implemented method of claim 14, wherein the syntactic parameter may include a syntactic relation.

16. The computer-implemented method of claim 14, wherein the semantic parameter may include a semantic relation.

17. A method for providing a result of a search, the method comprising:

for each sentence of a corpus of texts, the corpus of texts including a plurality of sentences, generating at least one syntactic structure for each sentence using at least one linguistic description associated with a source natural language;

building a language-independent semantic structure for each said sentence;

associating each generated syntactic structure and each semantic structure with a respective sentence;

indexing at least one meaning of linguistic parameters associated with each sentence;

indexing at least one lexical meaning associated with each lexical unit of each sentence;

indexing at least one value associated with linguistic parameters related to a syntactic structure of each sentence;

indexing at least one value associated with semantic parameters related to the language-independent semantic structure of each sentence; and

performing a syntactic search using the indexing of the linguistic parameters, the indexing of the lexical meaning, the indexing of the value associated with the linguistic parameter related to each syntactic structure, and the indexing of the value associated with a semantic parameter related to each semantic structure, wherein the performing the syntactic search includes storing at least one of the indexing of meaning of linguistic parameters, the indexing of the at least one lexical meaning, the indexing of the at least one value associated with linguistic parameters related to a syntactic structure of each sentence, and the indexing of the at least one value associated with semantic parameters related to the language-independent semantic structure of each sentence.

18. The method of claim 17, wherein the performing the search includes resolving an ambiguity of lexical unit meanings within at least one of the sentences of the corpus of text.

19. The method of claim 17, wherein the language-independent semantic structure for each said sentence includes a stylistic characteristic, a pragmatic characteristic or a communicative characteristic.

20. The method of claim 17, wherein the language-independent semantic structure for each said sentence may include information about non-tree links (relations) in the sentence.

21. The method of claim 17, wherein the performing the search includes translating a query term into a semantic language-independent query term.

22. A system for facilitating a semantic search, the system comprising:

a first processor and a computer readable memory;

a corpus of natural language texts including a plurality of sentences;

a syntactic analyzer in the computer readable memory capable of;

performing a syntactic analysis on each sentence using a linguistic description associated with a language of the sentence;

generating, based at least in part on the syntactic analysis, at least one syntactic structure for each sentence;

generating a semantic structure for each sentence of the corpus of natural language texts;

associating each generated syntactic structure and the semantic structure with a respective sentence; and

storing each generated syntactic structure and the semantic structure for each sentence of the corpus of natural language text; and

an index generation component capable of;

creating an index for each meaning of a linguistic parameter of each of the generated syntactic structures;

creating an index for each meaning of a parameter of each of the semantic structures; and

storing the indexes.

23. The system of claim 22 wherein the performing the syntactic analysis includes resolving an ambiguity of lexical unit meanings within at least one of the sentences of the corpus of natural language texts.

24. The system of claim 23 wherein each sentence includes a language-specific property, wherein the generating the semantic structure includes encoding the language-specific property with a language-independent parameter.

25. The system of claim 24 wherein the index generation component is capable of:

indexing at least one meaning of each language-independent parameter.

26. The system of claim 25 wherein the system is capable of:

after storing each generated syntactic structure and the semantic structure for each sentence of the corpus of natural language texts, performing a search and sending a semantic search result based at least in part upon the indexing the at least one meaning of each language-independent parameter or upon a language-specific property.

27. The system of claim 26 wherein the search may be performed in a corpus of natural language texts of different languages.

28. The system of claim 27 wherein the semantic search result includes sentences in different languages.

29. The system of claim 27 wherein the semantic search result is sent in a source language.

30. The system of claim 22 wherein the system is further capable of indexing a meaning of each sentence.

31. The system of claim 22 wherein the system is further capable of indexing combinations of two or more meanings of syntactic parameters.

32. The system of claim 22 wherein the system is further capable of indexing combinations of two or more meanings of semantic parameters.

33. The system of claim 22 wherein the system is further capable of:

after storing each generated syntactic structure and the semantic structure for each sentence of the corpus of natural language text, performing a search query and sending a syntactic search result based at least in part upon the indexes of meanings of linguistic parameters and the indexes of meanings of the parameters associated with the semantic structures.

34. The system of claim 22 wherein the system further comprises:

after storing each generated syntactic structure and the semantic structure for each sentence of the corpus of natural language text, performing a search query and sending a semantic search result based at least in part upon the indexes of meanings of the parameters associated with the semantic structures.

35. The system of claim 22, where each linguistic description includes at least a semantic description, a syntactic description, a lexical description, and a morphological description.

36. The system of claim 35 wherein the system is further capable of generating an overall index from:

a value of a semantic parameter associated with the semantic description,

a value of the syntactic parameter associated with the syntactic description,

a value of a lexical parameter associated with the lexical description, and

a value of a morphological parameter associated with the morphological description.

37. The system of claim 36, wherein the syntactic parameter may include a syntactic relation.

38. The system of claim 36, wherein the semantic parameter may include a semantic relation.

39. One or more computer-readable media having computer-executable instructions, which when executed perform steps, comprising:

identifying a corpus of natural language texts including a plurality of sentences;

performing a syntactic analysis on each sentence using a linguistic description associated with a language of the sentence;

generating, based at least in part on the syntactic analysis, at least one syntactic structure for each sentence;

generating a semantic structure for each sentence of the corpus of natural language texts;

associating each generated syntactic structure and the semantic structure with a respective sentence;

creating an index for each meaning of a linguistic parameter of each of the generated syntactic structures;

creating an index for each meaning of a parameter of each of the semantic structures; and

storing each generated syntactic structure and the semantic structure for each sentence of the corpus of natural language text.

40. The one or more computer-readable media of claim 39 wherein the performing the syntactic analysis includes resolving an ambiguity of lexical unit meanings within at least one of the sentences of the corpus of natural language texts.

41. The one or more computer-readable media of claim 39 wherein each sentence includes a language-specific property, wherein the building the semantic structure includes encoding the language-specific property with a language-independent parameter.

42. The one or more computer-readable media of claim 39 wherein the computer-executable instructions further comprise:

indexing at least one meaning of each language-independent parameter.

43. The one or more computer-readable media of claim 42 wherein the computer-executable instructions further comprise:

after storing each generated syntactic structure and the semantic structure for each sentence of the corpus of natural language texts, performing a search and sending a semantic search result based at least in part upon the indexing the at least one meaning of each language-independent parameter or upon a language-specific property.

44. The one or more computer-readable media of claim 43 wherein the search may be performed in a corpus of natural language texts of different languages.

45. The one or more computer-readable media of claim 44 wherein the semantic search result includes sentences in different languages.

46. The one or more computer-readable media of claim 45 wherein the semantic search result is sent in a source language.

47. The one or more computer-readable media of claim 46 wherein the instructions further comprise indexing a meaning of each sentence.

48. The one or more computer-readable media of claim 39 wherein the instructions further comprise indexing combinations of two or more meanings of syntactic parameters.

49. The one or more computer-readable media of claim 39 wherein the instructions further comprise indexing combinations of two or more meanings of semantic parameters.

50. The computer-implemented method of claim 39 wherein the instructions further comprise:

after storing each generated syntactic structure and the semantic structure for each sentence of the corpus of natural language text, performing a search query and sending a syntactic search result based at least in part upon the indexes of meanings of linguistic parameters and the indexes of meanings of the parameters associated with the semantic structures.

51. The one or more computer-readable media of claim 39 wherein the instructions further comprise:

after storing each generated syntactic structure and the semantic structure for each sentence of the corpus of natural language text, performing a search query and sending a semantic search result based at least in part upon the indexes of meanings of the parameters associated with the semantic structures.

52. The one or more computer-readable media of claim 39, where each linguistic description includes at least a semantic description, a syntactic description, a lexical description, and a morphological description.

53. The one or more computer-readable media of claim 52 wherein the instructions of further comprise generating an overall index from:

a value of a semantic parameter associated with the semantic description,

a value of the syntactic parameter associated with the syntactic description,

a value of a lexical parameter associated with the lexical description, and

a value of a morphological parameter associated with the morphological description.

54. The one or more computer-readable media of claim 53, wherein the syntactic parameter may include a syntactic relation.

55. The one or more computer-readable media of claim 53, wherein the semantic parameter may include a semantic relation.