Method and system for semantic searching

• US 9,098,489 B2
• Filed: 06/30/2011
• Issued: 08/04/2015
• Est. Priority Date: 10/10/2006
• Status: Active Grant

• Current Assignee: ABBYY Development LLC
• Original Assignee: ABBYY InfoPoisk LLC
• Inventors: Zuev, Konstantin, Rylov, Alexander
• Primary Examiner(s): ROBERTS, SHAUN A
• Application Number:
  US13/173,369
• Publication Number:
  US 20110257963A1
• Time in Patent Office: 1,496 Days
• Field of Search: 704 8- 9, 707/705, 707/708, 707/711, 707/713, 707/736, 707/760
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

A method comprising a preliminary automated analysis of at least one corpus of natural language text is disclosed. For each sentence of a corpus, the method includes performing a syntactic analysis using linguistic descriptions to generate at least one syntactic structure for the sentence, building a semantic structure for the sentence, associating each generated syntactic and semantic structure with the sentence, and saving each structure. For each corpus text that was preliminary analyzed, performing an indexing operation to index lexical meanings and values of linguistic parameters of each syntactic structure and each semantic structure associated with sentences in the corpus text. A semantic search includes at least one automatic preliminary analyzed corpus of sentences comprising searched values of linguistic, syntactic and semantic parameters. Due to a deep semantic analysis of a corpus, the search may be executed in various languages, in resources of various languages, and in the text of corpora of various languages regardless of the language of the query.

Claims

1. A computer-implemented method for facilitating a semantic search, the method comprising:

identifying a corpora of natural language texts including a plurality of sentences;

performing a syntactic-semantic analysis on each sentence of the plurality of sentences using a linguistic description associated with a language of the sentence, wherein the syntactic-semantic analysis comprises;

generating a graph of generalized constituents for each sentence of the plurality of sentences; and

generating one or more syntactic trees based on the graphs of generalized constituents to represent the corresponding sentences;

generating at least one syntactic structure for each sentence of the plurality of sentences by selecting a best syntactic tree from the generated one or more syntactic trees to represent the at least one syntactic structure of the sentence;

generating a semantic structure for each sentence of the corpora of natural language texts, based on the generated at least one syntactic structure of the sentence, wherein the semantic structure is language independent and wherein the semantic structure comprises semantic classes, semantemes, deep slots, and non-tree links;

associating the generated syntactic structures and the generated semantic structures with the respective sentences;

creating syntactic index for each meaning of at least one linguistic parameter of each of the generated syntactic structures;

creating a semantic index for each meaning of at least one parameter of the semantic structures;

receiving a search query comprising semantic language-independent terms;

searching the semantic index based on the semantic language-independent terms and the language-independent semantic structures; and

receiving semantic search results from the semantic index, wherein the search results from the corpora of natural language texts includes sentences in different languages.

2. The computer-implemented method of claim 1, wherein the performing the syntactic-semantic analysis includes resolving an ambiguity of lexical unit meanings within at least one of the sentences of the corpora of natural language texts.

3. The computer-implemented method of claim 1, wherein each sentence includes one or more language-specific properties, and wherein generating the language independent semantic structure comprises encoding each language-specific property with a language-independent parameter.

4. The computer-implemented method of claim 3, wherein the method further comprises indexing at least one meaning of each language-independent parameter.

5. The computer-implemented method of claim 1, wherein the search is performed in a corpora of natural language texts of different languages.

6. The computer-implemented method of claim 1, wherein the semantic search results are received in a source language.

7. The computer-implemented method of claim 1, wherein the method further comprises indexing combinations of two or more meanings of syntactic parameters.

8. The computer-implemented method of claim 1, wherein the method further comprises indexing combinations of two or more meanings of semantic parameters.

9. The computer-implemented method of claim 1, wherein the method further comprises:

performing a search query and sending a search result based at least in part upon the syntactic index.

10. The computer-implemented method of claim 1, wherein the method further comprises:

performing a search query and sending a search result based at least in part upon the semantic index.

11. The computer-implemented method of claim 1, where the linguistic description includes at least a semantic description, a syntactic description, a lexical description, and a morphological description.

12. The computer-implemented method of claim 11, wherein the method further comprises generating an overall index from:

at least one value of a semantic parameter associated with the semantic description,

at least one value of the syntactic parameter associated with the syntactic description,

at least one value of a lexical parameter associated with the lexical description, and

at least one value of a morphological parameter associated with the morphological description.

13. The computer-implemented method of claim 12, wherein the syntactic parameter may include a syntactic relation.

14. The computer-implemented method of claim 12, wherein the semantic parameter may include a semantic relation.

15. A computer-implemented method for providing a result of a search, the method comprising:

for each sentence of a corpus of texts, the corpus of texts including a plurality of sentences, generating at least one syntactic structure for each sentence of the plurality of sentences, using at least one linguistic description associated with a source natural language by performing a syntactic-semantic analysis of the sentences comprising;

generating graph of generalized constituents for each sentence of the plurality of sentences;

generating one or more syntactic trees based on the graphs of generalized constituents to represent the corresponding sentences; and

selecting a best syntactic tree from the generated one or more syntactic trees to represent the at least one syntactic structure;

building a language-independent semantic structure for each said sentence based on the at least one syntactic structure of the sentence, wherein the semantic structure comprises semantic classes, semantemes, deep slots, and non-tree links;

associating each generated syntactic structure and each language independent semantic structure with a respective sentence;

indexing at least one meaning of linguistic parameters associated with each sentence;

indexing at least one lexical meaning associated with each lexical unit of each sentence;

indexing at least one value associated with linguistic parameters related to a syntactic structure of each sentence;

indexing at least one value associated with semantic parameters related to the language-independent semantic structure of each sentence; and

receiving a search query comprising semantic language-independent terms;

searching the index of at least one value associated with semantic parameters based on the semantic language-independent terms and the language-independent semantic structures; and

receiving semantic search results from the index of at least one value associated with semantic parameters, wherein the search results include sentences from the corpus of text in different languages.

16. The computer-implemented method of claim 15, wherein the searching includes resolving an ambiguity of lexical unit meanings within at least one of the sentences of the corpus of text.

17. The computer-implemented method of claim 15, wherein the language-independent semantic structure for each said sentence includes a stylistic characteristic, a pragmatic characteristic or a communicative characteristic.

18. The computer-implemented method of claim 15, wherein the performing the search includes translating a query term into a semantic language-independent query term.

19. A system for facilitating a semantic search, the system comprising:

a first processor and a computer readable memory;

a corpus of natural language texts including a plurality of sentences;

an analyzer in the computer readable memory configured to;

perform a syntactic-semantic analysis on each sentence of the plurality of sentences using a linguistic description associated with a language of the sentence wherein the syntactic-semantic analysis comprises;

generating a graph of generalized constituents for each sentence of the plurality of sentences; and

generating one or more syntactic trees based on the graphs of generalized constituents to represent the corresponding sentences;

generating at least one syntactic structure for each sentence of the plurality of sentences by selecting a best syntactic tree from the generated one or more syntactic trees to represent the at least one syntactic structure of the sentence;

generating a semantic structure for each sentence of the corpus of natural language texts, based on the generated at least one syntactic structure of the sentence, wherein the semantic structure is language independent and wherein the semantic structure comprises semantic classes, semantemes, deep slots, and non-tree links; and

associate the generated syntactic structures and the generated semantic structures with the respective sentences;

an index generation component configured to;

create a syntactic index for each meaning of at least one linguistic parameter of the generated syntactic structures; and

create a semantic index for each meaning of at least one parameter of the language independent semantic structures;

and

the first processor configured to;

receive a search query comprising semantic language-independent terms;

search the semantic index of at least one value associated with semantic parameters based on the semantic language-independent terms and the language-independent semantic structures; and

receive semantic search results from the semantic index of at least one value associated with semantic parameters, wherein the search results include sentences from the corpus of text in different languages.

20. The system of claim 19, wherein to perform the syntactic-semantic analysis the analyzer is configured to resolve an ambiguity of lexical unit meanings within at least one of the sentences of the corpus of natural language texts.

21. The system of claim 20, wherein each sentence includes a language-specific property, wherein the generating the language independent semantic structure includes encoding the language-specific property with a language-independent parameter.

22. The system of claim 21, wherein the index generation component is further configured to index at least one meaning of each language-independent parameter.

23. The system of claim 19, wherein the search may be performed in a corpus of natural language texts of different languages.

24. The system of claim 19, wherein the semantic search result is sent in a source language.

25. The system of claim 19, wherein the index generation component is further configured to index a meaning of each sentence.

26. The system of claim 19, wherein the index generation component is further configured to index combinations of two or more meanings of syntactic parameters.

27. The system of claim 19, wherein the index generation component is further configured to index combinations of two or more meanings of semantic parameters.

28. The system of claim 19, wherein the first processor is further configured to:

after each generated syntactic structure and the language independent semantic structure for each sentence of the corpus of natural language text is stored, perform a search query and send a search result based at least in part upon the syntactic index.

29. The system of claim 19, where each linguistic description includes at least a semantic description, a syntactic description, a lexical description, and a morphological description.

30. The system of claim 29, wherein the index generation component is further configured to generate an overall index from:

at least one value of a semantic parameter associated with the semantic description,

at least one value of the syntactic parameter associated with the syntactic description,

at least one value of a lexical parameter associated with the lexical description, and

at least one value of a morphological parameter associated with the morphological description.

31. The system of claim 30, wherein the syntactic parameter may include a syntactic relation.

32. The system of claim 30, wherein the semantic parameter may include a semantic relation.

33. One or more non-transitory computer-readable media having computer-executable instructions, which when executed perform steps, comprising:

instructions for identifying a corpus of natural language texts including a plurality of sentences;

instructions for performing a syntactic-semantic analysis on each sentence of the plurality of sentences using a linguistic description associated with a language of the sentence, wherein the syntactic-semantic analysis comprises;

generating graph of generalized constituents for each sentence of the plurality of sentences; and

generating one or more syntactic trees based on the graphs of generalized constituents to represent the corresponding sentences;

generating at least one syntactic structure for each sentence of the plurality of sentences by selecting a best syntactic tree from the generated one or more syntactic trees to represent the at least one syntactic structure of the sentence;

generating a semantic structure for each sentence of the corpus of natural language texts, wherein the semantic structure is language independent and wherein the semantic structure comprises semantic classes, semantemes, deep slots, and non-tree links;

instructions for associating the language independent semantic structure with a respective sentence;

instructions for creating a syntactic index for each meaning of at least one linguistic parameter of the generated syntactic structures;

instructions for creating a semantic index for each meaning of at least one parameter of the language independent semantic structures;

instructions for receiving a search query comprising semantic language-independent terms;

instructions for searching the semantic index of at least one value associated with semantic parameters based on the semantic language-independent terms and the language-independent semantic structures; and

instructions for receiving semantic search results from the semantic index of at least one value associated with semantic parameters, wherein the search results include sentences from the corpus of text in different languages.

34. The one or more non-transitory computer-readable media of claim 33, wherein the instructions for performing the syntactic-semantic analysis include instructions for resolving an ambiguity of lexical unit meanings within at least one of the sentences of the corpus of natural language texts.

35. The one or more non-transitory computer-readable media of claim 33, wherein each sentence includes a language-specific property, wherein generating the language independent semantic structure includes encoding the language-specific property with a language-independent parameter.

36. The one or more non-transitory computer-readable media of claim 33, wherein the computer-executable instructions further comprise instructions for indexing at least one meaning of each language-independent parameter.

37. The one or more non-transitory computer-readable media of claim 33, wherein the search may be performed in a corpus of natural language texts of different languages.

38. The one or more non-transitory computer-readable media of claim 33, wherein the semantic search result is sent in a source language.

39. The one or more non-transitory computer-readable media of claim 38, wherein the instructions further comprise indexing a meaning of each sentence.

40. The one or more non-transitory computer-readable media of claim 33, wherein the instructions further comprise indexing combinations of two or more meanings of syntactic parameters.

41. The one or more non-transitory computer-readable media of claim 33, wherein the instructions further comprise indexing combinations of two or more meanings of semantic parameters.

42. The one or more non-transitory computer-readable media of claim 33, wherein the instructions further comprise:

performing a search query and sending a search result based at least in part upon the syntactic index.

43. The one or more non-transitory computer-readable media of claim 33, where each linguistic description includes at least a semantic description, a syntactic description, a lexical description, and a morphological description.

44. The one or more non-transitory computer-readable media of claim 43, wherein the instructions further comprise instructions for generating an overall index from:

at least one value of a semantic parameter associated with the semantic description,

at least one value of the syntactic parameter associated with the syntactic description,

at least one value of a lexical parameter associated with the lexical description, and

at least one value of a morphological parameter associated with the morphological description.

45. The one or more non-transitory computer-readable media of claim 44, wherein the syntactic parameter may include a syntactic relation.

46. The one or more non-transitory computer-readable media of claim 44, wherein the semantic parameter may include a semantic relation.