METHOD AND SYSTEM FOR SEMANTIC SEARCHING

• US 20140114649A1
• Filed: 12/27/2013
• Published: 04/24/2014
• Est. Priority Date: 10/10/2006
• Status: Active Grant

• Current Assignee: ABBYY Development LLC
• Original Assignee: ABBYY InfoPoisk LLC
• Inventors: Zuev, Konstantin, Danielyan, Tatiana, Rakhmatulina, Elmira
• Granted Patent: US 9,645,993 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 40/237 Lexical tools
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation

Abstract

A method and system for facilitating a semantic search based on one or more corpuses of natural language texts are provided. One or more corpuses of natural language texts are received including indexed linguistic parameters and semantic structures of lexical units. The linguistic parameters and semantic structures are generated during a preliminary syntactico-semantic analysis. Searching for text fragments satisfying a query in the one or more corpuses is performed. Relevance of the search results is estimated.

Claims

1. A computer-implemented method for facilitating a semantic search based on one or more corpuses of natural language texts, the method comprising:

receiving one or more corpuses of natural language texts including indexed linguistic parameters and semantic structures of lexical units, the linguistic parameters and semantic structures generated during a preliminary syntactico-semantic analysis;

searching for text fragments satisfying a query in the one or more corpuses;

estimating relevance of search results; and

ranking search results according to estimated relevance.

2. The method of claim 1, wherein the syntactico-semantic analysis of each corpus of natural language texts comprises for each sentence of the corpus:

generating all potentially possible relations between lexical units of a sentence;

building a graph of generalized constituents;

generating from the graph of generalized constituents at least one syntactic tree;

establishing non-tree links and building a syntactic structure based on the syntactic tree, and storing the syntactic structure in a memory; and

generating a semantic structure for each sentence of the corpora of natural language texts, and storing the semantic structure in a memory.

3. The method of claim 2, wherein the syntactico-semantic analysis further comprises performing a lexical selection for each lexical unit;

and selecting a syntactic structure of the sentence.

4. The method of claim 3, wherein the lexical selection for each lexical unit consists of selection a most probable lexical meaning of the lexical unit from a plurality of lexical meanings using priori ratings and integral calculated ratings.

5. The method of claim 3, wherein the lexical choice for each lexical unit in a sentence comprises assigning to the lexical unit a set of possible lexical meanings, with each of lexical meaning having a corresponding probability of occurrence in the sentence.

6. The method of claim 1, wherein the syntactico-semantic analysis comprises identifying and retrieving values of morphological, lexical, syntactical and semantic parameters of the texts of the corpus.

7. The method of claim 1, wherein the syntactico-semantic analysis further comprises assigning notions of an ontology to the units of the corpus.

8. The method of claim 1, wherein the syntactico-semantic analysis further comprises restoring referential links between sentences.

9. The method of claim 1, wherein indexing comprises creating at least one index of lexical meanings;

and storing the at least one index into a memory.

10. The method of claim 1, wherein indexing comprises creating at least one index for a linguistic parameter value;

and storing the at least one index into a memory.

11. The method of claim 1, wherein indexing comprises creating at least one index of syntactic structures;

and storing the at least one index into a memory.

12. The method of claim 1, wherein indexing comprises creating at least one index of semantic structures;

and storing the at least one index into a memory.

13. The method of claim 1, wherein at least one unit of the query is a lexical meaning.

14. The method of claim 13, wherein the query is a combination of lexical meaning with using at least one Boolean operator from a set comprising AND, OR and NOT.

15. The method of claim 13, wherein the lexical meaning of a unit of the query is selected by a user by selection of a semantic class from a list of semantic classes.

16. The method of claim 13, wherein the lexical meaning of a unit of the query is selected by a user by selection of an interpretation from a list of interpretations of the lexical unit.

17. The method of claim 13, wherein the lexical meaning of a unit of the query is determined using account probability estimates.

18. The method of claim 13, wherein the lexical meaning of a unit of the query is determined using pragmatic and context analysis.

19. The method of claim 13, wherein the lexical meaning of a unit of the query is determined automatically based on syntactico-semantic analyzing of the query comprising at least syntactic analysis of the query, determining lexical meanings of elements of the query and recognizing a semantic structure of the query.

20. The method of claim 1, wherein an element of the query is a syntactic structure.

21. The method of claim 1, wherein an element of the query is a semantic structure.

22. The method of claim 1, wherein the query is a sentence in a natural language.

23. The method of claim 22, wherein the query is an interrogative sentence in a natural language.

24. The method of claim 22, wherein the query is in a form of a sentence in a natural language having a lacuna.

25. The method of claim 1, further comprising assigning a notion of ontology to at least one unit of the query.

26. The method of claim 25, wherein assigning the notion of the ontology to at least one unit of the query is performed automatically.

27. The method of claim 1, wherein searching is performed using restored referential links between sentences.

28. The method of claim 1, wherein searching further comprises searching lexical meanings similar to the searched lexical meaning.

29. The method of claim 1, wherein searching comprises finding syntactic structures that are identical and similar to syntactic structures of the query.

30. The method of claim 1, wherein searching comprises finding semantic structures that are identical and similar to semantic structures of the query.

31. The method of claim 1, wherein estimating relevance of each search result is performed using a measure of relevance based on proximity of a found lexical meaning to a corresponding item in the query.

32. The method of claim 1, wherein relevance estimating each search result is performed using a probability estimate of a searched lexical meaning in a text fragment determined during lexical selection during the process of preliminary analysis of the text corpus.

33. The method of claim 1, wherein the search results include information about lexical meanings of the elements of the query in found fragments.

34. A system for facilitating a semantic search based on one or more corpuses of natural language texts, the system comprising:

one or more data processors; and

one or more storage devices storing instructions that, when executed by the one or more data processors, cause the one or more data processors to perform operations comprising;

receiving one or more corpuses of natural language texts including indexed linguistic parameters and semantic structures of lexical units, the linguistic parameters and semantic structures generated during a preliminary syntactico-semantic analysis;

searching for text fragments satisfying a query in the one or more corpuses;

estimating relevance of search results; and

ranking search results according to estimated relevance.

35. The system of claim 34, wherein the syntactico-semantic analysis of each corpus of natural language texts comprises for each sentence of the corpus:

generating all potentially possible relations between lexical units of a sentence;

building a graph of generalized constituents;

generating from the graph of generalized constituents at least one syntactic tree;

establishing non-tree links and building a syntactic structure based on the syntactic tree, and storing the syntactic structure in a memory; and

generating a semantic structure for each sentence of the corpora of natural language texts, and storing the semantic structure in a memory.

36. The system of claim 35, wherein the syntactico-semantic analysis further comprises performing a lexical selection for each lexical unit;

and selecting a syntactic structure of the sentence.

37. A computer-readable storage medium having machine instructions stored therein, the instructions being executable by a processor to cause the processor to perform operations comprising:

receiving one or more corpuses of natural language texts including indexed linguistic parameters and semantic structures of lexical units, the linguistic parameters and semantic structures generated during a preliminary syntactico-semantic analysis;

searching for text fragments satisfying a query in the one or more corpuses;

estimating relevance of search results; and

ranking search results according to estimated relevance.