Method and system for semantic searching

• US 9,645,993 B2
• Filed: 12/27/2013
• Issued: 05/09/2017
• Est. Priority Date: 10/10/2006
• Status: Active Grant

• Current Assignee: ABBYY Development LLC
• Original Assignee: ABBYY InfoPoisk LLC
• Inventors: Zuev, Konstantin, Danielyan, Tatiana, Rakhmatulina, Elmira
• Primary Examiner(s): ROBERTS, SHAUN A
• Application Number:
  US14/142,701
• Publication Number:
  US 20140114649A1
• Time in Patent Office: 1,229 Days
• Field of Search: 704 9
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 40/237 Lexical tools
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation

Abstract

A method and system for facilitating a semantic search based on one or more corpuses of natural language texts are provided. One or more corpuses of natural language texts are received including indexed linguistic parameters and semantic structures of lexical units. The linguistic parameters and semantic structures are generated during a preliminary syntactico-semantic analysis. Searching for text fragments satisfying a query in the one or more corpuses is performed. Relevance of the search results is estimated.

Claims

1. A computer-implemented method for performing a semantic search in a natural language text, the method comprising:

performing, by a processor, a syntactico-semantic analysis of a natural language text to produce a plurality of semantic structures representing a plurality of sentences of the natural language text, wherein the syntactico-semantic analysis of a sentence of the natural language text comprises;

generating a plurality of relationships between lexical units of the sentence,

producing a graph of generalized constituents of the sentence,

generating a plurality of syntactic trees based on the graph of generalized constituents,

selecting an optimal syntactic tree in view of determined non-tree links,

producing a semantic structure based on the selected syntactic tree, and

storing the semantic structure in a memory;

producing, using the plurality of semantic structures, an index indicative of positions, within the natural language text, of certain values of at least one of morphological parameters, syntactic parameters, lexical parameters, or semantic parameters, and wherein the index further comprises weight values associated with the parameter values;

searching, using the index, the natural language text for text fragments satisfying a search query;

estimating relevance of search results based on weight values specified by the index; and

ranking search results according to estimated relevance.

2. The method of claim 1, wherein the syntactico-semantic analysis further comprises performing a lexical selection for each lexical unit.

3. The method of claim 2, wherein performing the lexical selection for each lexical unit further comprises selecting a most probable lexical meaning of the lexical unit from a plurality of lexical meanings using a priori ratings and integral calculated ratings.

4. The method of claim 2, wherein performing the lexical selection for each lexical unit further comprises assigning to the lexical unit a set of possible lexical meanings, each of lexical meaning having a corresponding probability of occurrence in the sentence.

5. The method of claim 1, wherein performing the syntactico-semantic analysis further comprises identifying values of at least one of:

morphological, lexical, syntactical or semantic parameters of the natural language text.

6. The method of claim 1, wherein performing the syntactico-semantic analysis further comprises associating a concept of an ontology with at least part of the sentence.

7. The method of claim 1, wherein performing the syntactico-semantic analysis further comprises restoring referential links between sentences.

8. The method of claim 1, wherein at least one lexical unit of the query comprises a lexical meaning.

9. The method of claim 8, wherein the query query comprises a combination of lexical meanings using at least one Boolean operator.

10. The method of claim 8, wherein the lexical meaning represents a user selection of a semantic class from a list of semantic classes.

11. The method of claim 8, wherein the lexical meaning represents a user selection of an interpretation from a list of interpretations of the lexical unit.

12. The method of claim 8, wherein the lexical meaning is determined using probability estimates.

13. The method of claim 8, wherein the lexical meaning is determined using pragmatic and context analysis.

14. The method of claim 8, wherein the lexical meaning is determined based on syntactico-semantic analysis of the query.

15. The method of claim 1, wherein an element of the query comprises a syntactic structure.

16. The method of claim 1, wherein an element of the query comprises a semantic structure.

17. The method of claim 1, wherein the query comprises a sentence in a natural language.

18. The method of claim 1, wherein the query comprises an interrogative sentence in a natural language.

19. The method of claim 1, wherein the query comprises a sentence in a natural language having a lacuna.

20. The method of claim 1, further comprising assigning a concept of ontology to at least one unit of the query.

21. The method of claim 1, wherein searching is performed using restored referential links between sentences of the natural language text.

22. The method of claim 1, wherein searching further comprises identifying, in the natural language text, lexical meanings similar to a lexical meaning referenced by the query.

23. The method of claim 1, wherein searching comprises identifying, in the natural language text, syntactic structures that are similar to syntactic structures of the query.

24. The method of claim 1, wherein searching comprises identifying, in the natural language text, semantic structures that are similar to semantic structures of the query.

25. The method of claim 1, wherein estimating relevance of a search result is performed using a measure of relevance based on proximity of an identified lexical meaning to a corresponding item in the query.

26. The method of claim 1, wherein estimating relevance of a search result is performed using a probability estimate of an identified lexical meaning in a fragment of the natural language text.

27. The method of claim 1, wherein the search results include information about lexical meanings of the elements of the query in identified fragments of the natural language text.

28. A system for performing a semantic search in a natural language text, the system comprising one or more processors coupled to a memory, the processors configured to:

perform a syntactico-semantic analysis of a natural language text to produce a plurality of semantic structures representing a plurality of sentences of the natural language text, wherein the syntactico-semantic analysis of a sentence of the natural language text comprises;

generating a plurality of relationships between lexical units of the sentence,

producing a graph of generalized constituents of the sentence,

generating a plurality of syntactic trees based on the graph of generalized constituents,

selecting an optimal syntactic tree in view of determined non-tree links,

producing a semantic structure based on the selected syntactic tree, and

storing the semantic structure in the memory;

produce, using the plurality of semantic structures, an index indicative of positions, within the natural language text, of certain values of at least one of morphological parameters, syntactic parameters, lexical parameters, or semantic parameters, and wherein the index further comprises weight values associated with the parameter values;

search, using the index, the natural language text for text fragments satisfying a search query;

estimate relevance of search results based on weight values specified by the index; and

rank search results according to estimated relevance.

29. The system of claim 28, wherein the syntactico-semantic analysis further comprises performing a lexical selection for each lexical unit.

30. A non-transitory computer-readable storage medium having machine instructions stored therein, that when executed by a processor, cause the processor to:

performing a syntactico-semantic analysis of a natural language text to produce a plurality of semantic structures representing a plurality of sentences of the natural language text, wherein the syntactico-semantic analysis of a sentence of the natural language text comprises;

generating a plurality of relationships between lexical units of the sentence,

producing a graph of generalized constituents of the sentence,

generating a plurality of syntactic trees based on the graph of generalized constituents,

selecting an optimal syntactic tree in view of determined non-tree links,

producing a semantic structure based on the selected syntactic tree, and

storing the semantic structure in the memory;

producing, using the plurality of semantic structures, an index indicative of positions, within the natural language text, of certain values of at least one of morphological parameters, syntactic parameters, lexical parameters, or semantic parameters, and wherein the index further comprises weight values associated with the parameter values;

searching, using the index, the natural language text for text fragments satisfying a search query;

estimating relevance of search results based on weight values specified by the index; and

ranking search results according to estimated relevance.