Method and system for semantic searching of natural language texts

• US 9,069,750 B2
• Filed: 06/30/2011
• Issued: 06/30/2015
• Est. Priority Date: 10/10/2006
• Status: Expired due to Fees

• Current Assignee: ABBYY Production LLC (ABBYY Software)
• Original Assignee: ABBYY InfoPoisk LLC
• Inventors: Zuev, Konstantin
• Primary Examiner(s): ROBERTS, SHAUN A
• Application Number:
  US13/173,649
• Publication Number:
  US 20110270607A1
• Time in Patent Office: 1,461 Days
• Field of Search: 704 9- 10, 707/705, 707/708, 707/711, 707/713, 707/736
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 40/237 Lexical tools
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

A method and system comprising an automated analysis of at least one corpus of natural language text is disclosed. For each sentence of a corpus, the analysis includes performing a syntactic analysis using linguistic descriptions to generate at least one syntactic structure for the sentence, building a semantic structure for the sentence, associating each generated syntactic and semantic structure with the sentence, and saving each generated syntactic and semantic structure. For each corpus text that was preliminary analyzed, performing an indexing operation to index lexical meanings and values of linguistic parameters of each syntactic structure and each semantic structure associated with sentences in the corpus text. A semantic search as disclosed herein includes at least one automatic preliminary analyzed corpus of sentences comprising searched values of linguistic, syntactic and semantic parameters. Due to deep semantic analysis of one or more corpora, the search may be executed in various languages, in resources of various languages, and in text corpora of various languages, regardless of the language of the query.

Claims

1. A computer-implemented method for facilitating a semantic search based on one or more previously analyzed and indexed corpuses of natural language texts, the method comprising:

identifying the one or more previously analyzed and indexed corpuses of natural language texts, each corpus including a plurality of sentences and indexes where indexes include indexes of meanings of linguistic parameters of the sentences and of their lexical units and their relations;

wherein the linguistic parameters comprise parameters determined based on syntactico-semantic analysis of at least one sentence in the corpuses; and

the syntactico-semantic analysis of the at least one sentence comprises

performing a rough syntactic analysis of the at least one sentence generating a graph of generalized constituents of the at least one sentence;

performing a precise syntactic analysis on the graph of generalized constituents by generating one or more syntactic trees and determining a syntactic structure of the at least one sentence,

wherein the performing of the precise syntactic analysis comprises assessing parts of the one or more syntactic trees using prior and

statistical ratings, and

generating the one or more syntactic trees in an order of decreasing assessment;

semantically analyzing the syntactic structure of the sentence generating a language-independent semantic structure of the at least one sentence; and

generating one or more of the parameters based on the language-independent semantic structure of the at least one sentence;

creating a query for searching sentences satisfying the query;

searching for sentences of the previously analyzed and indexed corpuses satisfying the query; and

displaying the sentences satisfying the query.

2. The method of claim 1, wherein indexing each corpus of natural language texts comprises:

creating an index for each meaning of linguistic parameters of generated syntactic structures;

creating an index for each meaning of parameters of semantic structures;

storing each generated syntactic structure and the semantic structure for each sentence of the corpora of natural language texts; and

storing each created index.

3. The method of claim 2, wherein indexing each corpus of natural language texts further comprises:

creating indexes of combinations of two or more meanings of linguistic parameters of generated syntactic structures;

creating indexes of combinations of two or more meanings of parameters of semantic structures; and

storing each created index.

4. The method of claim 1, wherein the performing the syntactic analysis includes resolving an ambiguity of lexical unit meanings within at least one of the sentences of the corpora of text.

5. The computer-implemented method of claim 1, wherein the generating the language-independent semantic structure includes encoding the language-specific properties of the sentence and of its lexical units and their relations by means of language-independent parameters.

6. The method of claim 1, wherein the query for searching is expressed in a specific query language.

7. The method of claim 1, wherein the query for searching is expressed in a natural language.

8. The method of claim 7, wherein the query for searching is represented in a natural language sentence with lacunas, where said lacunas may be covered during searching, at least, with one or more separate words, one or more word combinations, a phrase, a group of words represented by a bound constituent (substructure), or one or more embedded clauses.

9. The method of claim 1, wherein the query for searching is expressed in terms of requested parameters, a part of which may be indefinite, or may be defined by means of one or more variables, or may be defined as a range of meanings of these variables.

10. The method of claim 9, wherein said requested parameters represent, at least one of the following:

relationships, non-tree links, lexical classes, semantic classes, words or words combinations by grammatical properties, lexical properties, syntactical or semantic properties, relations by syntactical properties, or clauses by any (lexical, syntactical, semantic) properties.

11. The method of claim 9, wherein said requested parameters represent properties of relationships including surface or deep slots with or without checking a parent relationship, an anaphoric relationship or any quantity of nodes and relationships with said properties.

12. The method of claim 1, wherein the query for searching is expressed in a natural language with requested parameters, a part of which may be not defined, or may be defined by means of variables, or may be defined as a range of meanings of these variables.

13. The method of claim 12, wherein said requested parameters represent properties of searched nodes of syntactic or semantic structures, including grammatical values, classes with all descendants (superclasses), and semantemes.

14. The method of claim 1, wherein displaying the sentences satisfying the query includes sorting a query result by frequency.

15. One or more non-transitory computer-readable media having computer-executable instructions, which when executed perform steps, comprising:

identifying one or more previously analyzed and indexed corpuses of natural language texts, each corpus including a plurality of sentences and indexes where indexes include indexes of meanings of linguistic parameters of the sentences and of their lexical units and their relations;

wherein the linguistic parameters comprise semantic parameters determined based on syntactico-semantic analysis of at least one sentence in the corpuses; and

the syntactico-semantic analysis of the at least one sentence comprises performing a rough syntactic analysis of the at least one sentence generating a graph of generalized constituents of the at least one sentence;

performing a precise syntactic analysis on the graph of generalized constituents by generating one or more syntactic trees and determining a syntactic structure of the at least one sentence,

wherein the performing of the precise syntactic analysis comprises

assessing parts of the one or more syntactic trees using prior and statistical ratings, and

generating the one or more syntactic trees in an order of decreasing assessment;

semantically analyzing the syntactic structure of the sentence generating a language-independent semantic structure of the at least one sentence;

generating one or more of the parameters based on the language-independent semantic structure of the at least one sentence;

creating a query for searching sentences satisfying the query;

searching for sentences of the previously analyzed and indexed corpuses satisfying the query; and

displaying the sentences satisfying the query.

16. The one or more non-transitory computer-readable media of claim 15, wherein indexing each corpus of natural language texts comprises:

creating an index for each meaning of linguistic parameters of generated syntactic structures;

creating an index for each meaning of parameters of semantic structures;

storing each generated syntactic structure and the semantic structure for each sentence of the corpora of natural language texts; and

storing each created index.

17. The one or more non-transitory computer-readable media of claim 16, wherein indexing each corpus of natural language texts further comprises:

creating indexes of combinations of two or more meanings of linguistic parameters of generated syntactic structures;

creating indexes of combinations of two or more meanings of parameters of semantic structures; and

storing each created index.

18. The one or more non-transitory computer-readable media of claim 15, wherein the performing the syntactic analysis includes resolving an ambiguity of lexical unit meanings within at least one of the sentences of the corpora of text.

19. The one or more non-transitory computer-readable media of claim 15, wherein the generating the language-independent semantic structure includes encoding the language-specific properties of the sentence and of its lexical units and their relations by means of language-independent parameters.

20. The one or more non-transitory computer-readable media of claim 15, wherein the query for searching is expressed in a specific query language.

21. The one or more non-transitory computer-readable media of claim 15, wherein the query for searching is expressed in a natural language.

22. The one or more non-transitory computer-readable media of claim 15, wherein the query for searching is expressed in terms of one or more requested parameters, one or more of which may be indefinite, or may be defined by means of one or more variables, or may be defined as a range of meanings of these variables.

23. The one or more non-transitory computer-readable media of claim 22, wherein said requested parameters represent, at least one of the following:

a relationship, a non-tree link, a lexical class, a semantic class, one or more word or word combinations of grammatical properties, one or more word or word combinations of lexical properties, one or more word or word combinations of syntactical or semantic properties, one or more relationships represented by one or more syntactical properties, or clauses of a (lexical, syntactical, or semantic) property. parameters, one or more of which may be indefinite, or may be defined by means of one or more variables, or may be defined as a range of meanings of these variables.

24. The one or more non-transitory computer-readable media of claim 22, wherein said requested parameters represent properties of relationships including surface or deep slots with or without checking a parent relationship, an anaphoric relationship or any quantity of nodes and relationships with said properties.

25. The one or more non-transitory computer-readable media of claim 15, wherein the query for searching is expressed in a natural language with requested parameters, a part of which may be indefinite, or may be defined by means of variables, or may be defined as a range of meanings of these variables.

26. The one or more non-transitory computer-readable media of claim 25, wherein said requested parameters represent properties of searched nodes of syntactic or semantic structures, including grammatical values, classes with all descendant (superclass), semantemes (for example, “

time”

regardless of a form of its expression).

27. The one or more non-transitory computer-readable media of claim 15, wherein the query for searching is represented in a natural language sentence with one or more lacunas, where said lacunas may be covered during searching, at least, with one or more separate words, one or more word combinations, a phrase, a group of words represented by a bound constituent (substructure), or one or more embedded clauses.

28. The one or more non-transitory computer-readable media of claim 15, wherein displaying the sentences satisfying the query includes sorting a query result by frequency.