METHOD AND SYSTEM FOR ANALYZING VARIOUS LANGUAGES AND CONSTRUCTING LANGUAGE-INDEPENDENT SEMANTIC STRUCTURES

• US 20080091405A1
• Filed: 10/10/2006
• Published: 04/17/2008
• Est. Priority Date: 10/10/2006
• Status: Active Grant

• Current Assignee: ABBYY Development LLC
• Original Assignee: ABBYY Software
• Inventors: Zuev, Konstantin, Anisimovich, Konstantin, Selegey, Vladimir
• Granted Patent: US 8,078,450 B2
• Time in Patent Office: Days
• US Class Current: 704/4
• CPC Class Codes: G06F 40/268 Morphological analysis
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/55 Rule-based translation

Abstract

A method and computer system for analyzing sentences of various languages and constructing a language-independent semantic structure are provided. On the basis of comprehensive knowledge about languages and semantics, exhaustive linguistic descriptions are created, and lexical, morphological, syntactic, and semantic analyses for one or more sentences of a natural or artificial language are performed. A computer system is also provided to implement, analyze and store various linguistic structures and to perform lexical, morphological, syntactic, and semantic analyses. As result, a generalized data structure, such as a semantic structure, is generated and used to describe the meaning of one or more sentences in language-independent form, applicable to automated abstracting, machine translation, control systems, Internet information retrieval, etc.

Claims

1. A method of analyzing a sentence in a source language, comprising:

performing a lexical analysis of the sentence in the source language;

performing a lexical-morphological analysis of the each element of the sentence and building a lexical-morphological structure for the whole sentence;

performing a rough syntactic analysis on the lexical-morphological structure of the sentence and generating a graph of generalized constituents;

performing a precise syntactic analysis on the graph of the generalized constituents and generating one or more syntactic trees for the sentence from the graph of the generalized constituents;

selecting a best syntactic structure for the sentence from the one or more syntactic trees; and

performing a semantic analysis on the best syntactic structure of the sentence and generating a language-independent semantic structure for the sentence of the language.

2. The method of claim 1, wherein the lexical-morphological analysis further comprises finding for each element of the sentence a complete set of its grammatical and lexical meanings.

3. The method of claim 1, further comprising:

using one or more morphological descriptions and one or more lexical descriptions of the source language to generate the lexical-morphological structure of the sentence.

4. The method of claim 1, further comprising:

using one or more lexical descriptions, one or more semantic descriptions, and one or more syntactic descriptions to perform the rough syntactic analysis.

5. The method of claim 4, wherein the rough syntactic analysis further comprises:

building a set of all possible generalized constituents for each element of the sentence; and

building the graph of the generalized constituents to represent the entire sentence from a set of all generalized constituents.

6. The method of claim 5, wherein filtering of the constituents is performed prior to and after building the graph of the generalized constituents.

7. The method of claim 5, wherein building the graph of the generalized constituents further comprises performing coordination processing.

8. The method of claim 5, wherein building the graph of the generalized constituents further comprises performing ellipsis restoration.

9. The method of claim 1, wherein the precise syntactic analysis further comprises:

generating a graph of precise constituents;

generating one or more syntactic structure variants from the graph of the precise constituents;

rating the precise constituents based on a plurality of rating scores independently obtained and calculated, including the rating scores of one or more lexical meanings for each element of the sentence, the rating scores of one or more syntactic constructions for each element of the sentence, the rating scores of the degree of correspondence of the precise constituents to their semantic descriptions, and the rating scores of the linear order;

using the rating scores to generate hypotheses about the overall syntactic structure of the sentence; and

selecting one or more best hypotheses about the overall syntactic structure of the sentence with the highest rating score from the hypotheses generated.

10. The method of claim 9, further comprising returning back to generating the graph of the precise constituents if the precise syntactic analysis is unsuccessful.

11. The method of claim 1, wherein the precise syntactic analysis further comprises generating one or more syntactic structure variants with established non-tree links and selecting the best syntactic structure for the sentence from the syntactic structure variants with established non-tree links.

12. The method of claim 1, further comprising returning back to performing the rough syntactic analysis if the precise syntactic analysis is unsuccessful.

13. The method of claim 1, wherein the semantic analysis further comprises:

establishing correspondences between each surface slot of the syntactic structure and deep semantic slot for the sentence using diathesis correspondences;

interpreting a grammatical value for each lexical meaning of the syntactic structures against a set of semantemes;

substituting each lexical meaning with a language-independent semantic class; and

registering distinctive semantic features of each lexical meaning with differentiating semantemes.

14. The method of claim 1, wherein the semantic analysis is performed using pragmatic context.

15. The method of claim 1, wherein the source language comprises a natural language selected from the group consisting of English, French, German, Italian, Russian, Spanish, Ukrainian, Dutch, Danish, Swedish, Finnish, Portuguese, Slovak, Polish, Czech, Hungarian, Lithuanian, Latvian, Estonian, Greek, Bulgarian, Turkish, Tatar, Hindi, Serbian, Croatian, Romanian, Slovenian, Macedonian, Japanese, Korean, Chinese, Arabic, Hebrew, and Swahili.

16. A method of analyzing a sentence in a source language, comprising:

performing a lexical analysis of the sentence in the source language;

performing a lexical-morphological analysis on each element of the sentence and building a lexical-morphological structure for the whole sentence;

performing a rough syntactic analysis on the lexical-morphological structure of the sentence using one or more lexical descriptions, one or more semantic descriptions, and one or more syntactic descriptions;

building a set of all possible generalized constituents for each element of the sentence;

building the graph of the generalized constituents to represent the entire sentence from a set of all generalized constituents;

performing a precise syntactic analysis on the graph of the generalized constituents and generating a graph of precise constituents;

generating one or more syntactic trees from the graph of the precise constituents;

selecting a best syntactic structure for the sentence from the one or more syntactic trees;

performing a semantic analysis on the best syntactic structure of the sentence; and

generating a language-independent semantic structure for the sentence of the language.

17. The method of claim 16, wherein filtering of the constituents is performed prior to and after building the graph of the generalized constituents.

18. The method of claim 16, wherein building the graph of the generalized constituents further comprises performing coordination processing and ellipsis restoration.

19. The method of claim 16, wherein the precise syntactic analysis further comprises:

rating the graph of the precise constituents based on a plurality of rating scores independently obtained and calculated, including the rating scores of one or more lexical meanings for each element of the sentence, the rating scores of one or more syntactic constructions for each element of the sentence, the rating scores of the degree of correspondence of the precise constituents to their semantic descriptions, and the rating scores of the linear order;

using the rating scores to generate hypotheses about the overall syntactic structure of the sentence; and

selecting one or more best hypotheses about the overall syntactic structure of the sentence with the highest rating score from the hypotheses generated.

20. The method of claim 16, wherein the precise syntactic analysis further comprises generating one or more syntactic structure variants with established non-tree links and selecting a best syntactic structure for the sentence from the syntactic structures with established non-tree links.

21. A computer readable medium comprising instructions for causing a computing system to carry out steps comprising:

performing a lexical analysis of the sentence in the source language;

performing a lexical-morphological analysis on the each element of the sentence and building a lexical-morphological structure for the whole sentence;

performing a rough syntactic analysis on the lexical-morphological structure of the sentence and generating a graph of generalized constituents from the lexical-morphological structure of the sentence;

performing a precise syntactic analysis on the graph of the generalized constituents and generating one or more syntactic structures for the sentence from the graph of the generalized constituents; and

performing a semantic analysis on the syntactic structures of the sentence and generating a language-independent semantic structure for the sentence of the language.

22. A computer system adapted to analyze a sentence of a language, comprising:

a lexical-morphological analyzer adapted to perform a lexical analysis and a lexical-morphological analysis on each element of the sentence and generating a lexical-morphological structure of the sentence;

a rough syntactic analyzer adapted to perform a rough syntactic analysis on the lexical-morphological structure of the sentence and generate a graph of generalized constituents from the lexical-morphological structure of the sentence;

a precise syntactic analyzer adapted to perform a precise syntactic analysis on the graph of the generalized constituents and generate a syntactic structure of the sentence from the graph of the generalized constituents; and

a semantic analyzer adapted to perform a semantic analysis on the syntactic structure of the sentence and generate a language-independent semantic structure for the sentence of the language.