Method and large syntactical analysis system of a corpus, a specialised corpus in particular

• US 20040181389A1
• Filed: 04/19/2004
• Published: 09/16/2004
• Est. Priority Date: 06/01/2001
• Status: Abandoned Application

• Current Assignee: Centre National De La Recherche Scientifique, Synomia
• Original Assignee: Centre National De La Recherche Scientifique, Synomia
• Inventors: Bourigault, Didier, Fabre, Cecile
• Application Number:
  US10/479,233
• Publication Number:
  US 20040181389A1
• Time in Patent Office: Days
• US Class Current: 704/1
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/216 using statistical methods

Abstract

Method for large syntactical analysis based on unsupervised learning on a corpus comprising an iterative sequencing of two phases: a learning phase wherein linguistic information is acquired using unambiguous analysis cases, and a resolution phase wherein ambiguous analysis cases are resolved using information acquired during the learning phase. The invention is used in particular for creating specialized terminological resources for an information processing system, for creating an ontology for a specialized information search engine on the web, for a terminological lexicon for an automatic translation system, or for a thesaurus for an automatic indexing system.

Claims

1. Method of broad syntactic analysis based on unsupervised learning using a corpus, characterized in that it comprises an iterative sequencing of two phases:

a learning phase, in which the linguistic data are acquired from unambiguous analysis cases,

a resolution phase, in which ambiguous analysis cases are resolved using the data acquired during the learning phase.

2. Method of broad syntactic analysis of a corpora, in particular of a specialized corpora, according to claim 1, characterized in that the phases of learning and of resolution follow each other in an iterative way so that the resolved cases during a resolution phase serve as a basis for a new learning phase, and so on until no new case is not resolved.

3. Method according to claim 2, characterized in that it also comprises sequences of identification of relationships of dependence between words of the corpus in which each dependency relationship is described in the form of a triplet (X, R, Y) where X is the governor word (the source of the relationship), R is the noun of the dependency relationship and Y is the governed word (the target of the relationship), and in which each anaphoric relationship is described in the form of a triplet (X, ANA, Y), where X is a pronoun, ANA is the noun of the anaphoric relationship and Y its antecedent, the identification of these anaphoric relationships allowing the updating of indirect-dependency relationships.

4. Method according to claim 3, characterized in that it is applied to an entry corpus which has previously undergone a morphosyntatic labelling.

5. Method according to one of claims 3 or 4, characterized in that the processing of dependency relationships is based on potential governors.

6. Method according to one of claims 3 or 4, characterized in that the processing of the dependency relationships is based on potential governed.

7. Method according to one of claims 5 or 6, characterized in that in a sequence of identification of dependency relationship, the starting point is a pivot word (governor or governed respectively) and a dependency relationship and a word is sought which enters into a dependency relationship with it (subject or governor respectively).

8. Method according to claim 7, characterized in that it also comprises a stage (0) of acquisition of data comprising an acquisition of earlier derivative morphological data in which, by analysis of the corpus, word pairs are acquired, from different categories, which are able to be in a relationship of morphological derivation.

9. Method according to claim 8, characterized in that the acquisition stage (0) is followed by a searching stage (1), for each pivot word (governor, governed respectively), candidate words to be governed (or governor).

10. Method according to claim 9, characterized in that the stage (1) of searching includes running sequentially through the words of a sentence starting from the pivot word.

11. Method according to claim 10, characterized in that at the end of the stage (1) of searching, each adopted candidate is assigned a coefficient of accessibility linked to the distance from the pivot word and to the type of words inserted between said candidate and said pivot word.

12. Method according to one of claims 9 to 11, characterized in that the stage (1) of searching includes an identification of the incompatible solutions.

13. Method according to one of claims 9 to 12, characterized in that the stage (1) of searching is followed by a stage (2) of endogenous learning comprising:

a recognition of triplets each constituted by a pivot word, a dependency relationship and a single candidate, leading to what are called resolved cases,

a recognition of triplets each constituted by a pivot word, a dependency relationship and several competing candidates, leading to what are called ambiguous cases.

14. Method according to claim 13, characterized in that the stage of endogenous learning includes an acquisition of data called complementation involving a word and a preposition in the analysed corpus, and an acquisition of distributional proximity data involving two words of the same category that are semantically close and distributed in more or less identical syntactic contexts in the analysed corpus.

15. Method according to claim 14, characterized in that the complementation data comprise what are called productivity coefficients and the distributional proximity data comprise what are called proximity coefficients.

16. Method according to claim 15, characterized in that the productivity coefficients include a governor productivity coefficient that corresponds, for a triplet constituted by a word M, a preposition Prep and a category C, to the number of different words Y, of category C, for which the dependency relationship (M, Prep, Y) has been identified.

17. Method according to one of claims 14 or 15, characterized in that the productivity coefficients include a governed productivity coefficient that corresponds, for a triplet constituted by a word M, a preposition Prep and a category C, to the number of different words X, of category C, such that the dependency relationship (X, Prep, M) has been identified.

18. Method according to any one of claims 14 to 17, characterized in that the stage of endogenous learning also includes a processing of first-order syntactic contexts each corresponding to a pair (M, REL) where M is a word and REL is a dependency relationship.

19. Method according to any one of claims 14 to 18, characterized in that the endogenous learning stage also includes a processing of second-order syntactic contexts each corresponding to a quadruplet (M₁, M₂, REL₁ and REL₂) where M₁, and M₂ are words, and REL₁ and REL₂ relationships of dependence.

20. Method according to claims 18 and 19, characterized in that the endogenous learning stage also includes, for two words X, Y of the same category, a determination of a governed proximity coefficient between said two words X, Y:

governed proximity (X, Y)=a₁. N₁(X, Y)+a₂. N₂(X, Y)

where N₁(X, Y) is the number of first-order syntactic contexts in which X and Y have each been found, and N₂ (X, Y) is the number of second-order syntactic contexts in which X an Y have each been found.

21. Method according to claims 18 and 19 or claim 20, characterized in that the endogenous learning stage also includes a determination, for two first and second syntactic contexts (M₁,R₁) and (M₂,R₂), of a governor proximity coefficient equal to the number of words found in said first syntactic context and in said second syntactic context.

22. Method according to any one of the preceding claims, characterized in that the endogenous learning stage (2) is followed by a stage (3) of marking of the candidates, in which for each ambiguous case, each of the candidates is reviewed and is marked with one of the indicators, the values of which are calculated from data acquired during the endogenous learning phase.

23. Method according to claim 22, characterized in that during the stage (3) of marking, each candidate of each of the cases is assigned direct indicators calculated from data acquired from the candidate and from the pivot word themselves and derived indicators calculated from data acquired from morphological derived words linked to the candidate or to the pivot word.

24. Method according to claim 23, characterized in that the stage (3) of marking is followed by a stage (4) of resolution by default of the residual ambiguity cases if the data acquired during the endogenous learning stage (2) have not contributed to marking any candidate during the stage (3) of marking.

25. System of broad syntactic analysis on unsupervised learning on a corpus, using the process according to any one of the preceding claims, characterized in that it includes means of acquiring linguistic data on the unambiguous analysis cases, and means of resolving the ambiguous analysis cases comprising means of processing said acquired linguistic data.

26. System according to claim 25, characterized in that the data-acquisition means are set up to distinguish between unambiguous analysis cases and ambiguous analysis cases, and in that the processing means are set up to process the ambiguous analysis cases and to provide data allowing residual ambiguity cases to be resolved.

27. Use of the syntactic analysis method according to one of claims 1 to 24, for the construction of specialized terminological resources for a data-processing system.

28. Use of the method of syntactic analysis according to one of claims 1 to 24, for the construction of an ontology for a specialized-information search engine on the web.

29. Use of the method of syntactic analysis according to one of claims 1 to 24, for the construction of a terminological lexicon for an automatic translation system.

30. Use of the method of syntactic analysis according to one of claims 1 to 24, for the construction of a thesaurus for an automatic indexing system.