Method for tagging collocations in text

• US 5,383,120 A
• Filed: 03/02/1992
• Issued: 01/17/1995
• Est. Priority Date: 03/02/1992
• Status: Expired due to Fees

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Zernik, Uri
• Primary Examiner(s): Knepper, David D.
• Application Number:
  US07/844,026
• Time in Patent Office: 1,051 Days
• Field of Search: 395/2, 395/2.66, 364/419.02, 364/419.08, 364/419.07
• US Class Current: 704/10
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique

Abstract

A technique for injecting corpus-based preference into syntactic text parsing is provided. Specifically, the problem of tagging content-word pairs by part-of-speech is solved by using thematic analysis. A new measure of the fixed or variable nature of such word pairs is created and used to classify word pairs as either noun-verb, adjective-noun, or verb-noun.

Claims

1. A method for performing thematic part-of-speech tagging for collocations having content-word pairs in a natural language text processing system comprising the steps of:

identifying collocations of content-word pairs in a large corpus of text;

calculating, for each of said collocation content-word pair identified, a variability factor which is a measure of variability of said collocation content-word pairs occurring in said text;

storing said collocation content word pairs and associated variability factors in a collocation database; and

using said database to tag collocation content-word pairs according to said variability factors, wherein collocation content-word pairs with high variability factors are tagged as having a verb and a noun thereat and collocation content-word pairs with low variability factors are tagged as having an adjective and a noun thereat or a noun and noun thereat.

2. The method of claim 1 wherein a collocation content-word pair and associated variability factor are stored in said database when the mutual information score for said collocation content-word pair is above a selected threshold.

3. The method of claim 1 wherein said high variability factors exceed 0.75 and said low variability factors are less than or equal to 0.75.

4. The method of claim 1 comprising the additional step of using local context analysis to tag collocation content word pairs before using said collocation database.