Extraction of lexical kernel units from a domain-specific lexicon

• US 9,588,959 B2
• Filed: 01/09/2015
• Issued: 03/07/2017
• Est. Priority Date: 01/09/2015
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Boguraev, Branimir K., Manandise, Esme, Segal, Benjamin P.
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US14/593,015
• Publication Number:
  US 20160203119A1
• Time in Patent Office: 788 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/268 Morphological analysis
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation

Abstract

According to an aspect, a candidate lexical kernel unit that includes a word token sequence having two or more words is received. Domain terms that contain the two or more words are retrieved from a terminology resource file of domain terms associated with a domain. The candidate lexical kernel unit and the retrieved domain terms are analyzed to determine whether the candidate lexical kernel unit satisfies specified criteria for use as a building block by a natural-language processing (NLP) tool for building larger lexical units in the domain. Each of the larger lexical units includes a greater number of words than the candidate lexical kernel unit. The candidate lexical kernel unit is identified as a lexical kernel unit based on determining that the candidate lexical kernel unit satisfies the specified criteria. The lexical kernel unit is output to a domain-specific lexical kernel unit file for input to the NLP tool.

Claims

1. A computer program product comprising:

a tangible storage medium readable by a processor and storing instructions for execution by the processor to perform a method comprising;

receiving a candidate lexical kernel unit comprising a word token sequence that includes two or more words;

retrieving domain terms that contain the two or more words from a terminology resource file of domain terms associated with a domain;

analyzing the candidate lexical kernel unit and the retrieved domain terms to determine whether the candidate lexical kernel unit satisfies specified criteria for use as a building block by a natural-language processing (NLP) tool for building larger lexical units in the domain, each of the larger lexical units including a greater number of words than the candidate lexical kernel unit;

identifying the candidate lexical kernel unit as a lexical kernel unit based on determining that the candidate lexical kernel unit satisfies the specified criteria; and

outputting the lexical kernel unit to a domain-specific lexical kernel unit file for input to the NLP tool for use as a lexical resource in parsing natural language text in the domain, the parsing including identifying domain-specific terms in the natural language text in the domain.

2. The computer program product of claim 1, wherein linguistic information collected during the analyzing is further output based on determining that the candidate lexical kernel unit satisfies the specified criteria.

3. The computer program product of claim 1, wherein the specified criteria includes the number of domain terms retrieved from the terminology resource file.

4. The computer program product of claim 1, wherein the specified criteria includes one or more types of patterns in the retrieved domain terms.

5. The computer program product of claim 1, wherein at least two of the two or more words included in the candidate lexical kernel unit are not contiguous in least one of the retrieved domain terms.

6. The computer program product of claim 1, wherein the analyzing is based on heuristics.

7. The computer program product of claim 1, wherein

the retrieved domain terms are each part of an entry in the terminology resource file that includes a domain term, a tagged entry that identifies a headword in the entry, and an entry pattern that identifies a part-of-speech in the entry, and

the entry is input to the analyzing.

8. The computer program product of claim 1, wherein the method further comprises repeating the receiving, retrieving, analyzing, identifying, and outputting for additional candidate lexical kernel units.

9. A system comprising:

a memory having computer readable instructions; and

a processor for executing the computer readable instructions, the computer readable instructions including;

receiving a candidate lexical kernel unit comprising a word token sequence that includes two or more words;

retrieving domain terms that contain the two or more words from a terminology resource file of domain terms associated with a domain;

analyzing the candidate lexical kernel unit and the retrieved domain terms to determine whether the candidate lexical kernel unit satisfies specified criteria for use as a building block by a natural-language processing (NLP) tool for building larger lexical units in the domain, each of the larger lexical units including a greater number of words than the candidate lexical kernel unit;

identifying the candidate lexical kernel unit as a lexical kernel unit based on determining that the candidate lexical kernel unit satisfies the specified criteria; and

outputting the lexical kernel unit to a domain-specific lexical kernel unit file for input to the NLP tool for use as a lexical resource in parsing natural language text in the domain, the parsing including identifying domain-specific terms in the natural language text in the domain.

10. The system of claim 9, wherein linguistic information collected during the analyzing is further output based on determining that the candidate lexical kernel unit satisfies the specified criteria.

11. The system of claim 9, wherein the specified criteria includes at least one of the number of domain terms retrieved from the terminology resource file and one or more types of patterns in the retrieved domain terms.

12. The system of claim 9, wherein

the retrieved domain terms are each part of an entry in the terminology resource file that includes a domain term, a tagged entry that identifies a headword in the entry, and an entry pattern that identifies a part-of-speech in the entry, and

the entry is input to the analyzing.