DOMAIN-SPECIFIC LEXICALLY-DRIVEN PRE-PARSER

• US 20190057075A1
• Filed: 08/17/2017
• Published: 02/21/2019
• Est. Priority Date: 08/17/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Boguraev, Branimir K., Manandise, Esme, Segal, Benjamin P.
• Granted Patent: US 10,445,423 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

A computer program product including a computer-readable storage medium that has program instructions embodied therewith. The program instructions executable to cause the processor to perform operations including obtaining an input text at a text parser, identifying a first term in the input text, and accessing lexicon data to identify a first entry corresponding to the first term. The first entry includes non-core data that corresponds to domain-specific lexical information for the first term. The operations further include determining that the non-core data identifies a second term in the input text as a modifier of the first term. The operations also include generating a partially parsed and bracketed (PPB) version of the input text. The PPB version indicates that the second term modifies the first term in the input text. The operations further include generating a parsed version of the input text based on the PPB version of the input text.

Claims

1-13. -13. (canceled)

14. A computer program product for domain-specific parsing, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations comprising:

obtaining an input text at a text parser, the text parser includes a domain-specific lexically-driven pre-parser and a domain-independent rule-based parser;

identifying a first term in the input text;

accessing lexicon data to identify a first entry corresponding to the first term, the first entry including core data and non-core data, wherein the core data corresponds to domain-independent lexical information for the first term, and wherein the non-core data corresponds to domain-specific lexical information for the first term;

determining, at the domain-specific lexically-driven pre-parser, that the non-core data of the first entry identifies a second term in the input text as a modifier of the first term;

generating, at the domain-specific lexically-driven pre-parser, a partially parsed and bracketed version of the input text, wherein the partially parsed and bracketed version indicates that the second term modifies the first term in the input text; and

generating, at the domain-independent rule-based parser, a parsed version of the input text based on the partially parsed and bracketed version of the input text.

15. The computer program product of claim 14, wherein the domain-independent rule-based parser generates the parsed version of the input text independently of domain-specific rules.

16. A system comprising:

a memory configured to store input text and lexicon data;

a processor configured to;

identify a first term in the input text; and

access the lexicon data to identify a first entry corresponding to the first term, the first entry including core data and non-core data, wherein the core data corresponds to domain-independent lexical information for the first term, and wherein the non-core data corresponds to domain-specific lexical information for the first term;

a domain-specific lexically-driven pre-parser configured to;

determine that the non-core data of the first term identifies a second term in the input text as a modifier of the first term; and

generate a partially parsed and bracketed version of the input text, wherein the partially parsed and bracketed version indicates that the second term modifies the first term in the input text; and

a domain-independent rule-based parser configured to generate a parsed version of the input text based on the partially parsed and bracketed version of the input text.

17. The system of claim 16, wherein the processor is configured to generate co-occurrence statistics based on an analysis of a domain-specific corpus, wherein the co-occurrence statistics indicate a number of times the second term modifies the first term in the domain-specific corpus, and wherein the non-core data is based on at least one of user input or the co-occurrence statistics.

18. The system of claim 16, wherein the second term includes at least one of an adjectival modifier term, a preposition modifier term, or a nominal modifier term.

19. The system of claim 16, wherein the partially parsed and bracketed version of the input text is generated by the domain-specific lexically-driven pre-parser based on domain-specific parsing rules.

20. The system of claim 19, wherein the domain-specific parsing rules include at least one of a collocation rule, a morpho-semantic rule, a name-entity-based pattern rule, or a semantico-syntactic pattern rule.