Domain-specific lexically-driven pre-parser

• US 10,496,744 B2
• Filed: 11/03/2017
• Issued: 12/03/2019
• Est. Priority Date: 08/17/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Boguraev, Branimir K., Manandise, Esme, Segal, Benjamin P.
• Primary Examiner(s): Zhu, Richard Z
• Application Number:
  US15/803,275
• Publication Number:
  US 20190057076A1
• Time in Patent Office: 760 Days
• Field of Search: None
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

A method includes obtaining an input text, identifying a first term in the input text, and accessing lexicon data to identify a first entry corresponding to the first term. The first entry includes core data corresponding to domain-independent lexical information for the first term, and non-core data corresponding to domain-specific lexical information for the first term. The method also includes determining that the non-core data of the first entry identifies a second term in the input text as a modifier of the first term. The method further includes generating a partially parsed and bracketed version of the input text. The partially parsed and bracketed version indicates that the second term modifies the first term in the input text. The method also includes generating a parsed version of the input text based on the partially parsed and bracketed version of the input text.

Claims

1. A method comprising:

obtaining an input text at a text parser of a device, the text parser includes a domain-specific lexically-driven pre-parser and a domain-independent rule-based parser;

determining that the input text is associated with a first domain that is associated with a domain-specific corpus;

identifying, at the device, a first term in the input text;

accessing, at the device, lexicon data to identify a first entry corresponding to the first term, the first entry including core data and non-core data, wherein the core data corresponds to domain-independent lexical information for the first term, and wherein the non-core data corresponds to domain-specific lexical information for the first term and second domain-specific lexical information for the first term, wherein the domain-specific lexical information corresponds to the first domain and the second domain-specific lexical information corresponds to a second domain distinct from the first domain;

determining, at the domain-specific lexically-driven pre-parser, that the non-core data of the first entry identifies a second term in the input text as a modifier of the first term;

generating, at the domain-specific lexically-driven pre-parser, a partially parsed and bracketed version of the input text, wherein the partially parsed and bracketed version indicates that the second term modifies the first term in the input text, wherein, responsive to the input text not being associated with the second domain, the partially parsed and bracketed version is generated independent of the second domain-specific lexical information; and

generating, at the domain-independent rule-based parser, a parsed version of the input text based on the partially parsed and bracketed version of the input text.

2. The method of claim 1, wherein the non-core data is based on an analysis of a domain-specific corpus.

3. The method of claim 1, wherein the non-core data is based on at least one of user input or co-occurrence statistics, and wherein the co-occurrence statistics are generated based on an analysis of a domain-specific corpus.

4. The method of claim 1, wherein the second term includes at least one of an adjectival modifier term, a preposition modifier term, or a nominal modifier term.

5. The method of claim 1, wherein the partially parsed and bracketed version of the input text is generated by the domain-specific lexically-driven pre-parser based on domain-specific parsing rules.

6. The method of claim 5, wherein the domain-specific parsing rules include at least one of a collocation rule, a morpho-semantic rule, a name-entity-based pattern rule, or a semantico-syntactic pattern rule.

7. The method of claim 1, further comprising determining, at the domain-specific lexically-driven pre-parser, that the non-core data identifies one or more second particular terms in the input text as one or more additional domain-specific modifiers of the first term, wherein the partially parsed and bracketed version indicates that the one or more second particular terms modify the first term in the input text.

8. The method of claim 1, further comprising:

determining that the input text is associated with a third domain that is associated with a third domain-specific corpus, wherein the non-core data further corresponds to third domain-specific lexical information for the first term;

determining, at a second domain-specific lexically-driven pre-parser, that the non-core data of the first entry identifies a third term in the input text as a modifier of the first term;

generating, at the second domain-specific lexically-driven pre-parser, a second partially parsed and bracketed version of the input text, wherein the partially parsed and bracketed version indicates that the third term modifies the first term in the input text, and wherein the parsed version of the input text is generated based on the second partially parsed and bracketed version of the input text.

9. The method of claim 8, wherein the non-core data includes one or more additional modifier terms for the first term for one or more additional domains, wherein the first domain is distinct from each of the one or more additional domains.

10. The method of claim 9, wherein the first domain includes at least one of medicine, engineering, art, music, finance, oil &

gas, English, French, Spanish, or another specialized domain, and wherein the second domain of the additional domains includes at least one of medicine, engineering, art, music, finance, oil &

gas, English, French, or Spanish.

11. The method of claim 1, wherein the domain-independent rule-based parser generates the parsed version of the input text independently of domain-specific rules.

12. The method of claim 1, wherein the non-core data is based on at least one of co-occurrence statistics or user input, and wherein the co-occurrence statistics indicate a number of times the second term modifies the first term in a domain-specific corpus.

13. The method of claim 1, wherein software corresponding to the domain-specific lexically-driven pre-parser is provided as a service in a cloud computing environment.