Deep analysis of natural language questions for question answering system

• US 9,471,559 B2
• Filed: 03/15/2013
• Issued: 10/18/2016
• Est. Priority Date: 12/10/2012
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Castelli, Vittorio, Florian, Radu, Luo, Xiaoqiang, Maskey, Sameer, Raghavan, Hema
• Primary Examiner(s): Spooner, Lamont
• Application Number:
  US13/835,924
• Publication Number:
  US 20140163962A1
• Time in Patent Office: 1,313 Days
• Field of Search: 704/1, 704/9, 704/10
• US Class Current: 1/1
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

Creating training data for a natural language processing system may comprise obtaining natural language input, the natural language input annotated with one or more important phrases; and generating training instances comprising a syntactic parse tree of nodes representing elements of the natural language input augmented with the annotated important phrases. In another aspect, a classifier may be trained based on the generated training instances. The classifier may be used to predict one or more potential important phrases in a query.

Claims

1. A method for creating training data for a natural language processing system comprising:

obtaining natural language input, the natural language input annotated with one or more important phrases;

generating, by a processor, training instances comprising a syntactic parse tree of nodes representing elements of the natural language input augmented with the annotated important phrases,

the generating comprising at least creating a syntactic node, which augments the syntactic parse tree output by an information extraction pipeline, for a named mention annotated as an important phrase, responsive to determining that no existing non-terminal nodes in the syntactic parse tree span exactly the named mention; and

training a classifier based on the generated training instances,

wherein the natural language processing system executes the classifier in determining a semantic meaning of a given query.

2. The method of claim 1, wherein the training a classifier based on the generated training instances comprises computing a plurality of features associated with the training instances and assigning a score to each of the features.

3. The method of claim 2, wherein the plurality of features comprises one or more of lexical features, syntactic features, semantic features or corpus-based features, or combinations thereof.

4. The method of claim 1, wherein the classifier comprises a statistical classifier.

5. The method of claim 1, wherein the syntactic parse tree is generated by processing the natural language input with the annotated important phrases by an information extraction pipeline comprising syntactic parsing, mention detection and coreference resolution.

6. A method for natural language processing comprising steps of:

receiving a natural language query;

creating, by one or more processors, a query syntactic tree for the query;

using, by one or more of the processors, a trained model comprising a model syntactic tree to predict if a node in the query syntactic tree is important,

wherein the model syntactic tree comprises a syntactic parse tree of nodes representing syntactic elements of natural language input augmented with annotated important phrases, the model syntactic tree generated by at least creating a syntactic node, which augments the syntactic parse tree created by an information extraction pipeline for a named mention annotated as an important phrase, responsive to determining that no existing non-terminal nodes of the syntactic parse tree span exactly the named mention,

wherein a semantic meaning of the natural language query is determined based on the model syntactic tree.

7. The method of claim 6, further comprising assigning a score for each node in the query syntactic tree.

8. The method of claim 7, further comprising synthesizing scores for a plurality of nodes.

9. The method of claim 6, further comprising assigning a score for each node in the query syntactic tree and generating bi-lexical dependencies between nodes.

10. A non-transitory computer readable storage medium storing a program of instructions executable by a machine to perform a method of creating training data for a natural language processing system comprising:

obtaining natural language input, the natural language input annotated with one or more important phrases;

generating training instances comprising a syntactic parse tree of nodes representing elements of the natural language input augmented with the annotated important phrases,

the generating comprising at least creating a syntactic node, which augments the syntactic parse tree created by an information extraction pipeline for a named mention annotated as an important phrase, responsive to determining that no existing non-terminal nodes in the syntactic parse tree span exactly the named mention; and

training a classifier based on the generated training instances,

wherein the natural language processing system executes the classifier in determining a semantic meaning of a given query.

11. The computer readable storage medium of claim 10, wherein the training a classifier based on the generated training instances comprises computing a plurality of features associated with the training instances and assigning a score to each of the features.

12. The computer readable storage medium of claim 11, wherein the plurality of features comprises one or more of lexical features, syntactic features, semantic features or corpus-based features, or combinations thereof.

13. The computer readable storage medium of claim 10, wherein the classifier comprises a statistical classifier.

14. The computer readable storage medium of claim 10, further comprising predicting one or more potential important phrases in an input query using the trained classifier.

15. The computer readable storage medium of claim 10, wherein the syntactic parse tree is generated by processing the natural language input with the annotated important phrases by an information extraction pipeline comprising syntactic parsing, mention detection and coreference resolution.

16. A system for creating training data for a natural language processing system comprising:

a processor;

a training module operable to execute on the processor and further operable to obtain natural language input, the natural language input annotated with one or more important phrases, the training module further operable to generate training instances comprising a syntactic parse tree of nodes representing elements of the natural language input augmented with the annotated important phrases; and

a classifier built based on the training instances with plurality of features computed for the training instances,

wherein the training module is operable to generate training instances by at least creating a syntactic node, which augments the syntactic parse tree created by an information extraction pipeline, for a named mention annotated as an important phrase, responsive to determining that no existing non-terminal nodes in the syntactic parse tree span exactly the named mention, wherein a semantic meaning of a given natural language query is determined based on the augmented syntactic parse tree.

17. The system of claim 16, wherein the plurality of features comprises one or more of lexical features, syntactic features, semantic features or corpus-based features, or combinations thereof.

18. The system of claim 16, wherein the syntactic parse tree is generated by processing the natural language input with the annotated important phrases by an information extraction pipeline comprising syntactic parsing, mention detection and coreference resolution, and wherein the classifier is operable to predict one or more potential important phrases in an input query.