Displaying quality of question being asked a question answering system

• US 9,633,309 B2
• Filed: 06/19/2014
• Issued: 04/25/2017
• Est. Priority Date: 06/19/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: O'Keeffe, William G., Giffels, Kenneth M., Karle, Christopher J., Patel, Ketan T., Taieb, David D., Yee, Sabrina
• Primary Examiner(s): Holmes, Michael B
• Application Number:
  US14/309,637
• Publication Number:
  US 20150371137A1
• Time in Patent Office: 1,041 Days
• Field of Search: 706 46
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

A mechanism is provided in a data processing system for automatically generating question and answer pairs for training a question answering system for a given domain. The mechanism receives user input of question text for a question to be submitted to a question answering system in a user interface. The mechanism determines a question strength score for the question text. The question strength score represents a likelihood the question text will result in a correct answer with high confidence. The mechanism presents a graphical representation of the question strength score in the user interface.

Claims

1. A computer program product comprising a non-transitory computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a question answering system, causes the question answering system to:

receive question text comprising at least a portion of a question to be submitted to the question answering system in a user interface;

determine a question strength score for the question text, wherein the question strength score represents a likelihood the question text will result in a correct answer with high confidence;

present a graphical representation of the question strength score in the user interface;

generate a question forming tip for rephrasing the question text based on the comparison of a natural language processing structure of the question text and a question type of the question text to known good structures and types in a question history; and

present the question forming tip in the user interface.

2. The computer program product of claim 1, wherein determining the question strength score comprises:

parsing, by a parser in the question answering system, the question text to obtain the natural language processing structure of the question text; and

classifying, by a classifier in the question answering system, the question text to identify a question type.

3. The computer program product of claim 2, wherein determining the question strength score further comprises comparing the natural language processing structure of the question text and the question type to the known good question structures and question types in the question history.

4. The computer program product of claim 3, wherein determining the question strength score further comprises determining the question strength score based on confidence scores of the known good structures and types in the question history.

5. The computer program product of claim 1, wherein presenting the graphical representation of the question strength score comprises presenting the graphical representation in real time as a user inputs the question text into the user interface.

9. The computer program product of claim 2, wherein parsing the question text to obtain the natural language processing structure of the question text comprises identifying a focus or a lexical answer type.

10. The computer program product of claim 9, further comprising presenting a graphical indicator of the focus or the lexical answer type within the question text in the user interface.

11. The computer program product of claim 4, further comprising:

responsive to the user submitting the question text to the question answering system, generating a candidate answer having a respective confidence score; and

storing the question text, the natural language processing structure of the question text, the question type of the question text, and the confidence score in the question history.

12. The computer program product of claim 1, wherein presenting the question forming tip in the user interface comprises presenting the question forming tip in response to user interaction with the graphical representation of the question strength score.

6. An apparatus comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to;

receive question text comprising at least a portion of a question to be submitted to a question answering system in a user interface;

determine a question strength score for the question text, wherein the question strength score represents a likelihood the question text will result in a correct answer with high confidence;

present a graphical representation of the question strength score in the user interface;

generate a question forming tip for rephrasing the question text based on the comparison of a natural language processing structure of the question text and a question type of the question text to known good structures and types in a question history; and

present the question forming tip in the user interface.

7. The apparatus of claim 6, wherein determining the question strength score comprises:

parsing the question text to obtain a natural language processing structure of the question text;

classifying the question text to identify a question type;

comparing the natural language processing structure of the question text and the question type to known good structures and types in a question history; and

determining the question strength score based on confidence scores of the known good question structures and question types in the question history.

8. The apparatus of claim 6, wherein presenting the graphical representation of the question strength score comprises presenting the graphical representation in real time as a user inputs the question text into the user interface.

13. The apparatus of claim 7, wherein determining the question strength score further comprises comparing the natural language processing structure of the question text and the question type to the known good question structures and question types in the question history.

14. The apparatus of claim 13, wherein determining the question strength score further comprises determining the question strength score based on confidence scores of the known good structures and types in the question history.

15. The apparatus of claim 7, wherein parsing the question text to obtain the natural language processing structure of the question text comprises identifying a focus or a lexical answer type.

16. The apparatus of claim 15, further comprising presenting a graphical indicator of the focus or the lexical answer type within the question text in the user interface.

17. The apparatus of claim 14, further comprising:

responsive to the user submitting the question text to the question answering system, generating a candidate answer having a respective confidence score; and

storing the question text, the natural language processing structure of the question text, the question type of the question text, and the confidence score in the question history.

18. The apparatus of claim 6, wherein presenting the question forming tip in the user interface comprises presenting the question forming tip in response to user interaction with the graphical representation of the question strength score.