QUESTION CLASSIFICATION AND FEATURE MAPPING IN A DEEP QUESTION ANSWERING SYSTEM

• US 20140172756A1
• Filed: 12/17/2012
• Published: 06/19/2014
• Est. Priority Date: 12/17/2012
• Status: Active Grant

• Current Assignee: SINOEAST CONCEPT LIMITED (Tencent Holdings Limited)
• Original Assignee: International Business Machines Corporation
• Inventors: Clark, Adam T., Megerian, Mark G., Petri, John E., Stevens, Richard J.
• Granted Patent: US 9,754,215 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06Q 10/00 Administration; Management

Abstract

System, method, and computer program product to identify relevant features in a deep question answering system, by classifying a first case received by the deep question answering system, and, while training the deep question answering system to answer the first case, identifying a first feature in the first case, computing a first feature score for the first feature, the first feature score indicating a relevance of the first feature in generating a correct response to the first case, and, identifying the first feature as relevant in answering the classified first case upon determining that the first feature score exceeds a relevance threshold.

Claims

1. A computer program product to identify relevant features in a deep question answering system, the computer program product comprising:

a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code comprising;

computer-readable program code configured to classify a first case received by the deep question answering system; and

computer-readable program code configured to, while training the deep question answering system to answer the first case;

computer-readable program code configured to identify a first feature in the first case;

computer-readable program code configured to compute a similarity score for the first case and the second case; and

computer-readable program code configured to identify the first feature as relevant in generating a correct response to the second case upon determining that the similarity score exceeds a specified similarity threshold.

2. The computer program product of claim 1, further comprising:

computer-readable program code configured to, responsive to receiving a second case by the deep question answering system;

classify the second case;

compute a similarity score for the first case and the second case; and

identify the first feature as relevant in generating a correct response to the second case upon determining that the similarity score exceeds a specified similarity threshold.

3. The computer program product of claim 2, wherein the similarity score is based on at least one of:

(i) the classification of the first case and the classification second case, and (ii) a context of the first case and a context of the second case.

4. The computer program product of claim 2, wherein the first case and the second case are classified based on at least one of:

(i) an attribute of a user presenting the respective case to the deep question answering system, (ii) a content of a question in the respective case, and (iii) a type of the question in the respective case.

5. The computer program product of claim 4, wherein the attribute of the user is selected from the group comprising:

(i) a role of the user, (ii) an affiliation of the user, (iii) an expertise of the user, and (iv) a set of predefined preferences of the user.

6. The computer program product of claim 1, further comprising:

computer-readable program code configured to store a relationship between the first feature and the first case, wherein the relationship indicates that the first feature is relevant in answering the first case.

7. The computer program product of claim 1, wherein the first feature score is based on a number of relevant items of supporting evidence retrieved by the deep question answering system using the first feature.

8. The computer program product of claim 1, wherein the first case comprises at least one question presented to the deep question answering system, wherein the first feature comprises at least one of:

(i) a type, (ii) a subject matter, (iii) a variable, and (iv) a context of the at least one question.

9. A system, comprising:

one or more computer processors; and

a memory containing a program, which, when executed by the one or more computer processors, performs an operation to identify relevant features in a deep question answering system, the operation comprising;

classifying a first case received by the deep question answering system; and

while training the deep question answering system to answer the first case;

identifying a first feature in the first case;

computing a first feature score for the first feature by operation of one or more computer processors, wherein the first feature score indicates a relevance of the first feature in generating a correct response to the first case; and

identifying the first feature as relevant in answering the classified first case upon determining that the first feature score exceeds a relevance threshold.

10. The system of claim 9, further comprising:

responsive to receiving a second case by the deep question answering system;

classifying the second case;

computing a similarity score for the first case and the second case; and

identifying the first feature as relevant in generating a correct response to the second case upon determining that the similarity score exceeds a specified similarity threshold.

11. The system of claim 10, wherein the similarity score is based on at least one of:

(i) the classification of the first case and the classification second case, and (ii) a context of the first case and a context of the second case.

12. The system of claim 10, wherein the first case and the second case are classified based on at least one of:

(i) an attribute of a user presenting the respective case to the deep question answering system, (ii) a content of a question in the respective case, and (iii) a type of the question in the respective case.

13. The system of claim 12, wherein the attribute of the user is selected from the group comprising:

(i) a role of the user, (ii) an affiliation of the user, (iii) an expertise of the user, and (iv) a set of predefined preferences of the user.

14. The system of claim 9, further comprising:

storing a relationship between the first feature and the first case, wherein the relationship indicates that the first feature is relevant in answering the first case.

15. The system of claim 9, wherein the first feature score is based on a number of relevant items of supporting evidence retrieved by the deep question answering system using the first feature.

16. The system of claim 9, wherein the first case comprises at least one question presented to the deep question answering system, wherein the first feature comprises at least one of:

(i) a type, (ii) a subject matter, (iii) a variable, and (iv) a context of the at least one question.