Caching natural language questions and results in a question and answer system

• US 9,311,823 B2
• Filed: 06/20/2013
• Issued: 04/12/2016
• Est. Priority Date: 06/20/2013
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Byron, Donna K., Krishnamurthy, Lakshminarayanan, Seabolt, Edward E.
• Primary Examiner(s): Hill, Stanley K
• Assistant Examiner(s): Smith, Paulinho E
• Application Number:
  US13/923,029
• Publication Number:
  US 20140377735A1
• Time in Patent Office: 1,027 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

Mechanisms are provided for answering an input question. An input question to be answered from a source is received and processed to one or more extract features of the input question. The extracted one or more features are compared to cached features stored in one or more entries of a question and answer (QA) cache. A determination is made as to whether there is a matching entry in the one or more entries of the QA cache based on results of the comparing and, if so, candidate answer information is retrieved from the matching entry. The retrieved candidate answer information is returned to the source of the input question as candidate answer information for answering the input question.

Claims

1. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

receive an input question to be answered from a source;

process the input question to extract one or more features of the input question;

compare the extracted one or more features to cached features stored in one or more entries of a question and answer (QA) cache of the data processing system;

determine whether there is a matching entry in the one or more entries of the QA cache, wherein the computer readable program to determine whether there is a matching entry in the one or more entries of the QA cache further causes the computing device to;

generate, for each entry in the QA cache, a match value indicative of a degree of matching between the one or more extracted features of the input question to cached features of the entry in the QA cache; and

compare the match value to one or more threshold values indicating one or more requisite degrees of similarity between the input question and an entry in the QA cache, wherein;

in response to the match value equaling or exceeding a first threshold value the computer readable program further causes the computing device to determine a corresponding entry to match the input question, and

in response to the match value being less than the first threshold value but the match value being equal to or greater than a second threshold value, the computer readable program further causes the computing device to determine that the corresponding entry is sufficiently similar for updating the corresponding entry with the one or more extracted features of the input question;

retrieve, in response to a matching entry being present in the one or more entries of the QA cache, candidate answer information from the matching entry; and

return the retrieved candidate answer information to the source of the input question as candidate answer information for answering the input question.

2. The computer program product of claim 1, wherein, in response to determining that there is a matching entry in the one or more entries of the QA cache, processing of the input question by a predetermined set of QA system pipeline stages is circumvented.

3. The computer program product of claim 1, wherein the computer readable program further causes the computing device to:

process, in response to determining that there is not a matching entry in the one or more entries of the QA cache, the input question through a QA system pipeline to generate one or more generated candidate answers for answering the input question.

4. The computer program product of claim 3, wherein the computer readable program further causes the computing device to:

generate, in response to determining that there is not a matching entry in the one or more entries of the QA cache, a new cache entry in the QA cache for the input question, wherein the new entry in the QA cache comprises the extracted one or more features of the input question stored in association with the one or more generated candidate answers for the input question.

5. The computer program product of claim 3, wherein the computer readable program further causes the computing device to:

update, in response to determining that there is not a matching entry in the one or more entries of the QA cache and in response to determining that a selected entry in the QA cache has a predetermined degree of similarity to the input question, the selected entry with the extracted one or more features of the input question in association with the one or more generated candidate answers for the input question.

6. The computer program product of claim 1, wherein, in response to determining that there is not a matching entry in the one or more entries of the QA cache, the computer readable program further causes the computing device to:

determine if there is a subset of entries in the one or more entries that is similar to the input question; and

output a listing of the subset of entries to a user for user selection of an entry in the subset of entries to be retrieved and used to generate candidate answers for the input question.

7. The computer program product of claim 1, wherein entries in the QA cache comprise extracted features for previously processed questions with corresponding candidate answer information and confidence measures associated with the candidate answers.

8. The computer program product of claim 7, wherein the entries in the QA cache further comprise at least one of ranking information for the candidate answer, evidence information indicating the evidence used to generate the candidate answer, a determined domain of a corresponding previously processed question, information about a submitter of the corresponding previously processed question, or time of receipt of the corresponding previously processed question.

9. An apparatus comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to;

receive an input question to be answered from a source;

process the input question to extract one or more features of the input question;

compare the extracted one or more features to cached features stored in one or more entries of a question and answer (QA) cache of the data processing system;

determine whether there is a matching entry in the one or more entries of the QA cache, wherein the instructions to determine whether there is a matching entry in the one or more entries of the QA cache further cause the processor to;

generate, for each entry in the QA cache, a match value indicative of a degree of matching between the one or more extracted features of the input question to cached features of the entry in the QA cache; and

compare the match value to one or more threshold values indicating one or more requisite degrees of similarity between the input question and an entry in the QA cache, wherein;

in response to the match value equaling or exceeding a first threshold value the instructions further cause the processor to determine a corresponding entry to match the input question, and

in response to the match value being less than the first threshold value but the match value being equal to or greater than a second threshold value, the instructions further cause the processor to determine that the corresponding entry is sufficiently similar for updating the corresponding entry with the one or more extracted features of the input question;

retrieve, in response to a matching entry being present in the one or more entries of the QA cache, candidate answer information from the matching entry; and

return the retrieved candidate answer information to the source of the input question as candidate answer information for answering the input question.

10. The apparatus of claim 9, wherein, in response to determining that there is a matching entry in the one or more entries of the QA cache, processing of the input question by a predetermined set of QA system pipeline stages is circumvented.

11. The apparatus of claim 9, wherein the instructions further cause the processor to:

process, in response to determining that there is not a matching entry in the one or more entries of the QA cache, the input question through a QA system pipeline to generate one or more generated candidate answers for answering the input question.

12. The apparatus of claim 11, wherein the instructions further cause the processor to:

generate, in response to determining that there is not a matching entry in the one or more entries of the QA cache, a new cache entry in the QA cache for the input question, wherein the new entry in the QA cache comprises the extracted one or more features of the input question stored in association with the one or more generated candidate answers for the input question.

13. The apparatus of claim 11, wherein the instructions further cause the processor to:

update, in response to determining that there is not a matching entry in the one or more entries of the QA cache and in response to determining that a selected entry in the QA cache has a predetermined degree of similarity to the input question, the selected entry with the extracted one or more features of the input question in association with the one or more generated candidate answers for the input question.

14. The apparatus of claim 9, wherein, in response to determining that there is not a matching entry in the one or more entries of the QA cache, the instructions further cause the processor to:

determine if there is a subset of entries in the one or more entries that is similar to the input question; and

output a listing of the subset of entries to a user for user selection of an entry in the subset of entries to be retrieved and used to generate candidate answers for the input question.

15. The apparatus of claim 9, wherein entries in the QA cache comprise extracted features for previously processed questions with corresponding candidate answer information and confidence measures associated with the candidate answers.

16. The apparatus of claim 15, wherein the entries in the QA cache further comprise at least one of ranking information for the candidate answer, evidence information indicating the evidence used to generate the candidate answer, a determined domain of a corresponding previously processed question, information about a submitter of the corresponding previously processed question, or time of receipt of the corresponding previously processed question.