Monitoring a corpus for changes to previously provided answers to questions

• US 10,152,534 B2
• Filed: 07/02/2015
• Issued: 12/11/2018
• Est. Priority Date: 07/02/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bastide, Paul R., Cohen, Aaron M.
• Primary Examiner(s): Hoang, Son T
• Application Number:
  US14/790,121
• Publication Number:
  US 20170004204A1
• Time in Patent Office: 1,258 Days
• Field of Search: None
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems

Abstract

Mechanisms are provided for identifying changes, within a corpus of information, to answers to questions provided within the corpus of information. A mapping of a question and a previously provided answer to the question in a question/answer data structure is generated. A transaction referencing the previously provided answer, and having an update to the previously provided answer, is received and correlated with the question/answer data structure, which is updated based on the transaction. A determination is made as to whether the strength of the mapping of the first question to the previously provided answer based on the update to the question/answer data structure has fallen equal to or below a predetermined threshold. In response to this being the case, a change to the previously provided answer is propagated throughout the corpus of information.

Claims

1. A method, in a data processing system comprising a processor and a memory, for identifying changes, within a corpus of information, to answers to questions provided within the corpus of information, the method comprising:

configuring the data processing system to execute a corpus information source monitoring engine comprising question/answer mapping logic, answer change logic, and previous transaction update logic;

executing the corpus information source monitoring engine in the data processing system;

registering, by the question/answer mapping logic of the executing corpus information source monitoring engine, a mapping of a first question and a previously provided answer to the first question in a question/answer data structure;

receiving, by the answer change logic of the executing corpus information source monitoring engine, a transaction referencing the previously provided answer and having an update to the previously provided answer, wherein the transaction is a user input specifically directed to supporting or refuting the previously provided answer being a correct answer for the first question;

correlating, by the question/answer mapping logic, the transaction with the question answer data structure and updating the question/answer data structure at least by updating a measure of strength of the mapping of the first question to the previously provided answer based on the transaction;

determining, by the logic of the corpus information source monitoring engine, whether the strength of the mapping of the first question to the previously provided answer has fallen equal to or below a predetermined threshold, at least based on a calculation of the strength of the mapping as a weighted numerical calculation comprising quantified factors corresponding to temporal characteristics of transactions referencing the previously provided answer; and

propagating, by the previous transaction update logic of the executing corpus information source monitoring engine, a change to the previously provided answer throughout the corpus of information in response to the strength of the mapping of the first question falling equal to or below the predetermined threshold, wherein the receiving the transaction referencing the previously provided answer comprises receiving a transaction referencing a second question that has similar features to that of the first question, and wherein the change to the previously provided answer is a different answer from that of the previously provided answer being provided as an answer to the second question.

2. The method of claim 1, wherein the first question and the previously provided answer to the first question are part of the corpus of information itself, and wherein the change to the previously provided answer is an update that is added to the corpus of information and is part of the corpus of information.

3. The method of claim 1, wherein more recent transactions are weighted more highly than older transactions in the quantified factors of the weighted numerical calculation.

4. The method of claim 1, wherein the propagating the change to the previously provided answer throughout the corpus of information comprises:

maintaining, for each transaction referencing the previously provided answer, a tracking record in a tracking data structure to track utilization of the previously provided answer;

traversing the tracking data structure to identify transactions that utilized the previously provided answer; and

propagating the change based on the identified transactions in the tracking data structure.

5. The method of claim 1, wherein the propagating the change to the previously provided answer throughout the corpus of information further comprises sending a notification to one or more computing devices associated with users that viewed the previously provided answer, wherein the notification informs the users of the change to the previously provided answer.

6. The method of claim 1, wherein the receiving the transaction referencing the previously provided answer and having the update to the previously provided answer comprises receiving a transaction report message, from an agent deployed in a computing device of an information source, which identifies the transaction as referencing the previously provided answer, the previously provided answer being associated with a previous transaction identifier.

7. The method of claim 1, wherein the determining whether the strength of the mapping of the first question to the previously provided answer, based on the update to the question/answer data structure, has fallen equal to or below the predetermined threshold comprises calculating the strength of the mapping of the first question to the previously provided answer as the weighted numerical calculation which further comprises quantified factors corresponding to one or more of:

numerical counts of positive/negative votes associated with the previously provided answer,

analysis of natural language comments for indications of positive negative reinforcement of the previously provided answer, or

differences or similarities in one or more answers for one or more similar other questions in different portions of the corpus of information.

8. The method of claim 1, wherein the data processing system implements a question and answer (QA) system, and wherein the mapping of the question to the previously provided answer is generated by the QA system.

9. The method of claim 1, wherein the change to the previously provided answer is propagated throughout the corpus of information without re-processing the question to generate the change in the previously provided answer.

10. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, configures the computing device to execute a corpus information source monitoring engine comprising question/answer mapping logic, answer change logic, and previous transaction update logic, and causes the computing device to:

execute the corpus information source monitoring engine in the data processing system;

register, by the question/answer mapping logic of the executing corpus information source monitoring engine, a mapping of a first question and a previously provided answer to the first question in a question/answer data structure;

receive, by the answer change logic of the executing corpus information source monitoring engine, a transaction referencing the previously provided answer and having an update to the previously provided answer, wherein the transaction is a user input specifically directed to supporting or refuting the previously provided answer being a correct answer for the first question;

correlate, by the question/answer mapping logic, the transaction with the question answer data structure and updating the question/answer data structure at least by updating a measure of strength of the mapping of the first question to the previously provided answer based on the transaction;

determine, by the logic of the corpus information source monitoring engine, whether the strength of the mapping of the first question to the previously provided answer has fallen equal to or below a predetermined threshold, at least based on a calculation of the strength of the mapping as a weighted numerical calculation comprising quantified factors corresponding to temporal characteristics of transactions referencing the previously provided answer; and

propagate, by the previous transaction update logic of the executing corpus information source monitoring engine, a change to the previously provided answer throughout the corpus of information in response to the strength of the mapping of the first question falling equal to or below the predetermined threshold, wherein the computer readable program further causes the computing device to receive the transaction referencing the previously provided answer at least by receiving a transaction referencing a second question that has similar features to that of the first question, and wherein the change to the previously provided answer is a different answer from that of the previously provided answer being provided as an answer to the second question.

11. The computer program product of claim 10, wherein the first question and the previously provided answer to the first question are part of the corpus of information itself, and wherein the change to the previously provided answer is an update that is added to the corpus of information and is part of the corpus of information.

12. The computer program product of claim 10, wherein more recent transactions are weighted more highly than older transactions in the quantified factors of the weighted numerical calculation.

13. The computer program product of claim 10, wherein the computer readable program further causes the computing device to propagate the change to the previously provided answer throughout the corpus of information at least by:

maintaining, for each transaction referencing the previously provided answer, a tracking record in a tracking data structure to track utilization of the previously provided answer;

traversing the tracking data structure to identity transactions that utilized the previously provided answer; and

propagating the change based on the identified transactions in the tracking data structure.

14. The computer program product of claim 10, wherein the computer readable program further causes the computing device to propagate the change to the previously provided answer throughout the corpus of information further at least by sending a notification to one or more computing devices associated with users that viewed the previously provided answer, wherein the notification informs the users of the change to the previously provided answer.

15. The computer program product of claim 10, wherein the computer readable program further causes the computing device to receive the transaction referencing the previously provided answer and having the update to the previously provided answer at least by receiving a transaction report message, from an agent deployed in a computing device of an information source, which identifies the transaction as referencing the previously provided answer, the previously provided answer being associated with a previous transaction identifier.

16. The computer program product of claim 10, wherein the computer readable program further causes the computing device to determine whether the strength of the mapping of the first question to the previously provided answer based on the update to the question/answer data structure has fallen equal to or below the predetermined threshold at least by calculating the strength of the mapping of the first question to the previously provided answer as the weighted numerical calculation which further comprises quantified factors corresponding to one or more of:

numerical counts of positive/negative votes associated with the previously provided answer,

analysis of natural language comments for indications of positive/negative reinforcement of the previously provided answer, or

differences or similarities in one or more answers for one or more similar other questions in different portions of the corpus of information.

17. An apparatus comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, configures the processor to execute a corpus information source monitoring engine comprising question/answer mapping logic, answer change logic, and previous transaction update logic, and causes the processor to;

execute the corpus information source monitoring engine in the data processing system;

register, by the question answer mapping logic of the executing corpus information source monitoring engine, a mapping of a first question and a previously provided answer to the first question in a question/answer data structure;

receive, by the answer change logic of the executing corpus information source monitoring engine, a transaction referencing the previously provided answer and having an update to the previously provided answer, wherein the transaction is a user input specifically directed to supporting or refuting the previously provided answer being a correct answer for the first question;

correlate, by the question/answer mapping logic, the transaction with the question/answer data structure and updating the question/answer data structure at least by updating a measure of strength of the mapping of the first question to the previously provided answer based on the transaction;

determine, by the logic of the corpus information source monitoring engine, whether the strength of the mapping of the first question to the previously provided answer, has fallen equal to or below a predetermined threshold, at least based on a calculation of the strength of the mapping as a weighted numerical calculation comprising quantified factors corresponding to temporal characteristics of transactions referencing the previously provided answer; and

propagate, by the previous transaction update logic of the executing corpus information source monitoring engine, a change to the previously provided answer throughout the corpus of information in response to the strength of the mapping of the first question falling equal to or below the predetermined threshold, wherein the instructions further cause the processor to receive the transaction referencing the previously provided answer at least by receiving a transaction referencing a second question that has similar features to that of the first question, and wherein the change to the previously provided answer is a different answer from that of the previously provided answer being provided as an answer to the second question.