Method for a natural language question-answering system to complement decision-support in a real-time command center

• US 8,601,030 B2
• Filed: 09/09/2011
• Issued: 12/03/2013
• Est. Priority Date: 09/09/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bagchi, Sugato, Wynter, Laura
• Primary Examiner(s): Chbouki, Tarek
• Application Number:
  US13/228,709
• Publication Number:
  US 20130066886A1
• Time in Patent Office: 816 Days
• Field of Search: None
• US Class Current: 707/802
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems

Abstract

System and methods provide decision support by receiving suggested actions from a decision support system and automatically determining parameters of the suggested actions to serve as input to a question-answering system. System and methods also automatically generate questions based on the parameters, automatically search a corpus of unstructured data to retrieve answers to the questions, and automatically provide impact confidence values for each answer indicating the degree of impact the answers have on the suggested actions. The systems and methods can then output the questions, answers, and impact confidence values.

Claims

1. A method for providing decision support comprising:

receiving an action plan comprising suggested actions from a decision support system into a particular computerized machine;

automatically determining parameters of said suggested actions using said particular computerized machine to serve as input to a question-answering system operating on said particular computerized machine;

automatically generating questions based on said parameters using said question-answering system;

automatically searching a corpus of unstructured data to retrieve answers to said questions using said question-answering system;

automatically providing impact confidence values for each answer indicating degree of impact said answers have on said suggested actions using said question-answering system; and

if said answers have an impact confidence value and a relevance confidence value that are above predetermined confidence thresholds, causing said decision support system to reproduce said action plan with modified suggested actions that are different than said suggested actions using said questions, said answers, and said impact confidence values.

2. The method according to claim 1, further comprising ranking said answers according to said impact confidence values using said question-answering system and outputting rankings.

3. The method according to claim 1, said providing of said impact confidence values comprising calculating an impact correctness of each answer according to various dimensions of evidence sources using answer scoring modules within said question-answering system.

4. The method according to claim 1, further comprising automatically providing relevance confidence values for each answer indicating degree of relevance said answers have to said suggested actions using said question-answering system;

and outputting said relevance confidence values.

5. The method according to claim 1, said answers comprising events.

6. The method according to claim 1, said decision support system using a database of structured data to generate said suggested actions, and said question-answering system using said unstructured data from a variety of unrelated sources separate from said database of structured data to retrieve said answers.

7. The method according to claim 1, said answers being limited to ones that have an impact confidence value and a relevance confidence value that are above predetermined confidence thresholds.

8. The method according to claim 1, further comprising updating a learning model of said question-answering system based on input received in response to said questions, said answers, and said impact confidence values.

9. A method for providing decision support comprising:

receiving suggested actions from a decision support system into a particular computerized machine;

automatically determining parameters of said suggested actions using said particular computerized machine to serve as input to a question-answering system operating on said particular computerized machine;

automatically generating questions based on said parameters using said question-answering system;

automatically searching a corpus of unstructured data to retrieve answers to said questions using said question-answering system;

automatically providing impact confidence values for each answer indicating degree of impact said answers have on said suggested actions using said question-answering system;

automatically providing relevance confidence values for each answer indicating degree of relevance said answers have to said suggested actions using said question-answering system;

if said answers have an impact confidence value and a relevance confidence value that are above predetermined confidence thresholds, causing said decision support system to reproduce an action plan with modified suggested actions that are different than said suggested actions using said questions, said answers, and said impact confidence values instead of performing outputting of said questions, said answers, said relevance confidence values, and said impact confidence values; and

if said answers have an impact confidence value and a relevance confidence value that are below said predetermined confidence thresholds, performing said outputting of said questions, said answers, said relevance confidence values, and said impact confidence values on a graphic user interface without causing said decision support system to reproduce said action plan.

10. The method according to claim 9, further comprising ranking said answers according to said impact confidence values and said relevance confidence values using said question-answering system and outputting rankings using said graphic user interface.

11. The method according to claim 9, said providing of said impact confidence values comprising calculating an impact correctness of each answer according to various dimensions of evidence sources using answer scoring modules within said question-answering system.

12. The method according to claim 9, said providing of said relevance confidence values comprising calculating a relevance correctness of each answer according to various dimensions of evidence sources using answer scoring modules within said question-answering system.

13. The method according to claim 9, said answers comprising events.

14. The method according to claim 9, said decision support system using a database of structured data to generate said suggested actions, and said question-answering system using said unstructured data from a variety of unrelated sources separate from said database of structured data to retrieve said answers.

15. The method according to claim 9, said outputting being limited to said answers that have an impact confidence value and a relevance confidence value that are above predetermined confidence thresholds.

16. The method according to claim 9, further comprising updating a learning model of said question-answering system based on input received in response to said outputting of said questions, said answers, said relevance confidence values, and said impact confidence values.

17. A computerized machine for providing decision support for a command center comprising:

an input/output port receiving suggested actions from a decision support system;

a processor operatively connected to said input/output port, said processor;

automatically determining parameters of said suggested actions to serve as input to a question-answering system;

automatically generating questions based on said parameters;

automatically searching a corpus of unstructured data to retrieve answers to said questions; and

automatically providing impact confidence values for each answer indicating degree of impact said answers have on said suggested actions; and

a graphic user interface operatively connected to said processor, said graphic user interface outputting said questions, said answers, and said impact confidence values,

if said answers have an impact confidence value and a relevance confidence value that are above predetermined confidence thresholds, said processor causing said decision support system to reproduce an action plan with modified suggested actions that are different than said suggested actions using said questions, said answers, and impact confidence values instead of said graphic user interface outputting said questions, said answers, relevance confidence values, and said impact confidence values; and

if said answers have an impact confidence value and a relevance confidence value that are below said predetermined confidence thresholds, said graphic user interface outputting said questions, said answers, said relevance confidence values, and said impact confidence values without causing said decision support system to reproduce said action plan.

18. The system according to claim 17, said question-answering system ranking said answers according to said impact confidence values, and said graphic user interface outputting rankings.

19. The system according to claim 17, said question-answering system providing said impact confidence values by calculating an impact correctness of each answer according to various dimensions of evidence sources using answer scoring modules within said question-answering system.

20. The system according to claim 17, said question-answering system automatically providing relevance confidence values for each answer indicating degree of relevance said answers have to said suggested actions;

and said graphic user interface outputting said relevance confidence values.

21. The system according to claim 17, said answers comprising events.

22. The system according to claim 17, said decision support system using a database of structured data to generate said suggested actions, and said question-answering system using said unstructured data from a variety of unrelated sources separate from said database of structured data to retrieve said answers.

23. The system according to claim 17, said processor limiting said outputting of said answers to said answers that have a degree of impact confidence value and a relevance confidence value that are above predetermined confidence thresholds.

24. The system according to claim 17, further comprising a learning model of said question-answering system, said learning model being updated based on input received in response to said outputting of said questions, said answers, and said impact confidence values.

25. A computer readable storage medium readable by a computerized device, said computer readable storage medium storing instructions executable by said computerized device to perform a method for providing decision support for a command center, said method comprising:

receiving suggested actions from a decision support system;

automatically determining parameters of said suggested actions to serve as input to a question-answering system;

automatically generating questions based on said parameters;

automatically searching a corpus of unstructured data to retrieve answers to said questions;

automatically providing impact confidence values for each answer indicating degree of impact said answers have on said suggested actions;

if said answers have an impact confidence value and a relevance confidence value that are above predetermined confidence thresholds, causing said decision support system to reproduce an action plan with modified suggested actions that are different than said suggested actions using said questions, said answers, and impact confidence values instead of performing outputting of said questions, said answers, relevance confidence values, and said impact confidence values; and

if said answers have an impact confidence value and a relevance confidence value that are below said predetermined confidence thresholds, performing said outputting of said questions, said answers, said relevance confidence values, and said impact confidence values on a graphic user interface without causing said decision support system to reproduce said action plan.