Decision-support application and system for problem solving using a question-answering system

• US 9,002,773 B2
• Filed: 03/31/2011
• Issued: 04/07/2015
• Est. Priority Date: 09/24/2010
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bagchi, Sugato, Ferrucci, David A., Levas, Anthony T., Mueller, Erik T
• Primary Examiner(s): Wong, Lut
• Application Number:
  US13/077,464
• Publication Number:
  US 20120078837A1
• Time in Patent Office: 1,468 Days
• Field of Search: 706/52
• US Class Current: 706/52
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 5/00 Measuring for diagnostic purposes radiation diagnosis A61B6/00; diagnosis by ultrasonic, sonic or infrasonic waves A61B8/00; Identification of persons
• CPC Class Codes: G06F 16/2428 Query predicate definition using graphical user interfaces, including menus and forms G06F16/2423 takes precedence
• CPC Class Codes: G06F 16/24522 Translation of natural language queries to structured queries
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/31 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06N 5/027 Frames
• CPC Class Codes: G16H 10/20 for electronic clinical trials or questionnaires
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 15/00 ICT specially adapted for medical reports, e.g. generation or transmission thereof
• CPC Class Codes: G16H 40/20 for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients
• CPC Class Codes: G16H 70/00 : ICT specially adapted for the handling or processing of medical references
• CPC Class Codes: Y02A 90/10 : Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

A decision-support system for problem solving comprises software modules embodied on a computer readable medium, and the software modules comprise an input/output module and a question-answering module. The method receives problem case information using the input/output module, generates a query based on the problem case information, and generates a plurality of answers for the query using the question-answering module. The method also calculates numerical values for multiple evidence dimensions from evidence sources for each of the answers using the question-answering module and calculates a corresponding confidence value for each of the answers based on the numerical value of each evidence dimension using the question-answering module. Further, the method outputs the answers, the corresponding confidence values, and the numerical values of each evidence dimension for one or more selected answers using the input/output module.

Claims

1. A method comprising:

receiving, by a decision-support system for problem solving, an initial query and problem case information regarding a problem in a specific domain, said decision-support system comprising a computerized device that has access to evidence sources containing domain knowledge content comprising knowledge in said specific domain, said computerized device further being in communication with a repository and being programmed with instructions, said instructions causing said computerized device to perform processing comprising;

automatically extracting relevant information from said problem case information and using said relevant information to expand said initial query so as to generate a specific natural language query;

generating answers to said natural language query by applying natural language processing techniques to said domain knowledge content within said evidence sources;

calculating numerical values of evidence dimensions for each of said answers;

calculating corresponding confidence values for each of said answers based on said numerical values of each of said evidence dimensions;

outputting said natural language query, said answers, said corresponding confidence values, and said numerical values of each of said evidence dimensions for one or more selected answers to a decision-maker and to said repository, said natural language query, said answers, said corresponding confidence values and said numerical values of each of said evidence dimensions being displayed for said decision-maker on a display and usable by said decision-maker to make a decision regarding said problem;

storing, in said repository, said natural language query, said answers, said corresponding confidence values, and said numerical values of each of said evidence dimensions;

in response to new domain knowledge being added to said domain knowledge content in said evidence sources, automatically updating said answers to said natural language query, said numerical values of said evidence dimensions for each of said answers and said corresponding confidence values and determining whether previously generated answers and corresponding confidence values stored in said repository have changed based on said new domain knowledge; and

,

automatically sending an alert to said decision-maker based on a change occurring in any of said previously generated answers and corresponding confidence values, said alert being usable by said decision-maker to re-evaluate said decision.

2. The method according to claim 1, said processing further comprising:

automatically analyzing said problem case information in order to identify semantic concepts; and

automatically generating said natural language query from said semantic concepts.

3. The method according to claim 1, said automatically updating further being performed in response to new problem case information being received.

4. The method according to claim 3, said processing further comprising storing, in said repository, revised answers, numerical values and corresponding confidence values generated automatically in response to any of said new domain knowledge being added to said domain knowledge content and said new problem case information being received.

5. The method according to claim 1, said initial query being in a form of at least one of a free-form query, a free-form statement, and keyword search.

6. The method according to claim 5, said processing further comprising:

automatically analyzing said problem case information in order to identify semantic concepts; and

expanding said initial query using said semantic concepts.

7. The method according to claim 1, said processing further comprising outputting, for a selected evidence dimension, each piece of evidence supporting said selected evidence dimension and an associated provenance for said piece of evidence.

8. The method according to claim 7, said processing further comprising:

removing a selected piece of evidence from said evidence sources contributing to a specific numerical value of a specific evidence dimension of a specific answer;

recalculating said specific numerical value of said specific evidence dimension having said selected piece of evidence removed to produce a recalculated numerical value; and

recalculating a new confidence value of said specific answer based on said recalculated numerical value.

9. The method according to claim 1, said processing further comprising:

identifying information relevant to said answers that is not contained within said problem case information as missing information; and

outputting a request to add said missing information to said problem case information.

10. The method according to claim 9, said processing further comprising identifying an amount said missing information affects any of said corresponding confidence values of any of said answers.

11. The method according to claim 1, said alert being sent only when said change exceeds a difference threshold, said difference threshold being any of a percentage change in an answer, an answer polarity change, a percentage change in a confidence level and a confidence level polarity change.

12. A system comprising:

a repository maintaining problem case information regarding a problem in a specific domain; and

a computer processor operatively connected to said repository and having access to evidence sources containing domain knowledge content comprising knowledge in said specific domain,

said computer processor receiving an initial query and said problem case information from said repository,

said computer processor automatically extracting relevant information from said problem case information and using said relevant information to expand said initial query so as to generate a specific natural language query,

said computer processor generating answers to said natural language query by applying natural language processing techniques to said domain knowledge content within said evidence sources,

said computer processor calculating numerical values for multiple of evidence dimensions for each of said answers,

said computer processor calculating corresponding confidence values for each of said answers based on said numerical values of each of said evidence dimensions,

said computer processor outputting said natural language query, said answers, said corresponding confidence values, and said numerical values of each of said evidence dimensions to a decision-maker and to said repository,

said natural language query, said answers, said corresponding confidence values and said numerical values of each of said evidence dimensions being displayed for said decision-maker on a display and usable by said decision-maker to make a decision regarding said problem,

said repository storing said natural language query, said answers, said corresponding confidence values, and said numerical values of each of said evidence dimensions,

said computer processor, in response to new domain knowledge being added to said domain knowledge content in said evidence sources, automatically updating said answers to said natural language query, said numerical values of said evidence dimensions for each of said answers and said corresponding confidence values and determining whether previously generated answers and corresponding confidence values stored in said repository changed based on said new domain knowledge, and

said computer processor automatically sending an alert to said decision-maker based on a change occurring in any of said previously generated answers and corresponding confidence values, said alert being usable by said decision-maker to re-evaluate said decision.

13. The system according to claim 12, said natural language query being generated by analyzing said problem case information in order to identify semantic concepts and generating said natural language query from said semantic concepts.

14. The system according to claim 12, said computer processor further performing said automatically updating in response to new problem case information being received.

15. The system according to claim 12, said initial query being received as an input in a form of at least one of a free-form query, a free-form statement, and keyword search.

16. The system according to claim 12, said computer processor further outputting a piece of evidence supporting a selected evidence dimension and an associated provenance for said piece of evidence.

17. The system according to claim 16, said computer processor further removing a selected piece of evidence from said evidence sources contributing to a specific numerical value of a specific evidence dimension of a specific answer, recalculating said specific numerical value of said specific evidence dimension having said selected piece of evidence removed to produce a recalculated numerical value, and recalculating a new confidence value of said specific answer based on said recalculated numerical value.

18. The system according to claim 12, said computer processor further identifying information relevant to said answers that is not contained within said problem case information as missing information and outputting a request to add said missing information to said problem case information.

19. The system according to claim 18, said computer processor further identifying an amount said missing information affects any of said corresponding confidence values of any of said answers.

20. The system according to claim 12, said alert being sent only when said change exceeds a difference threshold, said difference threshold being any of a percentage change in an answer, an answer polarity change, a percentage change in a confidence level and a confidence level polarity change.

21. A computer program product comprising a computer readable storage medium storing computer readable program code comprising instructions executable by a computerized device to perform a method, said method comprising:

receiving an initial query and problem case information regarding a problem in a specific domain;

automatically extracting relevant information from said problem case information and using said relevant information to expand said initial query so as to generate a specific natural language query;

generating answers to said natural language query by applying natural language processing techniques to domain knowledge content within evidence sources, said domain knowledge content comprising knowledge in said specific domain;

calculating numerical values of evidence dimensions for each of said answers;

calculating corresponding confidence values for each of said answers based on said numerical values of each of said evidence dimensions;

outputting said natural language query, said answers, said corresponding confidence values, and said numerical values of each of said evidence dimensions for one or more selected answers to a decision-maker and to a repository, said natural language query, said answers, said corresponding confidence values and said numerical values of each of said evidence dimensions being displayed for said decision-maker on a display and usable by said decision-maker to make a decision regarding said problem;

storing, in said repository, said natural language query, said answers, said corresponding confidence values, and said numerical values of each of said evidence dimensions;

in response to new domain knowledge being added to said domain knowledge content in said evidence sources, automatically updating said answers to said natural language query, said numerical values of said evidence dimensions for each of said answers and said corresponding confidence values and determining whether previously generated answers and corresponding confidence values stored in said repository changed based on said new domain knowledge; and

,

automatically sending an alert to said decision-maker based on a change occurring in any of said previously generated answers and corresponding confidence values, said alert being usable by said decision-maker to re-evaluate said decision.