User interface for an evidence-based, hypothesis-generating decision support system

• US 9,153,142 B2
• Filed: 04/17/2012
• Issued: 10/06/2015
• Est. Priority Date: 05/26/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bagchi, Sugato, Barborak, Michael A., Daniels, Steven D., Ferrucci, David A., Levas, Anthony T.
• Primary Examiner(s): Yao, Sam
• Assistant Examiner(s): Saint-Vil, Eddy
• Application Number:
  US13/448,607
• Publication Number:
  US 20120301864A1
• Time in Patent Office: 1,267 Days
• Field of Search: 707/3, 707/5, 707/6, 707/7, 707/713, 707/715, 705/2, 705/3, 600/300, 434/322, 434/362, 706/14, 706/21, 706/50
• US Class Current: 1/1
• CPC Class Codes: A61B 5/00 Measuring for diagnostic purposes radiation diagnosis A61B6/00; diagnosis by ultrasonic, sonic or infrasonic waves A61B8/00; Identification of persons
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

Systems and methods display at least one subject, and display a location for at least one user to enter at least one problem related to the subject. The problem comprises unknown items to which the user would like more information. In response to the problem, such systems and methods automatically generate evidence topics related to the problem, and automatically generate questions related to the problem and the evidence topics. Further, such systems and methods can receive additional questions from the user. In response to the questions, such systems and methods automatically generate answers to the questions by referring to sources, automatically calculate confidence measures of each of the answers, and then display the questions, the answers, and the confidence measures. When the user identifies one of the answers as a selected answer, such systems and methods display details of the sources and the factors used to generate the selected answer.

Claims

1. A method comprising:

displaying, on a user interface, at least one subject;

displaying, on said user interface, a location for at least one user to enter at least one problem related to said subject;

in response to entry of said at least one problem through said user interface, automatically acquiring factors related to said at least one problem from a record associated with said subject, said factors being categorized into dimensions of evidence, said record being stored on at least one computerized storage medium, and said automatically acquiring being performed using at least one computerized device operatively connected to said user interface and said at least one computerized storage medium;

automatically generating questions related to said at least one problem and said factors using said at least one computerized device;

in response to said questions, automatically generating answers to said questions by referring to sources, said sources being accessible by said at least one computerized device—

and said generating of said answers being performed using said at least one computerized device;

for each dimension of evidence, automatically calculating confidence measures of each of said answers using said at least one computerized device;

combining said confidence measures of each of said answers for each dimension of evidence in order to calculate final confidence measures of each of said answers, said combining being performed using said at least one computerized device;

generating a comparative evidence profile using said at least one computerized device and displaying said answers and said final confidence measures for each of said answers on said user interface with said comparative evidence profile, said comparative evidence profile comprising a chart that provides visual indications on said user interface of relative contributions of each of said dimensions of evidence to each of said final confidence measures of each of said answers and that comprises selectable features;

displaying, on said user interface with said answers, said final confidence measures and said comparative evidence profile, details of said sources and said factors used to generate said answers on said user interface; and

,

upon receiving a selection of one of said selectable features of said chart through said user interface, automatically filtering said details of said sources and said factors so as to limit display of said details, said automatically filtering being performed by said at least one computerized device.

2. The method according to claim 1, further comprising maintaining a history of said questions, said answers, and said final confidence measures of each of said answers.

3. The method according to claim 1, said sources comprising first sources stored on said at least one computerized storage medium and second sources accessible online and said details of said sources comprising annotations to said sources.

4. The method according to claim 1, further comprising displaying how said sources contributed to said confidence measures of said answers on said user interface.

5. The method according to claim 1, said at least one problem comprising at least one question.

6. The method according to claim 1,

said chart comprising a bar chart comprising groups of bars,

each group of bars being associated with a specific dimension of evidence and being selectable to filter said details of said sources and said factors so as to limit displayed details of said sources and said factors to only those associated with said specific dimension of evidence, and,

each bar within said group being associated with a specific answer and being selectable to further filter said details of said source and said factors so as to further limit said displayed details of said sources and said factors to only those associated with both said specific answer and said specific dimension of evidence.

7. A method comprising:

displaying, on a user interface, at least one subject;

displaying, on said user interface, a location for at least one user to enter at least one problem related to said at least one subject, said at least one problem comprising a medical problem;

in response to entry of said at least one problem through said user interface, automatically acquiring factors related to said at least one problem from a medical record associated with said subject, said factors being categorized into dimensions of evidence, said dimensions of evidence comprising symptoms, family history and demographics, said record being stored on at least one computerized storage medium, and said automatically acquiring being performed using at least one computerized device operatively connected to said user interface and said at least one computerized storage medium;

automatically generating questions related to said at least one problem and said factors using said at least one computerized device;

receiving an indication of ones of said factors that should be ignored and ones of said factors that should be considered from said at least one user through said user interface;

in response to said questions, automatically generating answers to said questions by referring to said factors that should be considered and to sources, said sources being accessible by said at least one computerized device and said generating of said answers being performed using said at least one computerized device;

for each dimension of evidence, automatically calculating confidence measures of each of said answers using said at least one computerized device;

combining said confidence measures of each of said answers for each dimension of evidence in order to calculate final confidence measures of each of said answers, said combining being performed using said at least one computerized device;

generating a comparative evidence profile using said at least one computerized device and displaying said answers and said final confidence measures for each of said answers on said user interface with said comparative evidence profile, said comparative evidence profile comprising a chart that provides visual indications on said user interface of relative contributions of each of said dimensions of evidence to each of said final confidence measures of each of said answers and that comprises selectable features;

displaying, on said user interface with said answers, said final confidence measures and said comparative evidence profile, details of said sources and said factors used to generate said said answers; and

,

upon receiving a selection of one of said selectable features of said chart through said user interface, automatically filtering said details of said sources and said factors so as to limit display of said details, said automatically filtering being performed by said at least one computerized device.

8. The method according to claim 7, further comprising maintaining a history of said questions, said answers, and said final confidence measures of each of said answers.

9. The method according to claim 7, said sources comprising first sources stored on said at least one computerized storage medium and second sources accessible online and said details of said sources comprising annotations to said sources.

10. The method according to claim 7, further comprising displaying how said sources contributed to said confidence measures of said answers on said user interface.

11. The method according to claim 7, said at least one problem comprising at least one question.

12. The method according to claim 7,

said chart comprising a bar chart comprising groups of bars,

each group of bars being associated with a specific dimension of evidence and being selectable to filter said details of said sources and said factors so as to limit displayed details of said sources and said factors to only those associated with said specific dimension of evidence, and,

each bar within said group being associated with a specific answer and being selectable to further filter said details of said source and said factors so as to further limit said displayed details of said sources and said factors to only those associated with both said specific answer and said specific dimension of evidence.

13. A method comprising:

displaying, on a user interface, at least one subject;

displaying, on said user interface, a location for at least one user to enter at least one problem related to said subject, said at least one problem comprising a medical problem;

in response to entry of said at least one problem through said user interface, automatically acquiring factors related to said at least one problem from a medical record associated with said subject, said factors being categorized into dimensions of evidence, said dimensions of evidence comprising symptoms, family history and demographics, said record being stored on at least one computerized storage medium, and said automatically acquiring being performed using at least one computerized device operatively connected to said user interface and said at least one computerized storage medium;

displaying said factors categorized into said dimensions of evidence on said user interface;

receiving an indication of ones of said factors that should be ignored and ones of said factors that should be considered from said at least one user through said user interface;

automatically generating questions related to said at least one problem and said factors using said at least one computerized device;

in response to said questions, automatically generating answers to said questions by referring to said factors that should be considered and to sources, said sources being accessible by said at least one computerized device and said generating being performed using said at least one computerized device;

for each dimension of evidence, automatically calculating confidence measures of each of said answers using said at least one computerized device;

combining said confidence measures of each of said answers for each dimension of evidence in order to calculate final confidence measures of each of said answers;

generating a comparative evidence profile using said at least one computerized device and displaying said answers and said final confidence measures for each of said answers on said user interface with said comparative evidence profile, said comparative evidence profile comprising a chart that provides visual indications on said user interface of relative contributions of each of said dimensions of evidence to each of said final confidence measures of each of said answers and that has selectable features;

displaying, on said user interface with said answers, said final confidence measures and said comparative evidence profile, details of said sources and said factors used to generate said answers;

upon receiving a selection of one of said selectable features of said chart through said user interface, automatically filtering said details of said sources and said factors so as to limit display of said details, said automatically filtering being performed using said at least one computerized device;

receiving at least one of updated factors and updated information from said sources using said at least one computerized device;

automatically generating at least one updated answer based on said updated factors and updated information from said sources using said at least one computerized device;

automatically comparing said answers to said at least one updated answer to produce an update severity score for each question using said at least one computerized device, said update severity score being a measure of how much a set of answers for said question changed; and

displaying said update severity score for each question on said user interface.

14. The method according to claim 13, further comprising maintaining a history of said questions, said answers, and said final confidence measures.

15. The method according to claim 13, said sources comprising first sources stored on said at least one computerized storage medium and second sources accessible online and said details of said sources comprising annotations to said sources.

16. The method according to claim 13, further comprising displaying how said sources contributed to said confidence measures of said answers on said user interface.

17. The method according to claim 13, said at least one problem comprising at least one question.

18. The method according to claim 13, said chart comprising a bar chart comprising groups of bars,

each group of bars being associated with a specific dimension of evidence and being selectable to filter said details of said sources and said factors so as to limit displayed details of said sources and said factors to only those associated with said specific dimension of evidence, and,

each bar within said group being associated with a specific answer and being selectable to further filter said details of said source and said factors so as to further limit said displayed details of said sources and said factors to only those associated with both said specific answer and said specific dimension of evidence.

19. A non-transitory computer readable storage medium readable by at least one computerized device, said non-transitory computer readable storage medium storing instructions executable by said at least one computerized device to perform a method comprising:

displaying on a user interface at least one subject;

displaying on said user interface a location for at least one user to enter at least one problem related to said subject;

in response to entry of said at least one problem through said user interface, automatically acquiring factors related to said at least one problem from a record associated with said subject, said factors being categorized into dimensions of evidence;

automatically generating questions related to said at least one problem and said factors;

in response to said questions, automatically generating answers to said questions by referring to sources;

for each dimension of evidence, automatically calculating confidence measures of each of said answers;

combining said confidence measures of each of said answers for each dimension of evidence in order to calculate final confidence measures of each of said answers;

generating a comparative evidence profile and displaying said answers and said final confidence measures for each of said answers with said comparative evidence profile on said user interface, said comparative evidence profile comprising a chart that provides visual indications of relative contributions of each of said dimensions of evidence to each of said final confidence measures of each of said answers and that has selectable features;

displaying, on said user interface with said answers, said final confidence measures and said comparative evidence profile, details of said sources and said factors used to generate said;

and, upon receiving a selection of one of said selectable features of said chart through said user interface, automatically filtering said details of said sources and said factors so as to limit display of said details.

20. The non-transitory computer readable storage medium according to claim 19, said method further comprising maintaining a history of said questions, said answers, and said final confidence measures.

21. The non-transitory computer readable storage medium according to claim 19, said sources comprising first sources stored on said at least one computerized storage medium and second sources accessible online and said details of said sources comprising annotations to said sources.

22. The non-transitory computer readable storage medium according to claim 19, further comprising displaying how said sources contributed to said confidence measures of said answers.

23. The non-transitory computer readable storage medium according to claim 19, said at least one problem comprising at least one question.

24. The non-transitory computer readable storage medium according to claim 19,

said chart comprising a bar chart comprising groups of bars,

each group of bars being associated with a specific dimension of evidence and being selectable to filter said details of said sources and said factors so as to limit displayed details of said sources and said factors to only those associated with said specific dimension of evidence, and,

each bar within said group being associated with a specific answer and being selectable to further filter said details of said source and said factors so as to further limit said displayed details of said sources and said factors to only those associated with both said specific answer and said specific dimension of evidence.