Explanation generation system for a diagnosis support tool employing an inference system

• US 6,601,055 B1
• Filed: 08/02/1999
• Issued: 07/29/2003
• Est. Priority Date: 12/27/1996
• Status: Expired due to Fees

• Current Assignee: Linda M. Roberts
• Original Assignee: Linda M. Roberts
• Inventors: Roberts, Linda M.
• Primary Examiner(s): STARKS, WILBERT L
• Application Number:
  US09/366,482
• Time in Patent Office: 1,457 Days
• Field of Search: 706/45, 705/3, 704/257, 600/300
• US Class Current: 706/45
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G16H 10/20 for electronic clinical trials or questionnaires
• CPC Class Codes: G16H 15/00 ICT specially adapted for medical reports, e.g. generation or transmission thereof
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

An interactive multimodal explanation generation system for a computer-aided decision support tool employing an inference system is disclosed. The explanation generation system includes a user interface enhanced with a reasoning component that allows a user to ask, using mouse clicks and natural language text, the system questions about her health and about how the system is constructed. The explanation generation system produces interactive multimodal explanations (using textual displays, graphics, animation, or a combination of these) for the results generated by the inference system in the diagnostic support tool.

Claims

1. A computer-implemented method of providing medical decision support comprising the steps of

(A) initializing an inference engine;

(B) initializing a semantic network structure;

(C) initializing a discourse structure;

(D) initializing a parser;

(E) reading a lexicon file;

(F) waiting for user input;

(G) receiving a multimodal input from a user;

(H) determining a type of input received, wherein it is determined that said type of input is evidence; and

(I) processing said input based upon said determined type, wherein said processing step comprises the steps of

(i) storing said evidence in an inference engine evidence vector;

(ii) storing said evidence in a semantic network structure;

(iii) determining a sensitivity based upon said evidence; and

(iv) storing said determined sensitivity in said semantic network structure.

2. A computer-implemented method of providing computer-aided medical decision support comprising the steps of

(A) initializing an inference engine;

(B) initializing a semantic network structure;

(C) initializing a discourse structure;

(D) initializing a parser;

(E) reading a lexicon file;

(F) waiting for user input;

(G) receiving a multimodal input from a user;

(H) determining a type of input received, wherein it is determined that said type of input is a Professor question; and

(I) processing said input based upon said determined type, wherein said processing step comprises the steps of

(i) determining which node is under consideration;

(ii) interrogating said semantic network structure for node information;

(iii) preparing said information for display; and

(iv) constructing display specifications for displaying said information.

3. A computer-implemented method of providing computer-aided medical decision support comprising the steps of

(A) initializing an inference engine;

(B) initializing a semantic network structure;

(C) initializing a discourse structure;

(D) initializing a parser;

(E) reading a lexicon file;

(F) waiting for user input;

(G) receiving a multimodal input from a user;

(H) determining a type of input received, wherein it is determined that said type of input is a user question; and

(I) processing said input based upon said determined type, wherein said processing step determines that said user question is not understood, and said processing step comprises the additional steps of

(i) interrogating said discourse structure for context;

(ii) formulating a clarifying question; and

(iii) displaying said clarifying question.

4. A computer-implemented method of providing computer-aided medical decision support comprising the steps of

(A) initializing an inference engine;

(B) initializing a semantic network structure;

(C) initializing a discourse structure;

(D) initializing a parser;

(E) reading a lexicon file;

(F) waiting for user input;

(G) receiving a multimodal input from a user;

(H) determining a type of input received, wherein it is determined that said type of input is a user question; and

(I) processing said input based upon said determined type, wherein said processing step determines that said user question is understood, and said processing step comprises the additional steps of

(i) generating an abstract concept;

(ii) storing said abstract concept in a discourse structure; and

(iii) determining an action to take based upon said abstract concept.

5. A computer aided medical diagnostic decision support system comprising

an interactive multimodal explanation generation system, wherein said interactive multimodal explanation generation system comprises

a multimodal interactive user interface for receiving multimodal inputs from a user and for presenting multimodal outputs to the user;

a knowledge representation module in communication with said multimodal interactive user interface and with said Bayesian network inference engine module; and

a multimodal discourse module in communication with said knowledge representation module and with said multimodal interactive user interface,

wherein said multimodal interactive user interface comprises an input module and an output module, said input module in communication with said knowledge representation module, and said output module in communication with said multimodal discourse module, and wherein said knowledge representation module further comprises

a domain-specific lexicon;

a chart parser;

a semantic network structure; and

processing logic to process the flow of data and commands between said input module, said multimodal discourse module, and said Bayesian network inference engine module.

6. A method of generating interactive multimodal explanations in a medical diagnostic support tool using a Bayesian network inference engine, said method comprising the steps of

(A) waiting for an utterance from a user;

(B) constructing an input object from the utterance, the input object identifying a modality, a sequence, and a content of the utterance;

(C) inserting the input object into an input stream;

(D) sending the input stream to a knowledge representation module; and

(E) parsing and encoding the input object in the knowledge representation module into an abstract statement, defining a statement type, defining a statement origin, defining a statement modality, and defining a statement context for each input object.

7. A method of generating interactive multimodal medical explanations during a dialog between system participants, including a decision support tool and a user, the decision support tool using a Bayesian network inference engine, said method comprising the steps of

(A) receiving multimodal inputs from a user;

(B) synthesizing said multimodal inputs into a single sequenced stream of events;

(C) communicating said sequenced stream of events to a knowledge representation module;

(D) generating, within said knowledge representation module, an abstract statement from said sequenced stream of events, wherein step (D) further comprises the steps of

(i) reading a lexicon file, comprising lexicon words and corresponding lexicon semantic word types;

(ii) storing the lexicon words and corresponding lexicon semantic word types in a lexicon structure;

(iii) parsing said sequenced stream of events into noun phrases and verb phrases;

(iv) assigning a semantic type to each said parsed phrase;

(v) storing said parsed phrases and their said assigned semantic phrase types in a chart data structure;

(vi) comparing each said stored parsed phrase and its said assigned semantic phrase type from the chart data structure to the lexicon words and corresponding lexicon semantic word types stored in the lexicon structure trying to match patterns by trying to match general patterns followed by trying to match specific patterns; and

(vii) generating said abstract statement for matched patterns; and

(E) storing said abstract statement into an explicit discourse history structure comprising part of a multimodal discourse module.

8. A method of generating interactive multimodal medical explanations during a dialog between system participants, including a decision support tool and a user, the decision support tool using a Bayesian network inference engine, said method comprising the steps of

(A) receiving multimodal inputs from a user;

(B) synthesizing said multimodal inputs into a single sequenced stream of events;

(C) communicating said sequenced stream of events to a knowledge representation module;

(D) generating, within said knowledge representation module, an abstract statement from said sequenced stream of events, wherein step (D) further comprises the steps of

(i) parsing said sequenced stream of events, wherein said parsing step comprises using a bottom-up parsing strategy; and

(ii) encoding said parsed stream of events into said abstract statement, wherein said encoding step comprises generating a statement type, a statement origin, a statement modality, and a statement context for each abstract statement; and

(E) storing said abstract statement into an explicit discourse history structure comprising part of a multimodal discourse module.

9. A method of generating interactive multimodal medical explanations during a dialog between system participants, including a decision support tool and a user, the decision support tool using a Bayesian network inference engine, said method comprising the steps of

(A) receiving multimodal inputs from a user;

(B) synthesizing said multimodal inputs into a single sequenced stream of events;

(C) communicating said sequenced stream of events to a knowledge representation module;

(D) generating, within said knowledge representation module, an abstract statement from said sequenced stream of events, wherein step (D) further comprises the steps of

(i) parsing said sequenced stream of events; and

(ii) encoding said parsed stream of events into said abstract statement;

(E) storing said abstract statement into an explicit discourse history structure comprising part of a multimodal discourse module;

(F) using said multimodal discourse module to mediate an on-going dialog between the system participants based upon said discourse history structure;

(G) sending inquiries from said multimodal discourse module to said knowledge representation module;

(H) processing inquiries from said multimodal discourse module in said knowledge representation module;

(I) requesting, via said knowledge representation module, statistical processing by said Bayesian network inference engine;

(J) generating, within said knowledge representation module, said abstract statement based upon a result of said statistical processing;

(K) passing said abstract statement to said multimodal discourse module;

(L) determining, within said multimodal discourse module, a presentation strategy for presenting said result;

(M) communicating said presentation strategy to an output module; and

(N) presenting said result to the user via said output module.

10. The method of claim 9, wherein step (F) further comprises the steps of generating a plurality of dialog entries comprising command type information, modality type information, speaker type information, and context type information.