Uncertainty management in a decision-making system

• US 7,606,784 B2
• Filed: 08/02/2005
• Issued: 10/20/2009
• Est. Priority Date: 08/02/2005
• Status: Active Grant

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Northrop Grumman Corporation
• Inventors: Nixon, Mark Robert, Mathias, Keith Eugene, Talbot, Patrick James
• Primary Examiner(s): Vincent; David R
• Assistant Examiner(s): Bharadwaj; Kalpana
• Application Number:
  US11/196,647
• Publication Number:
  US 20070094216A1
• Time in Patent Office: 1,540 Days
• Field of Search: 706/52, 706/62, 709/223, 709/225, 709/228, 714/43
• US Class Current: 706/52
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

Systems and methods are provided for constructing and evaluating a story of interest. The system includes a plurality of decision algorithms. Each decision algorithm is operative to quantify at least one category of uncertainty associated with the story of interest as a set of at least one uncertainty parameter. An uncertainty management component is operative to reconcile the sets of uncertainty parameters from the plurality of decision algorithms as to produce a global uncertainty parameter for each of the plurality of uncertainty categories for the story of interest.

Claims

1. A computer implemented assisted decision-making system for constructing and evaluating a story of interest comprising:

a plurality of decision algorithms, each decision algorithm quantifying at least one category of uncertainty associated with the story of interest as at least one uncertainty parameter, the plurality of decision algorithms comprising an information extraction component that reduces evidence sources into a desired format; and

an uncertainty management component that reconciles the uncertainty parameters from the plurality of decision algorithms as to produce a global uncertainty parameter for each of the plurality of uncertainty categories for the story of interest, and provides a unified uncertainty assessment by combining the global uncertainty parameters for each of the plurality of uncertainty categories for the story of interest.

2. The system of claim 1, the at least one category of uncertainty quantified by the information extraction component including an uncertainty due to vague evidence, the information extraction component quantifying the uncertainty according to hedge words found within an associated evidence source.

3. The system of claim 1, the at least one category of uncertainty quantified by the information extraction component including an uncertainty due to evidence and hypotheses that are known to be missing, the information extraction component quantifying the uncertainty according to missing entries in an evidence template associated with the desired format.

4. The system of claim 1, the at least one category of uncertainty quantified by the information extraction component including an uncertainty due to ambiguous evidence, the information extraction component utilizing a classification system to reduce the evidence sources and quantifying the uncertainty according to a confidence value generated by the classification system.

5. The system of claim 1, the at least one category of uncertainty quantified by the information extraction component including uncertainty due to false evidence, the information extraction component quantifying the uncertainty according to at least one of the historical performance of a provider of the evidence or a comparison of multiple sources to resolve and discount false evidence.

6. The system of claim 1, the at least one category of uncertainty quantified by the information extraction component including an uncertainty due to obsolete evidence, the information extraction component quantifying the uncertainty according to an associated age of the evidence and at least one obsolescence method that defines a rate at which confidence in a given item of evidence decays over time.

7. The system of claim 1, the at least one category of uncertainty quantified by the information extraction component including uncertainty due to measurement error.

8. The system of claim 1, the at least one category of uncertainty quantified by the information extraction component including uncertainty due to random error.

9. The system of claim 1, further comprising a knowledge base comprising a plurality of executable stories, a given executable story comprising a belief network containing a plurality of associated hypothesis and evidence associated with the at least one hypothesis.

10. The system of claim 9, the system further comprising at least one inferencing algorithm that examines existing hypothesis within executable stories stored in the knowledge base to determine one or more appropriate hypotheses for a body of evidence.

11. The system of claim 10, the at least one category of uncertainty quantified by the at least one inferencing algorithm including an uncertainty due to unknown missing information.

12. A method for managing uncertainty in an assisted decision-making system:

constructing a story of interest, the story of interest comprising a structured argument having a plurality of associated elements, at least one link associated with the plurality of elements, evidence supporting the plurality of hypotheses, and a plurality of evidence sources;

searching an external database for at least one phrase associated with a given element associated with the story of interest;

determining an uncertainty parameter that represents an uncertainty due to misunderstandings of the element based on the results of the search;

evaluating at least two categories of uncertainty associated with the story of interest to produce respective global uncertainty parameters, wherein evaluating a first category of uncertainty comprises reconciling uncertainty parameters representing a plurality of hypotheses to provide a first global uncertainty parameter representing uncertainty due to misunderstood elements; and

displaying the global uncertainty parameters to a decision-maker.

13. The method of claim 12, further comprising reconciling the global uncertainty parameters as to produce a composite uncertainty parameter for the story of interest.

14. The method of claim 12, further comprising:

conducting a sensitivity analysis of the story of interest to evaluate the likelihood that the story can produce a satisfactory level of belief associated with a main hypothesis associated with the story of interest; and

producing a global uncertainty parameter representing the likelihood that insufficient evidence exists to support the main hypothesis associated with the story of interest from sensitivity analysis.

15. The method of claim 12, wherein displaying the global uncertainty parameters to the decision-maker includes displaying uncertainty to the decision-maker via a graphical representation.

16. A computer readable medium comprising:

a plurality of decision algorithms, each decision algorithm quantifying at least one category of uncertainty associated with the story of interest as at least one uncertainty parameter, the plurality of decision algorithms comprising an information extraction component that reduces evidence sources into a desired format; and

an uncertainty management component that reconciles the uncertainty parameters from the plurality of decision algorithms so as to produce a total uncertainty for each of the plurality of uncertainty categories for the story of interest and that combines the total uncertainty for each of the plurality of uncertainty categories for the story of interest to provide a unified uncertainty assessment.

17. The computer readable medium of claim 16, comprising a fusion engine that mathematically reconciles the story of interest when data within the story of interest is altered, the fusion engine determines an uncertainty parameter representing conflicting evidence for at least one hypothesis comprising the story of interest.

18. The computer readable medium of claim 16, further comprising a simulation component that perturbs at least one value within the story of interest according to a model, the simulation component determines an uncertainty parameter representing chaotic uncertainty by conducting a sensitivity analysis of the effects of the perturbation of the at least one value on a main hypothesis within the story of interest.