System and method for the automated discovery of unknown unknowns

• US 20060112048A1
• Filed: 10/29/2004
• Published: 05/25/2006
• Est. Priority Date: 10/29/2004
• Status: Abandoned Application

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Inventors: Ellis, Dennis Regan, Talbot, Patrick James, Sanders, J. Ryan
• Application Number:
  US10/977,588
• Publication Number:
  US 20060112048A1
• Time in Patent Office: Days
• US Class Current: 706/46
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

Systems and methods are provided for determining unknown unknowns for at least one assisted decision making system. An assisted decision making system has an associated set of at least one story of interest. The system further comprises a plurality of inferencing algorithms. A given inferencing algorithm is operative to generate story fragments from data stored within an associated knowledge base. A first arbitrator is associated with the first assisted decision making system. The arbitrator accepts a story fragment from the plurality of inferencing algorithms if the story fragment is sufficiently related to a story of interest from the set of at least one story of interest.

Claims

1. A system for determining unknown unknowns for at least one assisted decision making system comprising:

a first assisted decision making system having an associated first set of at least one story of interest;

a plurality of inferencing algorithms, a given inferencing algorithm being operative to generate story fragments from a data stored within an associated knowledge base; and

a first arbitrator, associated with the first assisted decision making system, that accepts a story fragment from the plurality of inferencing algorithms if the story fragment is sufficiently related to a story of interest from the first set of at least one story of interest.

2. The system of claim 1, further comprising:

a second assisted decision making system having an associated second set of at least one story of interest; and

a second arbitrator, associated with the second assisted decision making system, that accepts a story fragment from the plurality of inferencing algorithms if the story fragment is sufficiently related to a story of interest from the second set of at least one story of interest.

3. The system of claim 1, the knowledge base comprising a plurality of executable stories.

4. The system of claim 3, at least one of the plurality of executable stories comprising a Dempster-Shafer belief network.

5. The system of claim 1, the first assisted decision making system having an associated user interface, the first arbitrator providing a story fragment to a human analyst through the user interface if the story fragment is sufficiently related to a story of interest from the first set of at least one story of interest.

6. The system of claim 1, the system further comprising a fusion engine that is operative to fuse a story fragment to a story of interest and mathematically reconcile the resulting union.

7. The system of claim 1, further comprising at least one information extractor that reduces evidence from an external source into an evidence template that is appropriately formatted for the knowledge base.

8. The system of claim 7, further comprising at least one evidence classifier that evaluates the evidence template to associate the template with a hypothesis within the knowledge base.

9. The system of claim 1, the plurality of inferencing algorithms comprising an inductive reasoner that computes changes in rules based on new evidence within the knowledge base.

10. The system of claim 1, the plurality of inferencing algorithms comprising a unsupervised clustering algorithm that clusters evidence in the knowledge base according to associated characteristics of the evidence.

11. The system of claim 1, the plurality of inferencing algorithms comprising an evidential reasoner that reviews new stories in the knowledge base to generate story fragments and evidence that are related to but not represented in a story of interest.

12. The system of claim 1, the plurality of inferencing algorithms comprising an abductive reasoner that provides a best explanation for new data using positive and negative examples from evidence drawn from past stories related to a story of interest.

13. The system of claim 1, the plurality of inferencing algorithms comprising an analogical reasoner that determines similarities and differences between a present instantiation of a story of interest and past successful stories in the knowledge base.

14. The system of claim 1, the plurality of inferencing algorithms comprising a link analysis component that computes link values between hypotheses based on the characteristics of the hypotheses.

15. The system of claim 1, wherein the arbitrator is operative to combine a first story fragment from a first decision algorithm and a second story fragment from a second decision algorithm to produce a combined story fragment.

16. A method for discovering unknown unknowns for an assisted decision making system comprising:

generating a story fragment at a inferencing algorithm;

evaluating the story fragment to determine if it is sufficiently related to a story of interest associated with the assisted decision making system; and

providing the story fragment to a user interface if it is sufficiently related to the story of interest.

17. The method of claim 16, further comprising fusing the story fragment and the story of interest according to input from the user interface.

18. The method of claim 16, further comprising evaluating the story fragment at the inferencing algorithm according to an internal threshold of the inferencing algorithm.

19. A system for discovering unknown unknowns comprising:

means for decision making, having at least one associated story of interest;

means for generating a story fragment;

means for evaluating the story fragment to determine its relation to the at least one story of interest; and

means for fusing the story fragment with a story of interest from the at least one story of interest if the story fragment is related to the story of interest.

20. The system of claim 19, further comprising means for receiving user input concerning the relationship of the story fragment to the story of interest.

21. The system of claim 19, the means for evaluating including means for combining a first story fragment and a second story fragment.

22. A computer readable medium comprising

a user interface, associated with the decision making system;

a inferencing algorithm that generates a story fragment; and

an arbitrator that evaluates the story fragment to determine if it is sufficiently related to a story of interest associated with the assisted decision making system and provides the story fragment to the user interface if it is sufficiently related to the story of interest.

23. The computer readable medium of claim 22, further comprising a fusion engine that fuses the story fragment and the story of interest according to input from the user interface.

24. The computer readable medium of claim 22, further comprising a knowledge base comprising a plurality of executable stories, the inferencing algorithm generating the story fragment from data within the knowledge base.

25. The computer readable medium of claim 24, further comprising at least one information extractor that reduces evidence from an external source into an evidence template that is appropriately formatted for the knowledge base