System and method for deriving a hierarchical event based database optimized for analysis of criminal and security information

• US 7,752,154 B2
• Filed: 02/26/2007
• Issued: 07/06/2010
• Est. Priority Date: 02/26/2007
• Status: Expired due to Fees

• Current Assignee: ServiceNow Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Hennessy, Richard A., Kraemer, James R., Friedlander, Robert R.
• Primary Examiner(s): Jalil; Neveen Abel
• Assistant Examiner(s): KINSAUL, DANIEL W
• Application Number:
  US11/678,959
• Publication Number:
  US 20080208901A1
• Time in Patent Office: 1,226 Days
• Field of Search: 706 15- 62
• US Class Current: 706/47
• CPC Class Codes: G06F 16/2462 Approximate or statistical queries
• CPC Class Codes: G06F 2216/03 Data mining

Abstract

A computer implemented method, apparatus, and computer usable program code for inferring a probability of a first inference absent from a database at which a query regarding the inference is received. Each datum of the database is conformed to the dimensions of the database. Each datum of the plurality of data has associated metadata and an associated key. The associated metadata includes data regarding cohorts associated with the corresponding datum, data regarding hierarchies associated with the corresponding datum, data regarding a corresponding source of the datum, and data regarding probabilities associated with integrity, reliability, and importance of each associated datum. The query is used as a frame of reference for the search. The database returns a probability of the correctness of the first inference based on the query and on the data.

Claims

1. A computer implemented method for inferring a probability of a first inference, the computer implemented method comprising:

receiving a query at a database regarding a fact, wherein the first inference is absent from the database, wherein the database comprises a plurality of divergent data, wherein the plurality of divergent data includes a plurality of cohort data, wherein each datum of the plurality of divergent data in the database is conformed to dimensions of the database, wherein each datum of the plurality of divergent data has associated metadata and an associated key, wherein the associated metadata comprises data regarding cohorts associated with the corresponding datum, data regarding hierarchies associated with the corresponding datum, data regarding a corresponding source of the datum, and data regarding probabilities associated with integrity, reliability, and importance of each associated datum, wherein the cohorts (i) are a group of associated individuals or objects sharing a common characteristic, and (ii) are automatically generated using data stored at an atomic level;

establishing the fact as a frame of reference for the query;

applying a first set of rules to the query, wherein the first set of rules are determined for the query according to a second set of rules, wherein the frame of reference serves as an anchor for generating associations among the plurality of cohort data and is used to determine rules for limiting the plurality of divergent data that is searched, wherein the first set of rules determine how the plurality of divergent data are to be compared to the fact, and wherein the first set of rules determine a search space for the query;

executing the query to create the first inference and the probability of the first inference, wherein the probability of the first inference is determined from comparing the plurality of divergent data according to the first set of rules; and

storing the probability of the first inference.

2. The computer implemented method of claim 1 wherein the method repeats until a threshold is reached.

3. The computer implemented method of claim 1 wherein the second set of rules is selected from the group consisting of a heuristic set of rules, a statistical set of rules, a neural network, rules created by user input, and combinations thereof.

4. The computer implemented method of claim 1 wherein the database accesses a library when the query is executed, and wherein the library includes ontology information and taxonomy information regarding each associated metadata.

5. The computer implemented method of claim 4 wherein the library is part of the database.

6. The computer implemented method of claim 1 further comprising:

entering the first inference into the database as a first additional datum;

entering the probability of the first inference into the database as a second additional datum;

entering a categorization of the first inference into the database as a third additional datum; and

entering a categorization of the probability of the first inference into the database as a fourth additional datum.

7. The computer implemented method of claim 6 further comprising:

executing a second query to create a second inference and a second probability of the second inference, wherein the second probability of the second inference is determined from comparing the plurality of divergent data according to a third set of rules, and wherein the third set of rules are determined for the second query according to a fourth set of rules.

8. The computer implemented method of claim 1 further comprising:

responsive to receiving an additional datum in the database, establishing a second query;

applying a third set of rules to the query, wherein the third set of rules are determined for the second query according to a fourth set of rules, and wherein the third set of rules determine how the plurality of divergent data are to be compared to the fact; and

executing the second query to create a second inference and a second probability of the second inference, wherein the second probability of the second inference is determined from comparing the plurality of divergent data according to the third set of rules.

9. The computer implemented method of claim 1 wherein the first inference comprises at least one of an event, a fact, a pattern of events, a group of events, and a group of facts.

10. The computer implemented method of claim 1 wherein the metadata further comprises user-defined data.

11. The computer implemented method of claim 1 further comprising:

identifying an additional cohort associated with the fact.

12. The computer implemented method of claim 11 wherein the identifying an additional cohort associated with the fact is performed using at least one of a clustering algorithm, Baysian statistics, user defined rules, and a combination thereof.

13. The computer implemented method of claim 1 wherein the set of rules includes rules for adjusting the probability of the inference based on background data.

14. The computer implemented method of claim 1 wherein the data regarding hierarchies further comprises how the corresponding datum is categorized with other data in the plurality of divergent data.

15. The computer implemented method of claim 1 further comprising:

importing additional data into the plurality of divergent data, wherein the additional data initially is not associated with metadata and the additional data does not conform to the dimensions of the database;

conforming the additional data to the dimensions of the database; and

associating the metadata and a key with each datum of the additional data.

16. The computer implemented method of claim 15 wherein the additional data is imported according to a technique selected from the group consisting of federation and extraction, transformation, and loading.

17. A computer implemented method for building a database capable of inferring a probability of a first inference, the computer implemented method comprising:

establishing a database structure, wherein the database structure is adapted to receive a plurality of divergent data, wherein in the database the plurality of divergent data includes a plurality of cohort data, wherein the database is adapted such that each datum of the plurality of divergent data in the database is conformed to dimensions of the database, wherein the database is further adapted such that each datum of the plurality of divergent data has associated metadata and an associated key, wherein the associated metadata comprises data regarding cohorts associated with the corresponding datum, data regarding hierarchies associated with the corresponding datum, data regarding a corresponding source of the datum, and data regarding probabilities associated with integrity, reliability, and importance of each associated datum, wherein the cohorts (i) are a group of associated individuals or objects sharing a common characteristic, and (ii) are automatically generated using data stored at an atomic level;

establishing a first set of rules for the database structure, the first set of rules comprising rules for determining a second set of rules to be applied to a query submitted to the database, wherein the second set of rules determine how the plurality of divergent data are to be compared to a fact submitted with the query, wherein the fact serves as a frame of reference when executing the query, wherein the frame of reference serves as an anchor for generating associations among the plurality of cohort data and is used to determine rules for limiting the plurality of divergent data that is searched, wherein the second set of rules determine a search space for the query, and wherein the second set of rules are adapted to create the first inference and the probability of the first inference, wherein the probability of the first inference is determined from comparing the plurality of divergent data according to the second set of rules using the fact as the frame of reference;

receiving the plurality of divergent data in the database;

conforming the plurality of divergent data to the dimensions of the database to form a plurality of conformed data;

associating the metadata and a key with each datum in the plurality of conformed data; and

storing the database structure.

18. The computer implemented method of claim 17 further comprising:

receiving the plurality of divergent data by a method selected from the group consisting of federating the plurality of divergent data and extracting, transforming, and loading the plurality of divergent data.

19. A computer program product stored in a computer readable storage medium and operable for inferring a probability of a first inference when executed by a data processing system, the computer program product comprising:

a database comprising a plurality of divergent data stored in a data structure on the computer readable storage medium, wherein the plurality of divergent data includes a plurality of cohort data, wherein each datum of the plurality of divergent data in the database is conformed to dimensions of the database, wherein each datum of the plurality of divergent data has associated metadata and an associated key, wherein the associated metadata comprises data regarding cohorts associated with the corresponding datum, data regarding hierarchies associated with the corresponding datum, data regarding a corresponding source of the datum, and data regarding probabilities associated with integrity, reliability, and importance of each associated datum, wherein the cohorts (i) are a group of associated individuals or objects sharing a common characteristic, and (ii) are automatically generated using data stored at an atomic level;

computer usable program code for establishing a fact, received in a query, as a frame of reference for the query;

computer usable program code for applying a first set of rules to the query, wherein the first set of rules are determined for the query according to a second set of rules, wherein the frame of reference serves as an anchor for generating associations among the plurality of cohort data and is used to determine rules for limiting the plurality of divergent data that is searched, wherein the first set of rules determine how the plurality of divergent data are to be compared to the fact, and wherein the first set of rules determine a search space for the query;

computer usable program code for executing the query to create a first inference and a probability of the first inference, wherein the probability of the first inference is determined from comparing the plurality of divergent data according to the first set of rules; and

computer usable program code for storing the probability of the first inference.

20. The computer program product of claim 19 further comprising:

a library stored in the computer readable storage type medium, wherein the library includes ontology information and taxonomy information regarding each associated metadata.